COLMAP settings for metrics

[kxhit]

Hi @liuyuan-pal

What's the COLMAP settings for getting the metrics reported in paper Tab.1. It seems the default setting will fail. I can't find any details in the paper. Thanks!

[liuyuan-pal]

You may refer to this file. https://connecthkuhk-my.sharepoint.com/:u:/g/personal/yuanly_connect_hku_hk/EXxOnqGqvGRNsDKLC7RcNmMBmfZ9Fd56ihobg1jKNnCJSg?e=9Kowh5
It may fail due to the large viewpoint difference but works in general cases.

[Wi-sc]

Hi @liuyuan-pal, I notice you use the pose from external dependency.

from eval_utils import H, W, POSES, NUM_IMAGES, K

Could you provide an example about it?

[liuyuan-pal]

Hi, these are the predefined poses and intrinsics in SyncDreamer.

SyncDreamer/ldm/models/diffusion/sync_dreamer.py

Line 274 in 1fa5ecf

K, azs, _, _, poses = read_pickle(f'meta_info/camera-{self.view_num}.pkl')

[Wi-sc]

Hi @liuyuan-pal, as you suggest, I have tried to replace the code by predefined poses in the pickle file.

H, W = 256, 256
NUM_IMAGES = 16
with open(f'meta_info/camera-16.pkl', 'rb') as f:
     K, _, _, _, POSES = pickle.load(f)

But I cannot output reasonable point clouds for some cases. Colmap fails and cannot find any matching points, even though the images look good. Is it expected?

Here are camera.txt and images.txt from colmap.

# camera.txt
1 PINHOLE 256 256 280.000 280.000 128.000 128.000
2 PINHOLE 256 256 280.000 280.000 128.000 128.000
3 PINHOLE 256 256 280.000 280.000 128.000 128.000
4 PINHOLE 256 256 280.000 280.000 128.000 128.000
5 PINHOLE 256 256 280.000 280.000 128.000 128.000
6 PINHOLE 256 256 280.000 280.000 128.000 128.000
7 PINHOLE 256 256 280.000 280.000 128.000 128.000
8 PINHOLE 256 256 280.000 280.000 128.000 128.000
9 PINHOLE 256 256 280.000 280.000 128.000 128.000
10 PINHOLE 256 256 280.000 280.000 128.000 128.000
11 PINHOLE 256 256 280.000 280.000 128.000 128.000
12 PINHOLE 256 256 280.000 280.000 128.000 128.000
13 PINHOLE 256 256 280.000 280.000 128.000 128.000
14 PINHOLE 256 256 280.000 280.000 128.000 128.000
15 PINHOLE 256 256 280.000 280.000 128.000 128.000
16 PINHOLE 256 256 280.000 280.000 128.000 128.000

# images.txt
1 0.35355 0.61237 0.61237 -0.35355 -0.00000 0.00000 1.50000 1 000.png

2 0.27779 0.48114 0.72007 -0.41573 0.00000 0.00000 1.50000 2 001.png

3 0.19134 0.33141 0.80010 -0.46194 -0.00000 0.00000 1.50000 3 002.png

4 0.09755 0.16895 0.84938 -0.49039 -0.00000 0.00000 1.50000 4 003.png

5 0.00000 0.00000 -0.86603 0.50000 -0.00000 0.00000 1.50000 5 004.png

6 0.09755 0.16895 -0.84938 0.49039 0.00000 0.00000 1.50000 6 005.png

7 0.19134 0.33141 -0.80010 0.46194 0.00000 0.00000 1.50000 7 006.png

8 0.27779 0.48114 -0.72007 0.41573 -0.00000 0.00000 1.50000 8 007.png

9 0.35355 0.61237 -0.61237 0.35355 0.00000 0.00000 1.50000 9 008.png

10 0.41573 0.72007 -0.48114 0.27779 0.00000 0.00000 1.50000 10 009.png

11 0.46194 0.80010 -0.33141 0.19134 0.00000 0.00000 1.50000 11 010.png

12 0.49039 0.84938 -0.16895 0.09755 -0.00000 0.00000 1.50000 12 011.png

13 0.50000 0.86603 0.00000 -0.00000 0.00000 0.00000 1.50000 13 012.png

14 0.49039 0.84938 0.16895 -0.09755 0.00000 0.00000 1.50000 14 013.png

15 0.46194 0.80010 0.33141 -0.19134 0.00000 0.00000 1.50000 15 014.png

16 0.41573 0.72007 0.48114 -0.27779 0.00000 -0.00000 1.50000 16 015.png

Could you please help me check if the output is correct with your output? The camera.txt should contain the focal length and image size. The images.txt should be the quaternion. I suspect if I load it correctly. I would appreciate it if you can help me check it.

[liuyuan-pal]

Hi, the evaluation code has been released. You may refer to this code. COLMAP indeed fails sometimes due to sparse views so don't worry. :)