This repository is the official open-source implementation of the paper SpectralGV, RA-L 2023:
Spectral Geometric Verification: Re-Ranking Point Cloud Retrieval for Metric Localization
Kavisha Vidanapathirana, Peyman Moghadam, Sridha Sridharan, Clinton Fookes
IEEE Robotics and Automation Letters (RA-L), Volume 8, Issue 5, May 2023. arXiv, IEEEXplore
This repository contains the code for:
- A quick demo of SpectralGV (without the need for datasets or architecture dependencies).
- Integration of SpectralGV re-ranking with 3 open-source metric localization architectures (EgoNN, LCDNet and LoGG3D-Net).
- Evaluation of place recognition and metric localization with and without re-ranking on 5 open-source datasets (MulRan, Apollo-Southbay, KITTI, ALITA and KITTI-360).
SpectralGV is an efficient spectral method for geometric verification based re-ranking of point clouds. SpectralGV allows integration with any metric-localization architecture that extracts both local and global features for a given point cloud, without modifying the architectures and without further training of learning-based methods.
Set up base environment
- Create conda environment with python:
conda create --name sgv_env python=3.9.4
conda activate sgv_env
- Install PyTorch with suitable cudatoolkit version. See here:
pip3 install torch torchvision torchaudio
# Make sure the pytorch cuda version matches your output of 'nvcc --version'
- Install Open3d:
pip install open3d
- Test installation using:
python -c "import torch ; import open3d as o3d ; print(torch.cuda.is_available())"
Now you can run the quick demo:
Demo
This demo re-creates the results of Tab. 2 and Fig. 3 in our paper on the KITTI-360 09 dataset. It outputs results for place recognition both with and without re-ranking using SpectralGV, RANSAC-GV and alpha-QE.
- Download the demo data (~75 MB) from Dropbox here:
cd demo
wget --output-document demo_pickles.zip https://dl.dropboxusercontent.com/s/4elvyix9pp36469/demo_pickles.zip?dl=0
unzip demo_pickles.zip
Run the demo:
- SpectralGV
python demo_spectral_gv.py --n_topk 2
python demo_spectral_gv.py --n_topk 20
python demo_spectral_gv.py --n_topk 200
- RANSAC-GV
python demo_ransac_gv.py --n_topk 2
python demo_ransac_gv.py --n_topk 20
python demo_ransac_gv.py --n_topk 200
- alpha-QE
python demo_alpha_qe.py --n_topk 2
python demo_alpha_qe.py --n_topk 20
python demo_alpha_qe.py --n_topk 200
Observations:
- The 2 geometric verifciation methods (SpectralGV and RANSAC-GV) show increasing performance with increasing
n_topk
. - alpha-QE shows decreasing performance with increasing
n_topk
and is therefore not suitable for point cloud re-ranking. - Out of the 2 geometric verifciation methods, RANSAC-GV is inefficient at high
n_topk
values, thus limiting it's practical use. SpectralGV maintains almost constant runtime.
This project has been tested on a system with Ubuntu 22.04 and an RTX 2080. Set up the requirements as follows:
- Please set up the base conda environment and test the quick demo above.
- To recreate other results in the paper, the following dependencies are required:
Add dependencies for LoGG3D-Net:
- Install Torchpack:
pip install torchpack
- Install torchsparse-1.4.0
sudo apt-get install libsparsehash-dev
pip install --upgrade git+https://github.com/mit-han-lab/[email protected]
- Install mpi4py:
conda install mpi4py
conda install openmpi
- Test installation using:
python -c "import torch ; import torchsparse ; print('OK')"
Note: If stuck, see here for more details: https://github.com/csiro-robotics/LoGG3D-Net
Add dependencies for EgoNN:
- Install MinkowskiEngine 0.5.4
conda install openblas-devel -c anaconda
pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option="--blas_include_dirs=${CONDA_PREFIX}/include" --install-option="--blas=openblas"
- Install other dependencies:
pip install pytorch_metric_learning python-lzf wandb
- Test installation using:
python -c "import torch ; import MinkowskiEngine as ME ; print('OK')"
Note: If stuck, see here for more details: https://github.com/jac99/Egonn
Add dependencies for LCDNet:
Note: You will need to create a separate conda environment for LCDNet. See here for details: https://github.com/robot-learning-freiburg/LCDNet
This project supports the following 5 open-source LiDAR datasets:
MulRan:
We use the sequences Sejong and DCC.
- Download the MulRan dataset: ground truth data (*.csv) and LiDAR point clouds (Ouster.zip).
Apollo Southbay:
SunnyvaleBigLoop trajectory is used for evaluation, other 5 trajectories (BaylandsToSeafood, ColumbiaPark, Highway237, MathildaAVE, SanJoseDowntown) are used for training.
- Download the Apollo dataset.
KITTI odometry:
We use the improved ground truth poses provided with the SemanticKITTI dataset.
- Download the SemanticKITTI dataset (velodyne point clouds and calibration data for poses).
ALITA:
We evaluate on the data released at the ICRA 2022 UGV Challenge and use the validation sequence 5.
- Download the ALITA dataset.
Our test set for each dataset is defined in pickle format.
Genrate pickles for the 5 datasets:
We follow the pickle generation convention of EgoNN.
cd datasets/mulran
# For Sejong:
python generate_evaluation_sets.py --dataset_root <mulran_dataset_root_path>
# For DCC:
python generate_evaluation_sets.py --dataset_root <mulran_dataset_root_path>
cd ../southbay
python generate_evaluation_sets.py --dataset_root <apollo_southbay_dataset_root_path>
cd ../kitti
python generate_evaluation_sets.py --dataset_root <kitti_dataset_root_path>
cd ../alita
python generate_evaluation_sets.py --dataset_root <alita_dataset_root_path>
cd ../kitti360
python generate_evaluation_sets.py --dataset_root <kitti360_dataset_root_path>
This section re-creates the results of Tab. 3, Tab. 4 and Tab. 5 in our paper.
We integrate SpectralGV with the following architectures:
EgoNN + SpectralGV
- Clone the EgoNN codebase into
evaluation/EgoNN/
.
cd evaluation/EgoNN
git clone https://github.com/jac99/Egonn.git
- Copy our re-ranking eval script into the EgoNN code base:
cp -r SGV_EgoNN/ Egonn/eval/
cd Egonn/eval/SGV_EgoNN/
- Evaluate place recognition and metric localization with and without SpectralGV re-ranking:
python eval_egonn_sgv.py --dataset_type <dataset_eg_'kitti'> --dataset_root <dataset_root_path>
LoGG3D-Net + SpectralGV
- Download the pre-trained model (~103 MB) from Dropbox here:
cd evaluation/LoGG3D-Net/
wget --output-document logg3d.pth https://dl.dropboxusercontent.com/s/2mghsmkbz8p7ntx/logg3d.pth?dl=0
- Evaluate place recognition and metric localization with and without SpectralGV re-ranking:
python eval_logg3d_sgv.py --dataset_type <dataset_eg_'kitti'> --dataset_root <dataset_root_path>
Note: LoGG3D-Net does not support the parallel implementation fo SpectralGV as it outputs a varying number of local features/points for each point cloud.
LCDNet + SpectralGV
- Clone the LCDNet codebase into
evaluation/LCDNet/
.
cd evaluation/LCDNet
git clone https://github.com/robot-learning-freiburg/LCDNet.git
- Copy our re-ranking eval script into the EgoNN code base:
cp -r SGV_LCDNet/ LCDNet/evaluation/
cd LCDNet/
- Download the pre-trained model (~142 MB) from Dropbox here:
wget --output-document lcdnet.tar https://dl.dropboxusercontent.com/s/52sis2grvxias7u/lcdnet.tar?dl=0
mkdir checkpoints && mv lcdnet.tar ./checkpoints/
- Evaluate place recognition and metric localization with and without SpectralGV re-ranking:
python evaluation/SGV_LCDNet/eval_lcdnet_sgv.py --dataset_type <dataset_eg_'kitti'> --dataset_root <dataset_root_path>
If you find this work useful in your research, please cite:
@article{vidanapathirana2023sgv,
author={Vidanapathirana, Kavisha and Moghadam, Peyman and Sridharan, Sridha and Fookes, Clinton},
journal={IEEE Robotics and Automation Letters},
title={Spectral Geometric Verification: Re-Ranking Point Cloud Retrieval for Metric Localization},
year={2023},
publisher={IEEE},
volume={8},
number={5},
pages={2494-2501},
doi={10.1109/LRA.2023.3255560}}
Functions from 3rd party have been acknowledged at the respective function definitions or readme files.
SpectralGV was mainly inspired by SpectralMatching, PointDSC and SC2_PCR. The evaluation scripts used in this project are adapted from EgoNN.
For questions/feedback,