Skip to content

Go one dimension higher: can neural networks for point cloud analysis help in image recognition?

License

Notifications You must be signed in to change notification settings

chenzhaiyu/pointset-image-recognition

Repository files navigation

Go One Dimension Higher: Can Neural Networks for Point Cloud Analysis Help in Image Recognition? License: MIT

Introduction

Inspired by the sanity check made by Qi et al., which treats an image from MNIST dataset as a 2D point cloud, in this project we discuss whether neural networks designed for point cloud analysis can help image recognition.

While we focus on 3D point cloud learning, a sanity check experiment is to apply our network on a 2D point clouds - pixel sets.

Three neural networks originally for point cloud analysis are employed, namely PointNet, PointNet++ and PointCNN. We have no ambition to defeat the state-of-the-art CNNs. Instead, the goal of this project is to discover the potential of these networks designed for 3D point set on 2D pixel set. The impact of shape information and the shift variance characteristic are discussed. For more details, please refer to the blog.

Requirements

  • PyTorch 1.2

  • PyTorch Geometric 1.5

    Please refer to environment.yml for a complete list of required packages.

Usage

Create the environment from the environment.yml file:

conda env create -f environment.yml

Classification

# example training
python train_cls.py --batch_size 64 --data_dir 'path/to/data' --log_dir 'path/to/log' --dataset_name 'fashion' --epoch 250 --model 'pointcnn_cls' --num_class 10 --num_point 256

Semantic Segmentation

# example training
python train_semseg.py --batch_size 64 --data_dir 'path/to/data' --log_dir 'path/to/log' --epoch 250 --model 'pointnet2_sem_seg' --num_class 20 --num_point 4096

Tensorboard

tensorboard --logdir 'path/to/log'

About

Go one dimension higher: can neural networks for point cloud analysis help in image recognition?

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages