- Load RGB-D datasets Cityscapes, COCO, SUNRGBD, and SceneNetRGBD
- Optional RGB-D network input using 4th channel in first convolutional layer
- YACS configuration files
This is a PyTorch(1.4) implementation of DeepLab-V3-Plus. It can use Modified Aligned Xception and ResNet as backbone.
The code was tested with Anaconda and Python 3.8. After installing the Anaconda environment:
-
Clone the repo:
git clone https://github.com/jfzhang95/pytorch-deeplab-xception.git cd pytorch-deeplab-xception
-
Install dependencies:
For PyTorch dependency, see pytorch.org for more details.
Other dependencies:
conda install matplotlib pillow tqdm protobuf tabulate scipy numpy pandas pip install tensorboardX yacs
Coco tools
conda install -c conda-forge pycocotools scikit-image
-
Install dataloader repo
This is a seperate repo for use in different projects. For more details, see the repo README
git clone https://github.com/crmauceri/dataset_loaders.git cd dataset_loaders/datasets make cd ../.. pip install -e .
-
Compile CUDA code
The depth-aware convolution and depth-aware average pooling operations are under folder
modeling/backbone/ops/
, to build them, simply usepython setup.py install
to compile.cd modeling/backbone/ops/depthconv/ python setup.py install cd ../depthavgpooling/ python setup.py install cd ../../../..
-
Compile SceneNetRGBD protobuf files
cd dataloaders/datasets make
-
Install as module:
cd $root pip install -e .
From parent repo,
Backbone | Dataset | mIoU in val | Pretrained Model |
---|---|---|---|
ResNet (pretrained) | Pascal VOC 2012 | 78.43% | google drive |
New models,
Backbone | Dataset | mIoU in val | Pretrained Model |
---|---|---|---|
ResNet (pretrained) | COCO | google drive | |
ResNet | COCO | google drive |
Backbone | Dataset | mIoU in val | Pretrained Model |
---|---|---|---|
ResNet | COCO + Synthetic Depth | google drive | |
ResNet | Cityscapes | google drive | |
ResNet | SceneNetRGBD | google drive |
Follow steps below to train your model:
-
Configuration files use YACS. Full list of defaults is in
deeplab3/config/defaults.py
. Configurations used to train model zoo are inconfigs/
. You must setDATASET.ROOT
to match your dataset location, otherwise, you can change as many or as few parameters as you desire. -
Run training script as To train deeplabv3+ using COCO dataset and ResNet as backbone:
python deeplab3/train.py <config_file> <optional_parameter_overrides>
For example, to train deeplabv3+ using SUNRGBD dataset and ResNet as backbone:
python deeplab3/train.py configs/sunrgbd.yaml
Jupyter notebook COCO_Data_Browser.ipynb
is provided for visualizing data and trained network results.
Tensorboard can also be used to visualize loss and accuracy metrics during training.
tensorboard serve --log=run/<output_path>