Keywords: Python, TensorFlow, Deep Learning, Semantic Segmentation, UNet, Autonomous Driving, Carla simulator
- Installation
- Introduction
2.1 Goal
2.2 Results - Project structure
- Dataset
- Project usage
5.1 Record raw data to .tfrecord format
5.2 Train a UNet for Semantic Segmentation
5.3 Visualize predictions with trained model
5.4 Export trained model & run inference on Test set - Todo
- Resources
This project was designed for:
- Python 3.6
- TensorFlow 1.12.0
Please install requirements & project:
$ cd /path/to/project/
$ git clone https://github.com/filippogiruzzi/semantic_segmentation.git
$ cd semantic_segmentation/
$ pip3 install -r requirements.txt
$ pip3 install -e . --user --upgrade
The purpose of this project is to design and implement a real-time Semantic Segmentation algorithm based on Deep Learning.
The designed solution is based on a UNet model implemented in TensorFlow. I use a Focal loss to solve the unbalanced data problem among the classes.
The model was trained on Google Colab for approximately 200 epochs with the default parameters.
Model | Train loss. | Val loss. | Test loss. |
---|---|---|---|
UNet | 1141 | 1262 | 1262 |
The project semantic_segmentation/
has the following structure:
semseg/data_processing/
: data processing, recording & visualizationsemseg/training/
: data input pipeline, model & training / evaluation / prediction operationssemseg/inference/
: exporting trained model & inference
Please download this
Semantic Segmentation dataset
on Kaggle, which is based from the Lyft Udacity challenge
and extract all files to : /path/to/carla_semseg_data/
.
The dataset contains approximately 5000 simulated images from the Carla simulator . I split the Train / Val / Test sets with 3000 / 1000 / 1000 images with Semantic Segmentation labels.
$ cd /path/to/project/semantic_segmentation/semseg/
$ python3 data_processing/data_to_tfrecords.py --data_dir /path/to/carla_semseg_data/
This will record the splitted data to .tfrecord
format in /path/to/carla_semseg_data/tfrecords/
.
$ python3 training/train.py --data-dir /path/to/carla_semseg_data/tfrecords/
$ python3 training/train.py --data-dir /path/to/carla_semseg_data/tfrecords/
--mode predict
--model-dir /path/to/trained/model/dir/
--ckpt /path/to/trained/model/dir/
- Full training on Google Colab & update results
- Fix background loss bug & retrain
- Google Colab demo
- Inference scripts
- Add data augmentation
This project was widely inspired by:
- U-Net: Convolutional Networks for Biomedical Image Segmentation, O. Ronneberger, P. Fischer, T. Brox, 2015, Arxiv
- Losses for Image Segmentation, L. Nieradzik, Blog post
- Carla Simulator, A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, V. Koltun, Website
- CARLA: An Open Urban Driving Simulator, A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, V. Koltun, 2017, Arxiv
- Lyft Udacity challenge, Website