yes, this is a long title.
Hardware:
- PC (with CUDA enabled GPU)
- Raspberry Pi 3 B+
- Intel Neural Compute Stick 2
- PyCam / webcam
Software:
- Ubuntu 18.04.02 LTS
- Nvidia CUDA 10.1
- OpenVINO Toolkit 2019 R1.01
- Darknet 61c9d02 - Sep 14, 2018
- Raspbian Stretch Lite - version April 2019
- Python 3.5
To simplify things, this repository contains all other necessary git repositories as submodules. To clone them together with this repository use:
git clone --recurse-submodules https://github.com/eddex/tiny-yolov3-on-intel-neural-compute-stick-2.git
Note: You can also download the repositories seperately in later steps.
To use your graphics card when training the custom YOLOv3-tiny model, install CUDA.
How to: Install CUDA 10.1 on Ubuntu 18.04
If you haven't downloaded the submodules, clone https://github.com/AlexeyAB/darknet
git clone [email protected]:AlexeyAB/darknet.git
navigate to the darknet repo: cd darknet
Enable training on GPU (requires CUDA)
open the file Makefile
in your prefered text editor and set GPU=1
and OPENMP=1
.
GPU=1
CUDNN=0
CUDNN_HALF=0
OPENCV=0
AVX=0
OPENMP=1
LIBSO=0
ZED_CAMERA=0
Build darknet
Install build-essential:
sudo apt-get update
sudo apt-get install build-essential
Build:
make
Note: When using fish
instead of bash
, the build might fail. Just use bash in this case.
Train a custom model based on YOLOv3-tiny
Copy the config files to the darknet/
directory:
cp signals.names darknet/
cp signals.data darknet/
cp yolov3-tiny-signals.cfg darknet/
The config files in this repo are altered to fit the signals-dataset. To train a model on another dataset, follow the instructions here: https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects
Navigate to the signals-dataset folder and run the script to create a train/test split of the data.
cd signals-dataset
python3 create_train_test_split.py
This should generate a directory called yolov3
along with two files train.txt
and test.txt
.
Copy the directory and the two text files to the darknet directory:
cp signals-dataset/train.txt darknet/
cp signals-dataset/test.txt darknet/
cp -r signals-dataset/yolov3 darknet/
Download the pre-trained weights file:
cd darknet
wget https://pjreddie.com/media/files/yolov3-tiny.weights
Get pre-trained weights for convolutional layers (./darknet
is the binary inside the darknet/
directory):
./darknet partial cfg/yolov3-tiny.cfg yolov3-tiny.weights yolov3-tiny.conv.15 15
Start training:
./darknet detector train signals.data yolov3-tiny-signals.cfg yolov3-tiny.conv.15
To calculate mean average precision after every 1000 batches (1 iteration) start training with the -map
flag.
./darknet detector train signals.data yolov3-tiny-signals.cfg yolov3-tiny.conv.15 -map
The trained model is saved in darknet/backup/
as .weights
file for every iteration. When using -map
, the model with the highets precision is saved as ..._best.weights
.
Run the script to export the mAP data:
python3 calculate_map.py
The script creates a .map
file for each .weights
file (excluding best, final and last since they would be redundant).
You can also manually check the mAP of a .weights
file:
./darknet detector map signals.data yolov3-tiny-signals.cfg backup/yolov3-tiny-signals_final.weights
After exporting the data you can visualize it using the create_diagrams_from_mAP_data.ipynb
jupyter notebook.
Run the following command. it will then prompt you for a path to an image. Enter the path of an image in the test-set (or any other image)
./darknet detector test signals.data yolov3-tiny-signals.cfg backup/yolov3-tiny-signals_best.weights
Note: You need to register to download the OpenVINO Toolkit. In my case the registration form was broken. Here's a direct download link for Linux (version 2019 R1.0.1): http://registrationcenter-download.intel.com/akdlm/irc_nas/15461/l_openvino_toolkit_p_2019.1.133.tgz
Then follow the official installation guide:
Note: When using fish
instead of bash
, setting the environment variables might not work. This is not a problem. We'll use absolute paths where needed in the steps below.
To run the model on the Intel Neural Compute Stick 2, we need to convert it to an "Intermediate Representation".
There's no need to follow the official guide if you use the instructions below. But for reference, it can be found here: https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html
Dump YOLOv3 TensorFlow Model:
For this step you need to install tensorflow:
pip3 install tensorflow==1.12.2
IMPORTANT: The correct version of tensorflow has to be used, otherwise the conversion goes wrong! 1.12.2 works. >=1.13.0 does not work. See: https://software.intel.com/en-us/forums/computer-vision/topic/807383
then run:
python3 tensorflow-yolo-v3/convert_weights_pb.py --class_names signals.names --data_format NHWC --weights_file yolov3-tiny-signals.weights --tiny
The TensorFlow model is saved as frozen_darknet_yolov3_model.pb
Convert YOLOv3 TensorFlow Model to the IR:
python3 /opt/intel/openvino_2019.1.133/deployment_tools/model_optimizer/mo_tf.py --input_model frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config yolo_v3_tiny_custom.json --input_shape [1,416,416,3] --data_type FP16
The --input_shape
parameter is needed as otherwise it blows up due to getting -1 for the mini-batch size. Forcing this to 1 solves the problem.
To run the model on a MYRIAD processor (Intel Compute Stick 2), the parameter --data_type FP16
has to be passed.
The original yolo_v3_tiny.json
can be found in <OPENVINO_INSTALL_DIR>/deployment_tools/model_optimizer/extensions/front/tf/
.
The IR is generated and saved as frozen_darknet_yolov3_model.xml
and frozen_darknet_yolov3_model.bin
.
This section decribes how to setup and configure a Raspberry Pi 3 B+ to run the YOLOv3-tiny model on the Intel Neural Compute Stick 2.
- Download Raspbian Stretch Lite from https://www.raspberrypi.org/downloads/raspbian/
- Install the OS on an SD card (using https://www.balena.io/etcher/)
- Create a file called
ssh
in the root of theboot
partition of the SD card. - Create a file called
wpa_supplicant.conf
in theboot
partition of the SD card. - Make sure to change the “End of Line” setting set to “UNIX” for both files!
- In
wpa_supplicant.conf
add the following content:
update_config=1
ctrl_interface=/var/run/wpa_supplicant
network={
scan_ssid=1
ssid="MyNetworkSSID"
psk="MyNetworkPassword"
}
find Raspberry PI in local network:
sudo nmap -sP 192.168.1.0/24 | awk '/^Nmap/{ip=$NF}/B8:27:EB/{print ip}'
Follow the instruction in the official guide (v 2019 R1.01): https://docs.openvinotoolkit.org/latest/_docs_install_guides_installing_openvino_raspbian.html
Maybe it's necessary to add the inference engine libraries to the PATH. Add this to the bottom of ~/.bashrc
export PATH=$PATH:/opt/intel/openvino/inference_engine/lib
Maybe you need to add this to the top of your python scripts:
sys.path.insert(0, '/opt/intel/openvino/python/python3.5')
Install dependencies (can take quite long):
apt update
apt install python3-pip
pip3 install opencv-python
apt install libgtk-3-dev
At this point there should be an Intel Neural Compute Stick 2 and a camera connected to the Raspberry Pi. The camera can be a PyCam or any USB Webcam that can be detected by OpenCV.
Copy the files openvino_tiny-yolov3_test.py
, frozen_darknet_yolov3_model.xml
and frozen_darknet_yolov3_model.bin
to the Raspberry Pi:
scp openvino_tiny-yolov3_test.py [email protected]:~/openvino-python/
scp frozen_darknet_yolov3_model.xml [email protected]:~/openvino-python/
scp frozen_darknet_yolov3_model.bin [email protected]:~/openvino-python/
And run the python script on the Raspberry Pi:
python3 openvino_tiny-yolov3_test.py -d MYRIAD
or
python3 openvino_tiny-yolov3_MultiStick_test.py -numncs 1
For reference, the original python script can be found here: https://github.com/PINTO0309/OpenVINO-YoloV3
To run the examples on the CPU on UBUNTU, add the following to your ~/.bashrc
file.
export PATH="/opt/intel/openvino/deployment_tools/inference_engine/lib/intel64/:$PATH"
If you get an error that lib/libcpu_extension.so
if not found, change the line referencing the file to libcpu_extension_sse4.so
(remove lib/
!)