Back | Next | Contents
Image Recognition
In the previous step, we ran an application that came with the jetson-inference repo.
Now, we're going to walk through creating a new program from scratch for image recognition called my-recognition
. This program will be able to exist as a standalone project outside the repo, hence if you wish to use the jetson-inference library in your own projects and applications, you can follow this example.
For your convenience and reference, the completed files are available in the examples/my-recognition
directory of the repo, but the guide below will act like they reside in the user's home directory or in an arbitrary directory of your choosing.
You can store the my-recognition
example that we will be creating wherever you want on your Jetson.
For simplicity, this guide will create it in the user's home directory located at ~/my-recognition
.
Run these commands from a terminal to create the directory and files required:
$ mkdir ~/my-recognition
$ cd ~/my-recognition
$ touch my-recognition.cpp
$ touch CMakeLists.txt
$ wget https://github.com/dusty-nv/jetson-inference/raw/master/data/images/black_bear.jpg
$ wget https://github.com/dusty-nv/jetson-inference/raw/master/data/images/brown_bear.jpg
$ wget https://github.com/dusty-nv/jetson-inference/raw/master/data/images/polar_bear.jpg
Some test images are also downloaded to the folder with the wget
commands above.
Next, we'll add the code for the program to the empty source files we created here.
Open up my-recognition.cpp
in your editor of choice (or run gedit my-recognition.cpp
).
Let's start adding the necessary code to use the imageNet
class for recognizing images.
First, include a couple of headers that we'll need:
// include imageNet header for image recognition
#include <jetson-inference/imageNet.h>
// include loadImage header for loading images
#include <jetson-utils/loadImage.h>
note: these headers are installed under
/usr/local/include
during thesudo make install
step of building the repo
if you did not runsudo make install
, then these headers won't be found when we go to compile the example.
Next, declare your main()
method and verify that the program was launched with the image filename as an argument on the command line:
// main entry point
int main( int argc, char** argv )
{
// a command line argument containing the image filename is expected,
// so make sure we have at least 2 args (the first arg is the program)
if( argc < 2 )
{
printf("my-recognition: expected image filename as argument\n");
printf("example usage: ./my-recognition my_image.jpg\n");
return 0;
}
// retrieve the image filename from the array of command line args
const char* imgFilename = argv[1];
This example loads and classifies an image that the user specifies. It will be expected to be run like this:
$ ./my-recognition my_image.jpg
The desired image filename to be loaded should be substituted for my_image.jpg
. The code above makes sure that this command line argument was provided to the program.
Declare some variables that will store the dimensions of the image and pointers to it's memory, and then load the image from disk with the loadImageRGBA()
function.
// these variables will be used to store the image data and dimensions
// the image data will be stored in shared CPU/GPU memory, so there are
// pointers for the CPU and GPU (both reference the same physical memory)
float* imgCPU = NULL; // CPU pointer to floating-point RGBA image data
float* imgCUDA = NULL; // GPU pointer to floating-point RGBA image data
int imgWidth = 0; // width of the image (in pixels)
int imgHeight = 0; // height of the image (in pixels)
// load the image from disk as float4 RGBA (32 bits per channel, 128 bits per pixel)
if( !loadImageRGBA(imgFilename, (float4**)&imgCPU, (float4**)&imgCUDA, &imgWidth, &imgHeight) )
{
printf("failed to load image '%s'\n", imgFilename);
return 0;
}
The loaded image will be stored in shared memory that's mapped to both the CPU and GPU. There are two pointers available for access in the CPU and GPU address spaces, but there is really only one copy of the image in memory. Both the CPU and GPU pointers resolve to the same physical memory, without needing to perform memory copies (i.e. cudaMemcpy()
).
When accessing the image from CPU code, the imgCPU
pointer should be used, and when accessing the image from within a CUDA kernel on the GPU, the imgCUDA
pointer should be used. Since our operations in this example will run on the GPU with TensorRT, we will be using the imgCUDA
pointer.
The image is loaded in float4
RGBA format, with pixel values between 0.0 and 255.0.
Using the imageNet::Create()
function, the following code will load the GoogleNet model with TensorRT, which was already downloaded when you initially built the jetson-inference repo. The model is pre-trained on the ImageNet ILSVRC12 dataset, which can recognize up to 1000 different classes of objects, like different kinds of fruits and vegetables, many different species of animals, along with everyday man-made objects like vehicles, office furniture, sporting equipment, ect.
// load the GoogleNet image recognition network with TensorRT
// you can use imageNet::ALEXNET to load AlexNet model instead
imageNet* net = imageNet::Create(imageNet::GOOGLENET);
// check to make sure that the network model loaded properly
if( !net )
{
printf("failed to load image recognition network\n");
return 0;
}
If desired, you can load the AlexNet model instead of GoogleNet by calling imageNet::Create(imageNet::ALEXNET)
.
The AlexNet model is also trained on the same 1000 classes of objects from ILSVRC12.
Next, we are going to classify the image with the image recognition network using the imageNet::Classify()
function:
// this variable will store the confidence of the classification (between 0 and 1)
float confidence = 0.0;
// classify the image with TensorRT on the GPU (hence we use the CUDA pointer)
// this will return the index of the object class that the image was recognized as (or -1 on error)
const int classIndex = net->Classify(imgCUDA, imgWidth, imgHeight, &confidence);
imageNet::Classify()
accepts an image pointer in GPU memory, and performs the inferencing with TensorRT.
It returns the index of the object class that the image was recognized as, along with the confidence value of the result.
Unless the call to imageNet::Classify()
resulted in an error, let's print out the classification info of the recognized object:
// make sure a valid classification result was returned
if( classIndex >= 0 )
{
// retrieve the name/description of the object class index
const char* classDescription = net->GetClassDesc(classIndex);
// print out the classification results
printf("image is recognized as '%s' (class #%i) with %f%% confidence\n",
classDescription, classIndex, confidence * 100.0f);
}
else
{
// if Classify() returned < 0, an error occurred
printf("failed to classify image\n");
}
Since imageNet::Classify()
returns an integer-based index of the object class (between 0 and 1000 for ILSVRC12), we use the imageNet::GetClassDesc()
function to retrieve a human-readable description of the object.
These descriptions of the 1000 classes are parsed from ilsvrc12_synset_words.txt
when the network gets loaded (this file was previously downloaded when the jetson-inference repo was built).
Before exiting the program, delete
the network object to destroy the TensorRT engine and free CUDA resources:
// free the network's resources before shutting down
delete net;
// this is the end of the example!
return 0;
}
That's it! Remember to add the return statement and closing curly bracket to your main() method.
Next we just need to create a simple makefile for our new recognition program with CMake.
Open the file ~/my-recognition/CMakeLists.txt
in editor, and add the following code:
# require CMake 2.8 or greater
cmake_minimum_required(VERSION 2.8)
# declare my-recognition project
project(my-recognition)
# import jetson-inference and jetson-utils packages.
# note that if you didn't do "sudo make install"
# while building jetson-inference, this will error.
find_package(jetson-utils)
find_package(jetson-inference)
# CUDA and Qt4 are required
find_package(CUDA)
find_package(Qt4)
# setup Qt4 for build
include(${QT_USE_FILE})
add_definitions(${QT_DEFINITIONS})
# compile the my-recognition program
cuda_add_executable(my-recognition my-recognition.cpp)
# link my-recognition to jetson-inference library
target_link_libraries(my-recognition jetson-inference)
In the future you can use this CMakeLists as a template for compiling your own projects that use the jetson-inference
library. The most relevant bits are:
- Pull in the
jetson-utils
andjetson-inference
projects:find_package(jetson-utils) find_package(jetson-inference)
- Link against
libjetson-inference
:target_link_libraries(my-recognition jetson-inference)
note: these libraries are installed under
/usr/local/lib
during thesudo make install
step of building the repo
if you did not runsudo make install
, then these libraries won't be found when we go to compile the example.
Now that our source files are complete, run the following shell commands to compile the my-recognition
program:
$ cd ~/my-recognition
$ cmake .
$ make
If you encounter errors, make sure that you ran sudo make install
while building the jetson-inference repo.
You can also download the completed, working code of this example from the examples/my-recognition
directory of the repo.
Now that our program is compiled, let's classify the test images that we downloaded at the beginning of this guide:
$ ./my-recognition polar_bear.jpg
image is recognized as 'ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus' (class #296) with 99.999878% confidence
$ ./my-recognition brown_bear.jpg
image is recognized as 'brown bear, bruin, Ursus arctos' (class #294) with 99.928925% confidence
$ ./my-recognition black_bear.jpg
image is recognized as 'American black bear, black bear, Ursus americanus, Euarctos americanus' (class #295) with 98.898628% confidence
This is the conclusion of this section of the tutorial. Next, we'll classify a live video feed from the Jetson onboard camera.
Next | Running the Live Camera Recognition Demo
Back | Classifying Images with ImageNet
© 2016-2019 NVIDIA | Table of Contents