! check the workshop Link Build a Machine Learning Model Using Custom Vision
Azure Custom Vision allows you to create models which can classify and detect items in images. You could use this to detect a breed of dog, if someone is wearing a helmet, or the presence of other features. While this could be done by using code and manually processing the images, Custom Vision provides a web-based interface and tooling to streamline the process. No prior knowledge of machine learning or math is required! This repository is about building a model to detect dog breeds.
- [Part 0: Install and configure tools]
- [Part 1: Train your model]
- [Part 2: Test your model]
As with any project, a few tools are going to be needed. In particular you'll need a code editor, an Azure subscription, and a couple of keys for Custom Vision.
This workshop assumes core knowledge of Git, specifically the ability to clone a repository. During the workshop you will download the code from GitHub. You will need either Git to clone the repository.
You will build a project using Python. Basic knowledge of Python is assumed, specifically the ability to import packages and use open
and with
. You will also need Python installed locally. To test if you have Python and pip installed, open a command or terminal window and run the following commands.
python --version
pip3 --version
If Python and pip are installed a version number will be displayed, such as 3.8.10. Otherwise, an error message is displayed. You can install Python and pip as needed:
The instructions for this workshop assume you are using Visual Studio Code, an open source code editor. You will also need the Pylance extension, which will provide additional functionality when writing Python.
A starter project has been provided which includes the images for the workshop and the solution code. To obtain the project:
-
Navigate to https://github.com/alshubati99/custom-vission-MLmodel/
-
Open a terminal or command window and execute the following, replacing <URL> with the URL you copied in the prior step
# Windows,Linux, WSL or macOS: git clone <URL> cd custom-vission-MLmodel.
Azure for Students offers $100 US in free credit to be used across 12 months and a host of free services - including Custom Vision! If you don't already have an Azure subscription, you can enroll with Azure for Students by using your school email address. If your email address doesn't work, please ask your instructor for a code.
To use Custom Vision, you'll need two keys:
- a training key, used to create the model
- a prediction key, used to make predictions based on the model
We'll create these keys by using the Azure portal.
- Navigate to portal.azure.com
- Select Create a resource
- Search for Custom Vision and press enter
- Select Custom Vision from the list of services
- Select Create
- For Create Options ensure Both is selected
- Under Project Details configure the following:
- For Subscription select your Azure for Students subscription
- For Resource Group select Create new, enter custom-vision for the name, and select OK
- Under Instance Details configure the following:
- For Region select a region near you
- For Name, provide a unique name such as custom-vision-your-name, replacing your-name with your name
- For both Training pricing tier and Prediction pricing tier select Free F0
- Select Review + create
- Select Create
A custom vision model
A project is a custom vision model. It's used to label images with the appropriate tag (or category), and perform the training. Let's start by creating a project.
- Navigate to Custom Vision and sign in
- Select New Project
- Enter Dog Classification for the project name
- Next to Resource, select the key you created earlier, which will use the name you created during the set up.
- For Project Types select Classification
- For Classification Types select Multiclass, as our dogs will only have one breed
- For Domains select General [A2]
- Select Create project
Once the project is created it's time to upload images. These images are used to train the model.
Tip: As a general rule, the more images you can use to train a model the better. You want to include as much variety in the images as possible, including different lighting, angles, and settings.
- Select Add images
- Navigate to training-images
- Select all the images marked as american-staffordshire-terrier in the folder, and select Open
- Enter american-staffordshire-terrier for the tag and select Upload 8 files
- Select Done
- Repeat the above steps for the remaining breeds:
- australian-shepherd
- buggle
- german-wirehaired-pointer
- shorkie
- Select Train to open the training dialog
- Leave Quick Training selected and select Train to begin the training process
Note: Training the model will take a couple of minutes.
With the model trained it's time to turn our attention to using it. We'll start by testing it in the Custom Vision website. Then we'll explore how we can call the model from code by using an SDK.
Let's see how well our model works. It's important to use images which weren't used to train the model. After all, if the model has already seen the image it's going to know the answer.
- Select Quick Test
- Select Browse local files
- Navigate to testing-images and select one of the dog images
- Select Open
- Notice the tag and probability scores
The goal of creating a model in Custom Vision is to use it in different applications. To access it from outside of the Custom Vision website it needs to be published.
- In the 'Performance' tab, select Publish
- For Model name enter dogs
- For Prediction resource select the prediction key you created earlier.
- Select Publish
- Select Prediction URL to view the endpoint address
- Copy the value in the grey textbox under If you have an image file and paste it somewhere locally where you can find it later
- Select Got it
As with any service, we need to gather the keys and values to make our calls. You'll want to copy these keys into a text file; they'll be used in the next step.
- From the Performance screen, select Prediction URL
- Copy the URL under If you have an image file; this will be used to create the endpoint
- Copy the hex value for the Prediction-Key. It will look similar to: ba81ed4539cd46ec979a98569035a463
- Select Got it to close the window
- In the upper right corner of the Custom Vision interface, select the Gear icon for settings
- Copy the Project Id value
TIP: Project Id is the value on the left side of the screen
The SDK for Custom Vision uses a slightly different URL than the one you copied earlier. The value you copied will look something like the following:
https://customvisionworkshop-prediction.cognitiveservices.azure.com/customvision/v3.0/Prediction/0dd3299b-6a41-40fe-ab06-dd20e886ccd4/classify/iterations/Iteration1/image
To create the endpoint value, remove everything after azure.com. Your endpoint value should look like this:
https://customvisionworkshop-prediction.cognitiveservices.azure.com/
Whenever you're writing code it's a best practice to never hard-code sensitive values, such as passwords, or any value which may change. This can lead to security breaches or code that's difficult to maintain. To create our application we're going to follow generally accepted best practices. To do this we'll use a library named python_dotenv, which allows you to set environment variables with a text file.
-
Return to Visual Studio Code
-
In this project's root folder, create a new file named .env
-
Add the following values, replacing the placeholders
ENDPOINT=<YOUR_ENDPOINT> KEY=<YOUR_PREDICTION_KEY> PROJECT_ID=<YOUR_PROJECT_ID> PUBLISHED_ITERATION_NAME=dogs
-
Save the file by selecting File > Save
We'll be using two Python packages to support our project. The first we already introduced, python-dotenv. The next is the Custom Vision client library (or SDK), which you'll use to make the predictions.
-
Create a new file named requirements.txt
-
Inside requirements.txt, add the following:
python-dotenv azure-cognitiveservices-vision-customvision
-
Save the file by selecting File > Save
-
Open a new terminal window inside Visual Studio Code by selecting Terminal > New Terminal Window
-
Create a new Python environment and install the packages by running the following command:
# On Windows python -m venv venv .\venv\Scripts\activate pip install -r requirements.txt
# On Linux, WSL or macOS python3 -m venv venv source venv/bin/activate pip install -r requirements.txt
-
The packages will install.
With the setup work done, it's time to add your code! We'll break this down into a few sections.
- Create a new file named predict.py
Most Python applications start by importing the required libraries, and our program will follow the same pattern.
-
Inside predict.py, add the following to import the necessary libraries:
# Prediction client from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient # Key class for azure from msrest.authentication import ApiKeyCredentials # dotenv to load key from dotenv import load_dotenv # Import os to read environment variables import os
Earlier we setup a few environment variables like our key and endpoint. Let's load those into memory.
-
At the end of predict.py, add the following code to load the environment variables
# Load the key and endpoint values load_dotenv() # Set the values into variables key = os.getenv('KEY') endpoint = os.getenv('ENDPOINT') project_id = os.getenv('PROJECT_ID') published_name = os.getenv('PUBLISHED_ITERATION_NAME')
Now we can perform our prediction! We'll be using one of the images in the testing folder.
-
At the end of predict.py, add the following code to perform a prediction of dog breed
# Setup credentials for client credentials = ApiKeyCredentials(in_headers={'Prediction-key':key}) # Create client, which will be used to make predictions client = CustomVisionPredictionClient(endpoint, credentials) # Open the test file with open('testing-images/american-staffordshire-terrier-10.jpg', 'rb') as image: # Perform the prediction results = client.classify_image(project_id, published_name, image.read()) # Because there could be multiple predictions, we loop through each one for prediction in results.predictions: # Display the name of the breed, and the probability percentage print(f'{prediction.tag_name}: {(prediction.probability):.2%}')
With the program created, let's run it and see what happens!
-
Save all files by selecting File > Save All
-
Return to the terminal inside Visual Studio Code by selecting View > Terminal
-
Run the following command to execute the program
python predict.py
-
You should now see the following output:
american-staffordshire-terrier: 97.11% german-wirehaired-pointer: 1.46% australian-shepherd: 0.97% buggle: 0.46% shorkie: 0.00%
Note: Your percentages may not exactly match those above