- "Edge Application SDK" for AITRIOS™
"Edge Application SDK" for AITRIOS is a toolkit for developing AI models and post-processing applications that can be installed on Edge AI Devices. Post-processing applications are called "Edge Applications". The models and "Edge Applications" can be deployed to Edge AI Devices through "Console for AITRIOS".
- Use GitHub Codespaces (Dev Container) as development environment.
- You don't need to install any additional tools in your environment.
- Develop your AI models in the container.
- Develop "Edge Applications" using build environment and sample code included in the container.
- Import AI models and "Edge Applications" to "Console for AITRIOS" and deploy them to Edge AI Devices.
"Edge Application SDK" is provided as Development Container (Dev Container) that runs on GitHub Codespaces or Docker environment on Local PC. This container includes tools and jupyter notebooks that can be used for development.
graph TB;
%% definition
classDef object fill:#FFE699, stroke:#FFD700
classDef device fill:#FFFFFF
classDef sdk fill:#FFFFFF, stroke:#6b8e23, stroke-dasharray: 10 2
classDef external_service fill:#BFBFBF, stroke:#6b8e23, stroke-dasharray: 10 2
style legend fill:#FFFFFF, stroke:#000000
%% impl
subgraph legend["legend"]
process(Process)
object[Data/Artifact]:::object
sdk[SDK]:::sdk
extern[External Service]:::external_service
device[Device]:::device
end
%%{init: {'theme': 'default'}}%%
graph TB;
style DevContainer fill:#FFFFFF, stroke:#6b8e23, stroke-dasharray: 10 2
style Console fill:#BFBFBF, stroke:#6b8e23, stroke-dasharray: 10 2
style your_env fill:#BFBFBF, stroke:#6b8e23, stroke-dasharray: 10 2
classDef object fill:#FFE699, stroke:#FFD700
classDef device fill:#FFFFFF
prepare_sdk(Prepare Dataset)
train_sdk(Train Model)
quantize_sdk(Quantize Model)
train_yours(Train Model)
prepare_console(Prepare Dataset)
train_console(Create/Train Model)
quantize_console(Quantize Model)
deploy_console(Deploy)
eval_console(Evaluate)
img_sdk[Images]:::object
img_console[Images]:::object
data_sdk[Dataset]:::object
data_console[Dataset]:::object
ai_model[AI-Model]:::object
custom_model[Your<br>AI-Model]:::object
quantize_model_sdk[Quantized<br>TFLite-Model]:::object
quantize_model_console[Quantized<br>TFLite-Model]:::object
eval_result_sdk[Evaluation<br>Result]:::object
eval_result_console[Evaluation<br>Result]:::object
device[Edge AI Device]:::device
subgraph your_env[Your Training Environment]
train_yours
end
subgraph Console["Console for AITRIOS"]
img_console --> prepare_console
prepare_console --> data_console
data_console --> train_console
train_console --> ai_model
ai_model --> quantize_console
quantize_console --> quantize_model_console
quantize_model_console --> deploy_console
eval_console --> eval_result_console
end
deploy_console --> device
device --> eval_console
subgraph DevContainer[Edge Application SDK Dev Container]
prepare_sdk --> img_sdk
img_sdk --> prepare_sdk
prepare_sdk --> data_sdk
data_sdk --> train_sdk
train_sdk --> custom_model
custom_model --> quantize_sdk
%%img_sdk --> quantize_sdk
quantize_sdk --> quantize_model_sdk
quantize_sdk --> eval_result_sdk
end
data_sdk --> train_yours
img_sdk --> train_yours
train_yours --> custom_model
img_sdk --> img_console
quantize_model_sdk --> deploy_console
%%{init: {'theme': 'default'}}%%
graph TB;
style DevContainer fill:#FFFFFF, stroke:#6b8e23, stroke-dasharray: 10 2
style Console fill:#BFBFBF, stroke:#6b8e23, stroke-dasharray: 10 2
classDef object fill:#FFE699, stroke:#FFD700
classDef device fill:#FFFFFF
ppl_code[Application Code<br>C/C++]:::object
output_tensor[Output Tensor<br>.jsonc]:::object
ppl_parameter[PPL Parameter<br>.json]:::object
wasm[Appliaction<br>.wasm]:::object
wasm2[Appliaction<br>.wasm]:::object
aot[Application<br>.aot]:::object
eval_result[Evaluation<br>Result]:::object
develop(Develop Edge Application)
build_wasm(Build)
debug_wasm(Run / Debug)
compile_aot(Compile)
deploy(Deploy)
eval(Evaluate)
device[Edge AI Device]:::device
subgraph DevContainer[Edge Application SDK Dev Container]
develop --> ppl_code
ppl_code --> build_wasm
build_wasm --> wasm
wasm --> debug_wasm
output_tensor --> debug_wasm
ppl_parameter --> debug_wasm
end
subgraph Console["Console for AITRIOS"]
wasm2 --> compile_aot
compile_aot --> aot
aot --> deploy
eval --> eval_result
end
wasm --> wasm2
deploy --> device
device --> eval
- Console for AITRIOS
Following functions are available on "Console for AITRIOS":- manage device
- upload image from device
- import image from your local PC or storages
- import AI model
- import "Edge Application"
- deploy model and "Edge Application" to device
- create model
- annotate image (for AI model created on "Console for AITRIOS")
- train model (for AI model created on "Console for AITRIOS")
- Dev Container
Dev Container provides tools and notebooks to support cases where you want to create your own AI model and run it on Edge AI Devices.
Following functions are available on Dev Container.- Prepare dataset:
- Jupyter notebook for downloading images
- Tools for image annotation
- Prepare models:
- Jupyter notebook for training models
- Jupyter notebook for quantizing models
- Jupyter notebook for importing models to "Console for AITRIOS"
- Jupyter notebook for deploying models to Edge AI Devices
- Prepare applications:
- Tools for developing, building and debugging "Edge Applications"
- Jupyter notebook for importing "Edge Applications" to "Console for AITRIOS"
- Jupyter notebook for deploying "Edge Applications" to Edge AI Devices
- See Tutorials for details on each notebook and tool.
- Prepare dataset:
- AI model training
- Datasets for Object Detection created on the Dev Container cannot be used for training base AI models (only for training user's custom AI models).
- AI model quantization on Dev Container
- Supported AI models are based on "Model Compression Toolkit (MCT)"'s features.
- Jupyter specification
- Variables in jupyter notebook are cleared when GitHub Codespaces stop.
- Evaluation of output results
- Images and inference results output from devices cannot be downloaded from "Console for AITRIOS".
- Inference results displayed on "Console for AITRIOS" are serialized.
- To deserialize inference results, see README for deserializing.
- Device operation
- Edge AI Devices can only be operated through "Console for AITRIOS".
- Codespaces specification
- It takes about 15 minutes to start a container on Codespaces for the first time. From the second time onwards, it starts up within 1 minute.
- By default, a container on Codespaces continues running for 30 minutes without any operation. You can change the settings for up to 4 hours.
See Development Environment Setup Guide.
NOTE
- 4-core (8GB) or higher machine types are recommended when using the SDK on Codespaces.
- If 2-core is selected, an error may occur during Dev Container build.
- To ensure security of connection for CVAT, do NOT make Codespace's port forwarding sharing option "Public".
- If Jupyter Kernels picker displayed when you try to run Jupyter notebook first time, please select "Python Environment" and select "Python 3.8".
You can learn the development workflow using the samples in a day.
See "Edge Application SDK" samples.
You can start the development workflow using the tutorial.
See "Edge Application SDK" tutorials.
NOTE
If you are unfamiliar with "Edge Application SDK", it's recommended to learn with Samples first.
If you have already developed the "Edge Application" using SDK v0.2, you need to modify the "Edge Application" source code to migrate to SDK v1.0.
See "Edge Application" Migration Guide from SDK v0.2 to v1.0.
-
Prepare dataset
-
Prepare model
-
Prepare application
-
Setup "Console Access Library"
-
Import AI model and "Edge Applications" to "Console for AITRIOS"
-
Deploy AI model and "Edge Applications" to Edge AI Device
-
Development container
-
Version control
-
Deserialize
Before using Codespaces, please read the Site Policy of GitHub and understand the usage conditions.
This SDK is optimized for use with Sony's AITRIOS™ (https://developer.aitrios.sony-semicon.com/edge-ai-sensing/). If you are willing to take part in the AITRIOS and use services provided through AITRIOS, you must sign up with https://developer.aitrios.sony-semicon.com/edge-ai-sensing/ and comply with AITRIOS Terms of Use and other applicable terms of use in addition to the license terms of this SDK.
AITRIOS is a one-stop platform that provides tools and environments to facilitate software and application development and system construction.
Sony, with the aim of utilizing AI technology to enrich people's life styles and contribute to the development of society, Sony will pursue accountability and transparency while actively engaging in dialogue with stakeholders. Sony will continue to promote responsible AI in order to maintain the trust of products and services by stakeholders.
Users of this SDK should refer to and understand our thoughts and initiatives about AI. You can learn more here, including Sony Group AI Ethics Guidelines. https://www.sony.com/en/SonyInfo/sony_ai/responsible_ai.html
This repository aims to adhere to Semantic Versioning 2.0.0.
See the "Release Note" from [Releases] for this repository.
Each release is generated in the main branch. Pre-releases are generated in the develop branch. Releases will not be provided by other branches.