Kirin

Architecture

Doing inference on 12 CPUs

demo.mov

Roadmap

• Inference mode(default): Chat with the small language model at consumer grade hardware with 8 CPUs

• RAG mode: Chat with small language model based on pre-processed datasets

• CRM features: Multiple users login, chat/chat(RAG), save chat history and share the chat history by JSON format

• Neural Network define

  • Pre-define neural network as template, support load weights and fine-tune
  • Support multiple neural network types
  • Support custom simple neural network by UI

• Training Factory

  • Train the neural network with the given dataset
  • Quantize the model and upload it to the server with the given tokens
  • Visualize the neural network and the training process

Requirements

You need to make sure Docker and docker-compose on your environment. See Requirements.md

Quick setup

If you want to setup the project quickly, please follow the steps below:

git clone https://github.com/SkywardAI/kirin.git

cd kirin

make demo

See quick start

Build and deployment

Also See deployment. And if you are interested in docker-in-docker development, see Development.md

Tech stack

This is a repository is the API aggregator of SkywardAI. It's using the following tech stack:

• 🐳 Dockerized
• 🐘 Asynchronous PostgreSQL
• 🐍 FastAPI

When the Docker is started, these are the URL addresses:

• Backend Application (API docs) $\rightarrow$ http://localhost:8000/docs
• Database editor (Adminer) $\rightarrow$ http//localhost:8081

Why the above Tech-Stack?

Well, the easy answer is Asynchronousity and Speed!

• FastAPI is crowned as the fastest web framework for Python and thus we use it for our backend development.
• The database of my choice is the asynchronous version of PostgreSQL (via SQLAlchemy 2.0). Read this blog from Packt if you want to educate yourself further about the topic Asynchronous, Synchronous, Concurrency, and Parallelism.
• Docker is a technology that packages an application into standardized units called containers that have everything the software needs to run including libraries, system tools, code, and runtime.

Other Technologies

The above-listed technologies are just the main ones. There are other technologies utilized in this project template to ensure that your application is robust and provides the best possible development environment for your team! These technologies are:

• CodeCov $\rightarrow$ A platform that analyzes the result of your automated tests.
• PyTest $\rightarrow$ The testing framework for Python code.
• DBDiagram $\rightarrow$ A platform that lets your design your database by writing SQL and converting it into ERD. This platform provides a complete symbol for entity relationships (not like many other platforms!).
• GitHub Actions $\rightarrow$ The platform to setup our CI/CD by GitHub.
• SQLAlchemy 2.0 $\rightarrow$ The go-to database interface library for Python. The 2.0 is the most recent update where it provides an asynchronous setup.
• CODEOWNERS $\rightarrow$ A file for distributing the responsibilities in our project to each team/teammate.

Acknowledgements

• LocalAI
• FastAPI
• SQLAlchemy
• llama.cpp
• HuggingFace Community
• RMIT Race Hub
• Microsoft Phi-3 Series

LICENSE

This project is licensed under the terms of the Apache 2.0 license. See the LICENSE file.