llm-inference
Here are 18 public repositories matching this topic...
Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
-
Updated
Mar 15, 2024 - C++
Super easy to use library for doing LLaMA/GPT-J stuff! - Mirror of: https://gitlab.com/niansa/libjustlm
-
Updated
Mar 25, 2024 - C++
Multi-Model and multi-tasking llama Discord Bot - Mirror of: https://gitlab.com/niansa/discord_llama
-
Updated
Mar 27, 2024 - C++
CodeInferflow is a efficient inference engine based on Inferflow for code large language models (Code LLMs). With CodeInferflow, you can locally deploy popular code LLMs and efficiently use code completion in VSCode.
-
Updated
Jun 14, 2024 - C++
LLM in Godot
-
Updated
Jun 23, 2024 - C++
Leverage tensor parallelism techniques to run large language models in the CPU memory of edge devices.
-
Updated
Aug 9, 2024 - C++
Tensor parallelism is all you need. Run LLMs on an AI cluster at home using any device. Distribute the workload, divide RAM usage, and increase inference speed.
-
Updated
Aug 10, 2024 - C++
校招、秋招、春招、实习好项目,带你从零动手实现支持LLama的大模型推理框架。
-
Updated
Aug 19, 2024 - C++
DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including x86 and ARMv9.
-
Updated
Aug 27, 2024 - C++
Pure C++ implementation of several models for real-time chatting on your computer (CPU)
-
Updated
Aug 28, 2024 - C++
Aussie AI Base C++ Library is the source code repo for the book Generative AI in C++, along with various other AI/ML kernels.
-
Updated
Aug 30, 2024 - C++
LLMs as Copilots for Theorem Proving in Lean
-
Updated
Sep 2, 2024 - C++
High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
-
Updated
Sep 6, 2024 - C++
OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
-
Updated
Sep 6, 2024 - C++
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
-
Updated
Sep 7, 2024 - C++
A high-performance inference system for large language models, designed for production environments.
-
Updated
Sep 7, 2024 - C++
Improve this page
Add a description, image, and links to the llm-inference topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-inference topic, visit your repo's landing page and select "manage topics."