#edge-ai Repositories
84 repositories total
Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, or edge device. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.
Run Kimi K3 locally on CPU with ~55GB measured runtime RAM. A single-file Linux inference server powered by cPilot Runtime.
jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices, and scores with probabilities, on your laptop's CPU.
Build AI agents that run 100% on-device. Sub-100ms latency on Qualcomm NPU. Zero cloud dependency.
:zap: Based on Yolo's low-power, ultra-lightweight universal target detection algorithm, the parameter is only 250k, and the speed of the smart phone mobile terminal can reach ~300fps+.
AMD Ryzenโข AI Software includes the tools and runtime libraries for optimizing and deploying AI inference on AMD Ryzenโข AI powered PCs.
DefraDB is a Peer-to-Peer Edge-First Database. It's the core data storage system for the Source Ecosystem.
Android 17 local LLM prototype with Jetpack Compose and ONNX Runtime for offline AI inference experiments.
sanoTTS (เคธเคพเคจเฅ = โsmallโ in Nepali): a 294kโ2.27M-parameter neural TTS system that runs on a $3 chip, in the browser, or in Python, with low-latency streaming.
Vui Nano โ a small, context-aware text-to-speech model trained on real conversations. 219M active parameters (305M total), Apache 2.0, voice cloning, streaming, runs on CPU (dependency-free C build). Includes a full real-time voice assistant: WebRTC, ASR, local LLM, OpenAI Realtime API compatible.
The Hailo Model Zoo includes pre-trained models and a full building and evaluation environment
EfficientSAM3 compresses SAM3 into lightweight, edge-friendly models via progressive knowledge distillation for fast promptable concept segmentation and tracking.
A simple demo of yolov5s running on rk3588/3588s using c++ (about 142 frames). / A simple demo of yolov5s running on rk3588/3588s using c++ (about 142 frames).
On-device AI for Flutter โ Gemma and other LLMs, multimodal, function calling, embeddings, RAG and speech on Android, iOS, Web, macOS, Windows and Linux.
AI-in-a-Box leverages the expertise of Microsoft across the globe to develop and provide AI and ML solutions to the technical community. Our intent is to present a curated collection of solution accelerators that can help engineers establish their AI/ML environments and solutions rapidly and with minimal friction.
A curated list of awesome edge computing, including frameworks, simulators, tools, etc.
Local-first AI-powered document intelligence platform for investigative journalism