#on-device-ai Repositories
Discover 68 active open source GitHub repositories tagged with #on-device-ai, accumulating 183,992 total stars (averaging 2,705 stars per project). Leading this topic is cactus-compute/cactus with 14,864 stars.
69 repositories total
Device-native foundation models: 145B on a laptop, 35B on a phone. Latent MoE, SSD-streamed weights, a router that predicts loads one step ahead. Mac, iPhone, Android, Windows.
Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code
This repository contains the official implementation of "FastVLM: Efficient Vision Encoding for Vision Language Models" - CVPR 2025
LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.
A human-centric, resource-aware Android node for edge computing and federated learning. Orchestrates distributed intelligence with privacy-first localization and empathetic hardware management.
AI You Control: Choose your models. Own your data. Eliminate vendor lock-in.
Hit your Claude Code usage limit? Keep going free and offline on a local model: Qwen 3.8, Gemma 4, and Qwen 4 the day it ships. Runs on Apple Silicon Macs (MLX), Linux and Windows (llama.cpp) with no API key. Your code stays home.
An AI-powered file management tool that ensures privacy by organizing local texts, images. Using Llama3.2 3B and Llava v1.6 models with the Nexa SDK, it intuitively scans, restructures, and organizes files for quick, seamless access and easy retrieval.
Mano-P: Open-source GUI-VLA agent for edge devices. #1 on OSWorld (specialized, 58.2%). Runs locally on Apple M4 Mac mini/MacBook โ no data leaves your device. Mano-P is an open-source GUI-VLA project that supports local inference on Mac mini/MacBook or via compute stick, enabling purely vision-driven cross-platform GUI automation operations. Data is processed entirely locally, supporting planning and execution of complex multi-step tasks.
Production-grade C++ edge AI engine for video analytics and on-device VLM across Sophon, Rockchip RKNN, and x86, with visual orchestration, real-time OSD, events, and reproducible benchmarks.
Core ML model zoo for iOS/macOS โ PyTorch models converted to ready-to-use .mlpackage, each with a conversion script and SwiftUI sample app. Sibling repos cover Apple's Core AI framework (iOS/macOS 27) and on-device LLMs.
High-performance, privacy-first on-device AI inference library for React Native, powered by PyTorch's ExecuTorch runtime
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
Muesli: agent-native local meeting transcription + dictation for macOS (Granola + WisprFlow + Littlebird alternative)
Free, open-source voice typing for macOS and Windows. Local speech-to-text with Whisper and Parakeet, also works offline.
Local Laya typed decisions on Apple Core ML and Neural Engine. Validated ports, ~5 ms short decisions on M3 Max, reproducible speed and energy benchmarks.
Nativ is an open-source local AI app for Mac. It runs open AI models on Apple Silicon, and nothing leaves your Mac.
Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning โ natively on MLX. Unsloth-compatible API.
MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.
[ICLR 2019] ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
PhoneClaw turns phones into local AI agent runtimes with on-device models, native mobile Skills, LiveLand, and optional Mac Gateway inference.
A practical lab for building, testing, and evaluating apps with Apple's Foundation Models framework.
PokeClaw (PocketClaw) โ first on-device AI that controls your Android phone. Gemma 4, no cloud, no API key. Poke is short for Pocket.
Official PyTorch implementation of "EdgeSAM: Prompt-In-the-Loop Distillation for On-Device Deployment of SAM"
NativeMind: Your fully private, open-source, on-device AI assistant
Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, or edge device. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.
AI PC starter app for doing AI image creation, image stylizing, and chatbot on a PC powered by an Intelยฎ Arcโข GPU.
Sudoless Apple Silicon system monitor (native SwiftUI GUI) with ANE / Media Engine / memory-bandwidth tracking
[CVPR 2025] Official PyTorch implementation of "EdgeTAM: On-Device Track Anything Model"
jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices, and scores with probabilities, on your laptop's CPU.
Local-first, open-source AI assistant for your data. Unify tasks, notes, docs, photos, and bookmarks. Private, self-hosted, and extensible via APIs.
Shared single-file memory layer for all your agents, sub-millisecond RAG over text, photos, and video on Apple Silicon. No server. No API. One file. Pure Swift.
Open-source local AI SDK - run AI on-device with no cloud, no API keys. Supports GGUF, RAG, image, music, and video generation, speech-to-text, P2P inference, and more. Cross-platform: Linux, macOS, Windows, Android, iOS.
Frequently Asked Questions
What is the most popular #on-device-ai repository on GitHub?
The top ranked open source #on-device-ai project on GitHub is cactus-compute/cactus with 14,864 stars. Under-50 MB agentic, voice, and vision intelligence for consumer devices.
Which programming languages are most used in #on-device-ai repositories?
Among the top ranked #on-device-ai projects on GitHub, the most common programming languages are Python, Swift, C++, TypeScript.
How is the #on-device-ai repository ranking calculated?
We track 68 active open source repositories tagged with #on-device-ai (500+ stars), totaling 183,992 stars with an average of 2,705 stars per repository. Rankings update daily from the GitHub API.