#local-llm Repositories
122 repositories total
Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning โ natively on MLX. Unsloth-compatible API.
๐ Give a soul to your favorite characters. Soul of Waifu is a desktop roleplay & AI companion app featuring Live2D/VRM avatars, voice chat & local LLM. Evolve together across immersive chat, RPG adventures, and your desktop.
Open-source, privacy-first Voice of Customer analytics. A self-hosted alternative to Enterpret and Unwrap that strips PII from feedback before any LLM sees it.
Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.
Zero-GPU CPU inference for 30Bโ284B MoE models (Qwen3.8 176.5B cold-cache certified in 28.26/48.35 GiB Cgroup v2, DeepSeek-V4 284B, Nemotron-30B) + RouteCrack 2.0.
A simple toy demo of a local voice assistant with whisper and large language model.
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder, and MiniCPM-V on a second engine built for PolyStrata. One-click installation for Windows and Linux, with an OpenAI/Anthropic API on localhost.
Open-source autonomous AI penetration testing. 50 specialist agents across web, API, cloud, Active Directory, Kubernetes, CI/CD and AI/LLM (OWASP LLM Top 10), with an evidence-checked attack-path graph. Runs on a local LLM behind a Privacy Gateway. Integrates with GitHub, GitLab, Jenkins, VS Code, JetBrains, n8n, Grafana & Splunk.
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Open-source AI companions, desktop pets, long-term memory & proactive chat. A comprehensive directory of open-source human-AI romance projects. Enable your local machine AI to exist independently of official platforms, equipped with memory and proactivity.
Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable
Provides automated reverse engineering assistance through the use of local large language models (LLMs) on consumer hardware.
PokeClaw (PocketClaw) โ first on-device AI that controls your Android phone. Gemma 4, no cloud, no API key. Poke is short for Pocket.
Maximizing the ability to run different LLMs on the smallest devices possible
Build AI agents from first principles using a local LLM - no frameworks, no cloud APIs, no hidden reasoning.
Own the flow: one open-source AI platform for your knowledge, models and channels โ web, WhatsApp, email, Outlook, Nextcloud & ownCloud. Local and cloud models side by side, with the cost on every answer.
Capri โ an Edgerunners-inspired AI agent for your desktop. Supports Claude 3.7, Claude Code, OpenAI, and local Ollama/LLaMA. A customizable AI waifu companion with voice chat.
Sudoless Apple Silicon system monitor (native SwiftUI GUI) with ANE / Media Engine / memory-bandwidth tracking
Run Kimi K3 locally on CPU with ~55GB measured runtime RAM. A single-file Linux inference server powered by cPilot Runtime.
Rewrites AI drafts so they read like a person wrote them โ runs locally, keeps the facts
Bridge between Ollama and MCP servers, enabling local LLMs to use Model Context Protocol tools
jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices, and scores with probabilities, on your laptop's CPU.
One-command installer for a self-hosted AI stack on your own server: n8n, Ollama, Open WebUI, OpenClaw, Dify, Flowise, Supabase, ComfyUI, Qdrant & 30+ tools. Docker Compose, automatic HTTPS via Caddy, built-in monitoring. Free Zapier/Make alternative.
Nornicdb is a distributed low-latency, Graph+Vector, Temporal MVCC with all sub-ms HNSW search, graph traversal, and writes. Using Neo4j Bolt/Cypher and qdrant's gRPC means you can switch with no changes while adding intelligent features like schemas, managed embeddings, reranking+llm, GPU accel, Auto-TLP, Policy-based Memory Decay, and MCP server.
Build a RAG (Retrieval Augmented Generation) pipeline from scratch and have it all run locally.
Voice input for macOS with local models: transcribe, translate, and rewrite in any app. The successor to Voxt.
Open-source desktop app for AI-native game development. Build in Unreal Engine, Unity, Godot, Roblox, or Three.js with any AI, including local models.
Proxy that allows you to use ollama as a copilot like Github copilot
Android 17 local LLM prototype with Jetpack Compose and ONNX Runtime for offline AI inference experiments.
Run full Kimi K3 on a single device. And an OpenAI-compatible API server for local chat and coding agents.
Karpathyโs LLM Wiki, 100% local with Ollama. Drop Markdown notes โ AI extracts concepts โ your Obsidian wiki auto-links and grows. Zero sharing. Your notes stay yours.
Blazing-fast Apple Silicon inference engine optimized for Qwen & Nemotron. Compatible with the OpenAI and Anthropic APIs. Built by the team at nace.ai
Harness the power of local LLMs with this TUI MCP Client for Ollama. Featuring all core MCP primitives (tools, prompts, resources), agent mode, multi-server, model switching, streaming responses, human-in-the-loop, thinking mode, model params config, system prompts, and saved preferences.
Open-source local AI SDK - run AI on-device with no cloud, no API keys. Supports GGUF, RAG, image, music, and video generation, speech-to-text, P2P inference, and more. Cross-platform: Linux, macOS, Windows, Android, iOS.
Microsoft's GraphRAG + AutoGen + Ollama + Chainlit = Fully Local & Free Multi-Agent RAG Superbot
Up to 4ร faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding โ Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.
37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.