#llama-cpp Repositories
69 repositories total
llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
Blazing-fast Apple Silicon inference engine optimized for Qwen & Nemotron. Compatible with the OpenAI and Anthropic APIs. Built by the team at nace.ai
Orion-14B is a family of models includes a 14B foundation LLM, and a series of models: a chat model, a long context model, a quantized model, a RAG fine-tuned model, and an Agent fine-tuned model. The Orion-14B model series includes a 14 billion parameter multilingual foundation LLM and a series of related derivative models, including a chat model, a long context model, a quantized model, a RAG fine-tuned model, and an Agent fine-tuned model, etc.
ComfyUI nodes for vision-language models: Qwen3-VL, Moondream 3, Florence-2, SmolVLM2, InternVL, Gemma 3, MiniCPM-V. Plus open-vocabulary detection, SAM2/SAM3 segmentation, video temporal reasoning, GGUF via llama.cpp, and hosted LLM/VLM APIs.
An intelligent coding assistant plugin for Visual Studio Code, developed based on CodeShell
Open-source, bring-your-own-key (BYOK) multi-model AI chat client for iOS, Android and the web. One LLM client for OpenAI, Anthropic, Google Gemini, OpenRouter, DeepSeek and ten more providers, plus any OpenAI-, Anthropic- or Gemini-compatible endpoint โ Ollama, LM Studio, llama.cpp, vLLM. Local-first, self-hostable, no account. AGPL-3.0.
sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems. Live support: https://discord.com/invite/GH5kRgv6ZD
An easier way to use AI on your own computer. A free, open-source alternative to Claude and ChatGPT that runs open-weight models like Gemma and gpt-oss locally on Mac, Windows and Linux. Private, and works offline.
GraphRAG-rs is a high-performance, state-of-the-art Rust implementation of GraphRAG (Graph-based Retrieval Augmented Generation) that builds knowledge graphs from documents and enables natural language querying with configurable entity extraction and local LLM integration
A local-first, full-stack AI creative workstation: five modulesโAI chat, AI comics, comic drama creation, writing desk, and knowledge learning; multi-engine local inference with vLLM / llama.cpp / Transformers, intelligent VRAM scheduling, bulkhead isolation and self-healing. For learning purposes only; feedback and discussion are welcome. (The latest progress, excluding the plugin system, cannot currently be committed to the repository due to protocol restrictions.)
A pure Rust-based inference engine for LLMs, VLMs, VLAs, TTS, and OCR, powered by Candle and Rust. An alternative to llama.cpp that's much simpler and cleaner.
Arabic-first generative speech recognition โ Audar-ASR-V1 (Flash + Turbo). #1 on the Open Universal Arabic ASR Leaderboard. Model cards, benchmarks & inference.