#gguf Repositories
Discover 49 active open source GitHub repositories tagged with #gguf, accumulating 165,186 total stars (averaging 3,371 stars per project). Leading this topic is AlexsJones/llmfit with 39,174 stars.
50 repositories total
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
This is a simple technical็งๆฎ tutorial project, mainly focusing on explaining some interesting, cutting-edge technical concepts and principles. Each article strives to be completed within 5 minutes of reading.
๐ก๏ธ BLA (bla-agents / Blind LLM Agents) โ Open-source Blue Defense MCP server & CLI protecting JS/TS, Electron ASAR, WASM, Python, native binaries, SQLite, GGUF/Safetensors, and Android DEX from AI reverse-engineering agents (REA).
โก Pure-Rust WebGPU inference engine โ OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.
Atomic Agent is a local-first AI agent. Runs open-weight models on your own machine via llama.cpp.
The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac โ GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.
Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B), and Hy4-Flash (220B) on your own GPUs โ starting from 8 GB of VRAM, with NVIDIA and AMD support, one dashboard, and an OpenAI- and Anthropic-compatible API.
Maid is a free and open source application for interfacing with llama.cpp models locally, and with Anthropic, DeepSeek, Ollama, Mistral and OpenAI models remotely.
Hands-on Ollama, Deploy Large Models with Your CPU, Online Reading Address: https://datawhalechina.github.io/handy-ollama/
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.
LLM Agent Framework in ComfyUI includes MCP sever, Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llms with similar openai / aisuite interfaces, such as o1,ollama, gemini, grok, qwen, GLM, deepseek, kimi,doubao. Adapted to local llms, vlm, gguf such as llama-3.3 Janus-Pro, Linkage graphRAG
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level.
Local AI app and inference engine for agents. Run open-weight LLMs locally โ private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge
Fun-ASR speech recognition models, with native Hugging Face Transformers support for Fun-ASR-Nano and separate FunASR, vLLM and llama.cpp deployment paths.
An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.
Zero-GPU CPU inference for 30Bโ284B MoE models (Qwen3.8 176.5B cold-cache certified in 28.26/48.35 GiB Cgroup v2, DeepSeek-V4 284B, Nemotron-30B) + RouteCrack 2.0.
Open-source no-root Android real-time screen translator for games, visual novels, and manga. Supports on-device and cloud OCR, offline LLMs, multiple translation services, on-screen translations, and text-to-speech (TTS).
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder, and MiniCPM-V on a second engine built for PolyStrata. One-click installation for Windows and Linux, with an OpenAI/Anthropic API on localhost.
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, or edge device. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.
Rewrites AI drafts so they read like a person wrote them โ runs locally, keeps the facts
jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices, and scores with probabilities, on your laptop's CPU.
A CLI to estimate inference memory requirements for Hugging Face models, written in Python.
Open multimodal decision models, 0.8B to 27B, for text, images and video. Decide with probabilities over your own candidates and locate with bounding boxes, in place of generated text. Free API, ONNX, GGUF, vLLM and SGLang; fine-tuning code included.
LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.
37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, an iPhone app, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows, macOS, iOS, and Linux with full GPU capability.
Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.
ComfyUI nodes for vision-language models: Qwen3-VL, Moondream 3, Florence-2, SmolVLM2, InternVL, Gemma 3, MiniCPM-V. Plus open-vocabulary detection, SAM2/SAM3 segmentation, video temporal reasoning, GGUF via llama.cpp, and hosted LLM/VLM APIs.
Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files run on int8 and 4-bit tensor cores behind the /v1/systemone API, delivering up to 5.3ร lower latency than vLLM in FP8 with bit-identical answers.
Xolotl: run DeepSeek-V4.1-Flash, all 552B parameters, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s agent, ~41 fast, 512K context. Plus Inkling-2-Mini (Thinking Machines, own engine + MTP drafter), GLM-5.3-Flash, Qwen3.8, LFM3-Omni. MoE experts tiered across VRAM, RAM, and NVMe (Xolotl Direct). OpenAI + Anthropic API.
Local inference for diffusion language models (LLaDA, Dream), written in Mojo
Speech synthesis, speech recognition and voice activity detection for voice conversation, in C++ on ggml: Qwen3-TTS, Irodori-TTS, Qwen3-ASR, ReazonSpeech, parakeet and Silero VAD on Metal, Vulkan and the CPU, with an OpenAI-compatible server and live transcription.
Arabic-first generative speech recognition โ Audar-ASR-V1 (Flash + Turbo). #1 on the Open Universal Arabic ASR Leaderboard. Model cards, benchmarks & inference.
Frequently Asked Questions
What is the most popular #gguf repository on GitHub?
The top ranked open source #gguf project on GitHub is AlexsJones/llmfit with 39,174 stars. Hundreds of models & providers. One command to find what runs on your hardware.
Which programming languages are most used in #gguf repositories?
Among the top ranked #gguf projects on GitHub, the most common programming languages are Python, C++, TypeScript, Rust.
How is the #gguf repository ranking calculated?
We track 49 active open source repositories tagged with #gguf (500+ stars), totaling 165,186 stars with an average of 3,371 stars per repository. Rankings update daily from the GitHub API.