Git Stars Git Stars

#gguf Repositories

Discover 49 active open source GitHub repositories tagged with #gguf, accumulating 165,186 total stars (averaging 3,371 stars per project). Leading this topic is AlexsJones/llmfit with 39,174 stars.

49
Total Repositories
165,186
Total Stars
3,371
Avg Stars/Repo
1
AlexsJones/llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

AlexsJones
โญ 39,174 Rust ๐Ÿ‘ค AlexsJones User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2026-02-15
2
mozilla-ai/llamafile

Distribute and run LLMs with a single file.

mozilla-ai
โญ 26,681 C++ ๐Ÿ‘ค mozilla-ai Organization ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2023-09-10
3
MakazhanAlpamys/Soup

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

MakazhanAlpamys
โญ 10,703 Python ๐Ÿ‘ค MakazhanAlpamys User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-02-20
4
karminski/one-small-step

This is a simple technical็ง‘ๆ™ฎ tutorial project, mainly focusing on explaining some interesting, cutting-edge technical concepts and principles. Each article strives to be completed within 5 minutes of reading.

karminski
โญ 7,006 Python ๐Ÿ‘ค karminski User ๐Ÿ• Updated 2026-03-08 ๐Ÿ“… Created 2025-01-16
5
qybaihe/bla

๐Ÿ›ก๏ธ BLA (bla-agents / Blind LLM Agents) โ€” Open-source Blue Defense MCP server & CLI protecting JS/TS, Electron ASAR, WASM, Python, native binaries, SQLite, GGUF/Safetensors, and Android DEX from AI reverse-engineering agents (REA).

qybaihe
โญ 6,366 TypeScript ๐Ÿ‘ค qybaihe User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-25
6
Michael-A-Kuykendall/shimmy

โšก Pure-Rust WebGPU inference engine โ€” OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

Michael-A-Kuykendall
โญ 6,020 Rust ๐Ÿ‘ค Michael-A-Kuykendall User ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2025-08-28
7
ngxson/smolvlm-realtime-webcam

Real-time webcam demo with SmolVLM and llama.cpp server

ngxson
โญ 5,555 HTML ๐Ÿ‘ค ngxson User ๐Ÿ• Updated 2025-05-12 ๐Ÿ“… Created 2025-05-12
8
city96/ComfyUI-GGUF

GGUF Quantization support for native ComfyUI models

city96
โญ 4,284 Python ๐Ÿ‘ค city96 User ๐Ÿ• Updated 2026-01-12 ๐Ÿ“… Created 2024-08-15
9
AtomicBot-ai/atomic-agent

Atomic Agent is a local-first AI agent. Runs open-weight models on your own machine via llama.cpp.

AtomicBot-ai
โญ 3,589 TypeScript ๐Ÿ‘ค AtomicBot-ai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-04-21
10
off-grid-ai/OGAM

The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac โ€” GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.

off-grid-ai
โญ 3,406 TypeScript ๐Ÿ‘ค off-grid-ai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-01-29
11
mw00/project-maya

Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B), and Hy4-Flash (220B) on your own GPUs โ€” starting from 8 GB of VRAM, with NVIDIA and AMD support, one dashboard, and an OpenAI- and Anthropic-compatible API.

mw00
โญ 3,140 C++ ๐Ÿ‘ค mw00 User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-06
12
Mobile-Artificial-Intelligence/maid

Maid is a free and open source application for interfacing with llama.cpp models locally, and with Anthropic, DeepSeek, Ollama, Mistral and OpenAI models remotely.

Mobile-Artificial-Intelligence
โญ 2,785 TypeScript ๐Ÿ‘ค Mobile-Artificial-Intelligence Organization ๐Ÿ• Updated 2026-10-29 ๐Ÿ“… Created 2023-10-17
13
datawhalechina/handy-ollama

Hands-on Ollama, Deploy Large Models with Your CPU, Online Reading Address: https://datawhalechina.github.io/handy-ollama/

datawhalechina
โญ 2,560 Jupyter Notebook ๐Ÿ‘ค datawhalechina Organization ๐Ÿ• Updated 2026-01-15 ๐Ÿ“… Created 2024-07-30
14
ddalcu/mlx-serve

Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.

ddalcu
โญ 2,528 Zig ๐Ÿ‘ค ddalcu User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-02-17
15
heshengtao/comfyui_LLM_party

LLM Agent Framework in ComfyUI includes MCP sever, Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llms with similar openai / aisuite interfaces, such as o1,ollama, gemini, grok, qwen, GLM, deepseek, kimi,doubao. Adapted to local llms, vlm, gguf such as llama-3.3 Janus-Pro, Linkage graphRAG

heshengtao
โญ 2,393 Python ๐Ÿ‘ค heshengtao User ๐Ÿ• Updated 2026-07-29 ๐Ÿ“… Created 2024-04-13
16
withcatai/node-llama-cpp

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level.

withcatai
โญ 2,207 TypeScript ๐Ÿ‘ค withcatai Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2023-08-12
17
handy-computer/transcribe.cpp

ggml speech-to-text inference for 16+ model families

handy-computer
โญ 2,194 C++ ๐Ÿ‘ค handy-computer Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-04-07
18
AtomicBot-ai/Atomic-Chat

Local AI app and inference engine for agents. Run open-weight LLMs locally โ€” private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V

AtomicBot-ai
โญ 2,040 TypeScript ๐Ÿ‘ค AtomicBot-ai Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2026-03-31
19
sammcj/gollama

Go manage your Ollama models

sammcj
โญ 1,849 Go ๐Ÿ‘ค sammcj User ๐Ÿ• Updated 2026-10-21 ๐Ÿ“… Created 2024-05-30
20
StartLuxLabs/StartLux-Decision

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.

StartLuxLabs
โญ 1,780 Python ๐Ÿ‘ค StartLuxLabs Organization ๐Ÿ• Updated 2026-10-23 ๐Ÿ“… Created 2026-09-29
21
intel/auto-round

PTQ-Level Efficiency, QAT-Level Accuracy

intel
โญ 1,706 Python ๐Ÿ‘ค intel Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-01-04
22
nobodywho-ooo/nobodywho

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

nobodywho-ooo
โญ 1,699 Rust ๐Ÿ‘ค nobodywho-ooo Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2024-09-03
23
LlamaEdge/LlamaEdge

The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge

LlamaEdge
โญ 1,665 Rust ๐Ÿ‘ค LlamaEdge Organization ๐Ÿ• Updated 2026-02-08 ๐Ÿ“… Created 2023-10-09
24
QwenAudio/Fun-ASR

Fun-ASR speech recognition models, with native Hugging Face Transformers support for Fun-ASR-Nano and separate FunASR, vLLM and llama.cpp deployment paths.

QwenAudio
โญ 1,614 C ๐Ÿ‘ค QwenAudio Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2025-12-15
25
kitops-ml/kitops

An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.

kitops-ml
โญ 1,451 Go ๐Ÿ‘ค kitops-ml Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-02-02
26
edwko/OuteTTS

Interface for OuteTTS models.

edwko
โญ 1,440 Python ๐Ÿ‘ค edwko User ๐Ÿ• Updated 2026-03-23 ๐Ÿ“… Created 2024-11-04
27
synchopate/llama-modchip

Zero-GPU CPU inference for 30Bโ€“284B MoE models (Qwen3.8 176.5B cold-cache certified in 28.26/48.35 GiB Cgroup v2, DeepSeek-V4 284B, Nemotron-30B) + RouteCrack 2.0.

synchopate
โญ 1,398 Python ๐Ÿ‘ค synchopate User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-29
28
ciddwd/overlay-translator

Open-source no-root Android real-time screen translator for games, visual novels, and manga. Supports on-device and cloud OCR, offline LLMs, multiple translation services, on-screen translations, and text-to-speech (TTS).

ciddwd
โญ 1,369 Kotlin ๐Ÿ‘ค ciddwd User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2026-06-22
29
simplexai-labs/PolyStrata

Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder, and MiniCPM-V on a second engine built for PolyStrata. One-click installation for Windows and Linux, with an OpenAI/Anthropic API on localhost.

simplexai-labs
โญ 1,324 C++ ๐Ÿ‘ค simplexai-labs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2026-10-06
30
wici-ai/inferssd

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

wici-ai
โญ 1,277 C++ ๐Ÿ‘ค wici-ai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-10-22
31
Helldez/BigMoeOnEdge

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, or edge device. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

Helldez
โญ 1,066 C++ ๐Ÿ‘ค Helldez User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-07-10
32
jegly/Box

The most advanced, fully offline client-side AI suite on Android today.

jegly
โญ 1,035 Kotlin ๐Ÿ‘ค jegly User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-04-17
33
sgaofen/humanizer-local-model

Rewrites AI drafts so they read like a person wrote them โ€” runs locally, keeps the facts

sgaofen
โญ 1,010 Python ๐Ÿ‘ค sgaofen User ๐Ÿ• Updated 2026-10-19 ๐Ÿ“… Created 2026-09-11
34
feder-cr/jev

jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices, and scores with probabilities, on your laptop's CPU.

feder-cr
โญ 970 C++ ๐Ÿ‘ค feder-cr User ๐Ÿ• Updated 2026-10-18 ๐Ÿ“… Created 2024-08-15
35
alvarobartt/hf-mem

A CLI to estimate inference memory requirements for Hugging Face models, written in Python.

alvarobartt
โญ 956 Python ๐Ÿ‘ค alvarobartt User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2025-12-22
36
Valen-Team/Valen

Open multimodal decision models, 0.8B to 27B, for text, images and video. Decide with probabilities over your own candidates and locate with bounding boxes, in place of generated text. Free API, ONNX, GGUF, vLLM and SGLang; fine-tuning code included.

Valen-Team
โญ 932 Python ๐Ÿ‘ค Valen-Team Organization ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2026-09-23
37
eastriverlee/LLM.swift

LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.

eastriverlee
โญ 883 Swift ๐Ÿ‘ค eastriverlee User ๐Ÿ• Updated 2026-07-19 ๐Ÿ“… Created 2023-12-02
38
mukel/llama3.java

Llama 3+ inference in pure Java

mukel
โญ 818 Java ๐Ÿ‘ค mukel User ๐Ÿ• Updated 2026-04-24 ๐Ÿ“… Created 2024-04-25
39
SimpleBerry/LLaMA-O1

Large Reasoning Models

SimpleBerry
โญ 806 Python ๐Ÿ‘ค SimpleBerry Organization ๐Ÿ• Updated 2024-12-03 ๐Ÿ“… Created 2024-10-10
40
prabanta-dev/janas

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

prabanta-dev
โญ 738 C ๐Ÿ‘ค prabanta-dev Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-09-20
41
zhongkaifu/TensorSharp

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, an iPhone app, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows, macOS, iOS, and Linux with full GPU capability.

zhongkaifu
โญ 681 C# ๐Ÿ‘ค zhongkaifu User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-04-03
42
hybridgroup/yzma

Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.

hybridgroup
โญ 677 Go ๐Ÿ‘ค hybridgroup Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2025-09-21
43
gokayfem/ComfyUI_VLM_nodes

ComfyUI nodes for vision-language models: Qwen3-VL, Moondream 3, Florence-2, SmolVLM2, InternVL, Gemma 3, MiniCPM-V. Plus open-vocabulary detection, SAM2/SAM3 segmentation, video temporal reasoning, GGUF via llama.cpp, and hosted LLM/VLM APIs.

gokayfem
โญ 592 Python ๐Ÿ‘ค gokayfem User ๐Ÿ• Updated 2026-08-08 ๐Ÿ“… Created 2024-01-24
44
kelindar/search

Go library for embedded vector search and semantic embeddings using llama.cpp

kelindar
โญ 561 Go ๐Ÿ‘ค kelindar User ๐Ÿ• Updated 2026-09-19 ๐Ÿ“… Created 2024-09-29
45
StartLuxLabs/StartLux-Prism

Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files run on int8 and 4-bit tensor cores behind the /v1/systemone API, delivering up to 5.3ร— lower latency than vLLM in FP8 with bit-identical answers.

StartLuxLabs
โญ 560 Python ๐Ÿ‘ค StartLuxLabs Organization ๐Ÿ• Updated 2026-10-27 ๐Ÿ“… Created 2026-10-13
46
Indras-Mirror/Xolotl

Xolotl: run DeepSeek-V4.1-Flash, all 552B parameters, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s agent, ~41 fast, 512K context. Plus Inkling-2-Mini (Thinking Machines, own engine + MTP drafter), GLM-5.3-Flash, Qwen3.8, LFM3-Omni. MoE experts tiered across VRAM, RAM, and NVMe (Xolotl Direct). OpenAI + Anthropic API.

Indras-Mirror
โญ 549 C++ ๐Ÿ‘ค Indras-Mirror User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-06
47
dllm-labs/mojo-dllm

Local inference for diffusion language models (LLaDA, Dream), written in Mojo

dllm-labs
โญ 546 Python ๐Ÿ‘ค dllm-labs Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-10-04
48
nyosegawa/speech.cpp

Speech synthesis, speech recognition and voice activity detection for voice conversation, in C++ on ggml: Qwen3-TTS, Irodori-TTS, Qwen3-ASR, ReazonSpeech, parakeet and Silero VAD on Metal, Vulkan and the CPU, with an OpenAI-compatible server and live transcription.

nyosegawa
โญ 510 C++ ๐Ÿ‘ค nyosegawa User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-09-29
49
AudarAI/Audar-ASR-V1

Arabic-first generative speech recognition โ€” Audar-ASR-V1 (Flash + Turbo). #1 on the Open Universal Arabic ASR Leaderboard. Model cards, benchmarks & inference.

AudarAI
โญ 503 Python ๐Ÿ‘ค AudarAI Organization ๐Ÿ• Updated 2026-08-22 ๐Ÿ“… Created 2026-07-06
50
withcatai/catai

Run an AI โœจ assistant locally! with a simple API for Node.js ๐Ÿš€

withcatai
โญ 500 TypeScript ๐Ÿ‘ค withcatai Organization ๐Ÿ• Updated 2025-11-16 ๐Ÿ“… Created 2023-03-25

Frequently Asked Questions

1

What is the most popular #gguf repository on GitHub?

The top ranked open source #gguf project on GitHub is AlexsJones/llmfit with 39,174 stars. Hundreds of models & providers. One command to find what runs on your hardware.

2

Which programming languages are most used in #gguf repositories?

Among the top ranked #gguf projects on GitHub, the most common programming languages are Python, C++, TypeScript, Rust.

3

How is the #gguf repository ranking calculated?

We track 49 active open source repositories tagged with #gguf (500+ stars), totaling 165,186 stars with an average of 3,371 stars per repository. Rankings update daily from the GitHub API.