Git Stars Git Stars

#cpu-inference Repositories

Discover 17 active open source GitHub repositories tagged with #cpu-inference, accumulating 84,398 total stars (averaging 4,964 stars per project). Leading this topic is microsoft/BitNet with 42,358 stars.

17
Total Repositories
84,398
Total Stars
4,964
Avg Stars/Repo
Programming Languages: Python C++ C HTML Rust
1
microsoft/BitNet

Official inference framework for 1-bit LLMs

microsoft
โญ 42,358 C++ ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-07-27 ๐Ÿ“… Created 2024-08-05
2
FareedKhan-dev/kimi-k3-in-c

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

FareedKhan-dev
โญ 12,523 C ๐Ÿ‘ค FareedKhan-dev User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2026-08-01
3
google/gemma.cpp

lightweight, standalone C++ inference engine for Google's Gemma models.

google
โญ 7,089 C++ ๐Ÿ‘ค google Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-02-13
4
xxnuo/MTranServer

Offline translation model server with low resource consumption, fast speed, and private deployment capability. ไฝŽ่ต„ๆบๅ ็”จ้€Ÿๅบฆๅฟซๅฏ็งๆœ‰้ƒจ็ฝฒ็š„็ฆป็บฟ็ฟป่ฏ‘ๆจกๅž‹ๆœๅŠกๅ™จ

xxnuo
โญ 4,758 C++ ๐Ÿ‘ค xxnuo User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2025-02-06
5
OpenMOSS/MOSS-TTS-Nano

A 100M-parameter multilingual TTS model for real-time CPU inference, voice cloning, and 48 kHz stereo generation

OpenMOSS
โญ 4,711 Python ๐Ÿ‘ค OpenMOSS Organization ๐Ÿ• Updated 2026-09-06 ๐Ÿ“… Created 2026-04-10
6
fastino-ai/GLiNER2

Unified Schema-Based Information Extraction

fastino-ai
โญ 2,490 Python ๐Ÿ‘ค fastino-ai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2025-07-07
โญ 1,682 Python ๐Ÿ‘ค pipecat-ai Organization ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2025-03-05
8
dleemiller/WordLlama

Things you can do with the token embeddings of an LLM

dleemiller
โญ 1,448 Python ๐Ÿ‘ค dleemiller User ๐Ÿ• Updated 2025-12-01 ๐Ÿ“… Created 2024-06-12
9
synchopate/llama-modchip

Zero-GPU CPU inference for 30Bโ€“284B MoE models (Qwen3.8 176.5B cold-cache certified in 28.26/48.35 GiB Cgroup v2, DeepSeek-V4 284B, Nemotron-30B) + RouteCrack 2.0.

synchopate
โญ 1,398 Python ๐Ÿ‘ค synchopate User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-29
10
wangeai/K3Flight

Run Kimi K3 locally on CPU with ~55GB measured runtime RAM. A single-file Linux inference server powered by cPilot Runtime.

wangeai
โญ 1,019 HTML ๐Ÿ‘ค wangeai Organization ๐Ÿ• Updated 2026-07-31 ๐Ÿ“… Created 2026-07-30
11
feder-cr/jev

jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices, and scores with probabilities, on your laptop's CPU.

feder-cr
โญ 970 C++ ๐Ÿ‘ค feder-cr User ๐Ÿ• Updated 2026-10-18 ๐Ÿ“… Created 2024-08-15
12
Phocinae/Phocinae-Largha-150M-v1

Spotted seal โ€” Phocinae-Largha-150M-v1: a 144.3M bilingual decision model. Tiny model. Big decisions. Little seal, big decisions.

Phocinae
โญ 797 Python ๐Ÿ‘ค Phocinae Organization ๐Ÿ• Updated 2026-10-20 ๐Ÿ“… Created 2026-10-08
13
fluxions-ai/vui

Vui Nano โ€” a small, context-aware text-to-speech model trained on real conversations. 219M active parameters (305M total), Apache 2.0, voice cloning, streaming, runs on CPU (dependency-free C build). Includes a full real-time voice assistant: WebRTC, ASR, local LLM, OpenAI Realtime API compatible.

fluxions-ai
โญ 782 Python ๐Ÿ‘ค fluxions-ai Organization ๐Ÿ• Updated 2026-10-10 ๐Ÿ“… Created 2025-06-05
14
prabanta-dev/janas

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

prabanta-dev
โญ 738 C ๐Ÿ‘ค prabanta-dev Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-09-20
15
ai25395/FMatPix

A free portable Formula Ocr tool supporting latex and mathml

ai25395
โญ 614 Python ๐Ÿ‘ค ai25395 User ๐Ÿ• Updated 2025-04-21 ๐Ÿ“… Created 2024-10-20
16
lucienhuangfu/eLLM

eLLM: Run Long-Horizon Inference Faster on CPUs Than on GPUs

lucienhuangfu
โญ 602 Rust ๐Ÿ‘ค lucienhuangfu User ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2025-06-30
17
brontoguana/krasis

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

brontoguana
โญ 530 C++ ๐Ÿ‘ค brontoguana User ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2026-02-09

Frequently Asked Questions

1

What is the most popular #cpu-inference repository on GitHub?

The top ranked open source #cpu-inference project on GitHub is microsoft/BitNet with 42,358 stars. Official inference framework for 1-bit LLMs

2

Which programming languages are most used in #cpu-inference repositories?

Among the top ranked #cpu-inference projects on GitHub, the most common programming languages are Python, C++, C, HTML.

3

How is the #cpu-inference repository ranking calculated?

We track 17 active open source repositories tagged with #cpu-inference (500+ stars), totaling 84,398 stars with an average of 4,964 stars per repository. Rankings update daily from the GitHub API.