Git Stars Git Stars

#kv-cache Repositories

Discover 18 active open source GitHub repositories tagged with #kv-cache, accumulating 46,401 total stars (averaging 2,577 stars per project). Leading this topic is LMCache/LMCache with 12,405 stars.

18
Total Repositories
46,401
Total Stars
2,577
Avg Stars/Repo
1
LMCache/LMCache

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

LMCache
โญ 12,405 Python ๐Ÿ‘ค LMCache Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-05-28
2
bojieli/ai-infra-book

ใ€ŠIn-Depth Understanding of AI Infrastructure: Quantitative Analysis and System Designใ€‹ (by Li Bojie), an open-source book manuscript: Starting from hardware constraints and model architectures, it quantitatively derives the design of LLM inference and training systems. Includes the full text, PDF, accompanying computational tools, and experiments.

bojieli
โญ 8,179 Python ๐Ÿ‘ค bojieli User ๐Ÿ• Updated 2026-10-01 ๐Ÿ“… Created 2026-08-22
3
sgl-project/mini-sglang

A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.

sgl-project
โญ 5,623 Python ๐Ÿ‘ค sgl-project Organization ๐Ÿ• Updated 2026-05-17 ๐Ÿ“… Created 2025-09-01
4
HDT3213/godis

A Golang implemented Redis Server and Cluster. Redis Server and Distributed Cluster implemented in Go language

HDT3213
โญ 3,834 Go ๐Ÿ‘ค HDT3213 User ๐Ÿ• Updated 2025-09-14 ๐Ÿ“… Created 2019-06-01
5
syv-ai/HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

syv-ai
โญ 2,778 Python ๐Ÿ‘ค syv-ai Organization ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2026-08-15
6
AI-Efficiency/Awesome-Model-Quantization

A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.

AI-Efficiency
โญ 2,471 ๐Ÿ‘ค AI-Efficiency Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2018-10-18
7
datawhalechina/zero-to-sglang

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

datawhalechina
โญ 2,306 Python ๐Ÿ‘ค datawhalechina Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2026-08-27
8
Zefan-Cai/KVCache-Factory

Unified KV Cache Compression Methods for Auto-Regressive Models

Zefan-Cai
โญ 1,393 Python ๐Ÿ‘ค Zefan-Cai User ๐Ÿ• Updated 2026-08-13 ๐Ÿ“… Created 2024-06-05
9
NVIDIA/kvpress

LLM KV cache compression made easy

NVIDIA
โญ 1,258 Python ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2024-11-06
10
harleyszhang/llm_note

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

harleyszhang
โญ 897 Python ๐Ÿ‘ค harleyszhang User ๐Ÿ• Updated 2026-08-19 ๐Ÿ“… Created 2024-09-18
11
feifeibear/LLMSpeculativeSampling

Fast inference from large language models via speculative decoding

feifeibear
โญ 875 Python ๐Ÿ‘ค feifeibear User ๐Ÿ• Updated 2024-08-22 ๐Ÿ“… Created 2023-08-16
12
thu-nics/C2C

[ICLR'26] The official code implementation for "Cache-to-Cache: Direct Semantic Communication Between Large Language Models"

thu-nics
โญ 783 Python ๐Ÿ‘ค thu-nics Organization ๐Ÿ• Updated 2026-10-28 ๐Ÿ“… Created 2025-10-03
13
pegainfer-project/pegainfer

Pure Rust + CUDA LLM inference engine โ€” no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

pegainfer-project
โญ 748 Rust ๐Ÿ‘ค pegainfer-project Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-02-17
14
raymin0223/mixture_of_recursions

Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation (NeurIPS 2025)

raymin0223
โญ 651 Python ๐Ÿ‘ค raymin0223 User ๐Ÿ• Updated 2026-09-19 ๐Ÿ“… Created 2025-06-16
15
therealoliver/Deepdive-llama3-from-scratch

Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.

therealoliver
โญ 629 Jupyter Notebook ๐Ÿ‘ค therealoliver User ๐Ÿ• Updated 2025-02-24 ๐Ÿ“… Created 2025-02-19
16
zengxiao-he/tessera

From teacher to tiles โ€” a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.

zengxiao-he
โญ 556 Python ๐Ÿ‘ค zengxiao-he User ๐Ÿ• Updated 2026-10-09 ๐Ÿ“… Created 2026-06-05
17
OmniJev/Flide

๐Ÿ”ฅ๐Ÿ”ฅ The extremely fast inference engine for decision models

OmniJev
โญ 550 Python ๐Ÿ‘ค OmniJev Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-24
18
huawei-csl/KVarN

KVarN is a native vLLM KV-cache quantization backend for your agents: 3โ€“5ร— more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.

huawei-csl
โญ 521 Python ๐Ÿ‘ค huawei-csl Organization ๐Ÿ• Updated 2026-06-22 ๐Ÿ“… Created 2026-05-29

Frequently Asked Questions

1

What is the most popular #kv-cache repository on GitHub?

The top ranked open source #kv-cache project on GitHub is LMCache/LMCache with 12,405 stars. LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

2

Which programming languages are most used in #kv-cache repositories?

Among the top ranked #kv-cache projects on GitHub, the most common programming languages are Python, Go, Rust, Jupyter Notebook.

3

How is the #kv-cache repository ranking calculated?

We track 18 active open source repositories tagged with #kv-cache (500+ stars), totaling 46,401 stars with an average of 2,577 stars per repository. Rankings update daily from the GitHub API.