#kv-cache Repositories
Discover 18 active open source GitHub repositories tagged with #kv-cache, accumulating 46,401 total stars (averaging 2,577 stars per project). Leading this topic is LMCache/LMCache with 12,405 stars.
18 repositories total
ใIn-Depth Understanding of AI Infrastructure: Quantitative Analysis and System Designใ (by Li Bojie), an open-source book manuscript: Starting from hardware constraints and model architectures, it quantitatively derives the design of LLM inference and training systems. Includes the full text, PDF, accompanying computational tools, and experiments.
A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.
A Golang implemented Redis Server and Cluster. Redis Server and Distributed Cluster implemented in Go language
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.
Fast inference from large language models via speculative decoding
[ICLR'26] The official code implementation for "Cache-to-Cache: Direct Semantic Communication Between Large Language Models"
Pure Rust + CUDA LLM inference engine โ no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation (NeurIPS 2025)
Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.
From teacher to tiles โ a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.
KVarN is a native vLLM KV-cache quantization backend for your agents: 3โ5ร more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.
Frequently Asked Questions
What is the most popular #kv-cache repository on GitHub?
The top ranked open source #kv-cache project on GitHub is LMCache/LMCache with 12,405 stars. LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
Which programming languages are most used in #kv-cache repositories?
Among the top ranked #kv-cache projects on GitHub, the most common programming languages are Python, Go, Rust, Jupyter Notebook.
How is the #kv-cache repository ranking calculated?
We track 18 active open source repositories tagged with #kv-cache (500+ stars), totaling 46,401 stars with an average of 2,577 stars per repository. Rankings update daily from the GitHub API.