Git Stars Git Stars

#vllm Repositories

133
Total Repositories
554,542
Total Stars
4,169
Avg Stars/Repo
101
ModelTC/LightCompress

[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.

ModelTC
โญ 755 Python ๐Ÿ‘ค ModelTC Organization ๐Ÿ• Updated 2026-05-14 ๐Ÿ“… Created 2024-03-06
102
pegainfer-project/pegainfer

Pure Rust + CUDA LLM inference engine โ€” no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

pegainfer-project
โญ 748 Rust ๐Ÿ‘ค pegainfer-project Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-02-17
103
smg-project/smg

Engine-agnostic LLM gateway in Rust. Full OpenAI and Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini, and more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant authentication.

smg-project
โญ 736 Rust ๐Ÿ‘ค smg-project Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-11-22
104
microsoft/vidur

Accurate, large-scale, and extensible simulator for LLM inference Systems

microsoft
โญ 733 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-08-24 ๐Ÿ“… Created 2023-11-02
โญ 727 Python ๐Ÿ‘ค Tencent-Hunyuan Organization ๐Ÿ• Updated 2025-12-30 ๐Ÿ“… Created 2025-08-28
106
SimonZeng7108/efficientsam3

EfficientSAM3 compresses SAM3 into lightweight, edge-friendly models via progressive knowledge distillation for fast promptable concept segmentation and tracking.

SimonZeng7108
โญ 713 Jupyter Notebook ๐Ÿ‘ค SimonZeng7108 User ๐Ÿ• Updated 2026-08-11 ๐Ÿ“… Created 2025-10-16
โญ 703 Python ๐Ÿ‘ค stepfun-ai Organization ๐Ÿ• Updated 2026-04-29 ๐Ÿ“… Created 2025-11-11
108
ChenxinAn-fdu/POLARIS

Scaling RL on advanced reasoning models

ChenxinAn-fdu
โญ 697 Python ๐Ÿ‘ค ChenxinAn-fdu User ๐Ÿ• Updated 2025-10-20 ๐Ÿ“… Created 2025-06-18
109
IEIT-Yuan/Yuan-2.0

Yuan 2.0 Large Language Model

IEIT-Yuan
โญ 689 Python ๐Ÿ‘ค IEIT-Yuan Organization ๐Ÿ• Updated 2024-07-11 ๐Ÿ“… Created 2023-11-26
110
mlabonne/llm-autoeval

Automatically evaluate your LLMs in Google Colab

mlabonne
โญ 678 Python ๐Ÿ‘ค mlabonne User ๐Ÿ• Updated 2024-05-07 ๐Ÿ“… Created 2024-01-01
111
owenliang/qwen-vllm

Tongyi Qianwen VLLM Inference Deployment DEMO

owenliang
โญ 637 Python ๐Ÿ‘ค owenliang User ๐Ÿ• Updated 2024-03-28 ๐Ÿ“… Created 2024-01-21
112
firstbatchxyz/mem-agent-mcp

mem-agent mcp server

firstbatchxyz
โญ 628 Python ๐Ÿ‘ค firstbatchxyz Organization ๐Ÿ• Updated 2025-11-17 ๐Ÿ“… Created 2025-09-01
113
mlfoundations/evalchemy

Automatic evals for LLMs

mlfoundations
โญ 617 HTML ๐Ÿ‘ค mlfoundations Organization ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2024-11-15
114
jaylfc/taOS

Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).

jaylfc
โญ 609 Python ๐Ÿ‘ค jaylfc User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-04-05
115
HuiResearch/FlashTTS

Provides high-quality Chinese speech synthesis and voice cloning services based on models such as SparkTTS and OrpheusTTS.

HuiResearch
โญ 603 Python ๐Ÿ‘ค HuiResearch User ๐Ÿ• Updated 2025-05-18 ๐Ÿ“… Created 2025-03-13
116
MoonshotAI/K2-Vendor-Verifier

Verify Precision of all Kimi K2 API Vendor

MoonshotAI
โญ 602 Python ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2026-02-14 ๐Ÿ“… Created 2025-09-09
117
llmmanorg/llmman

Run any agent on any model, models stored as OCI images

llmmanorg
โญ 598 Rust ๐Ÿ‘ค llmmanorg Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-03-22
119
kacper-daftcode/vLLM-Moet

Frontier MoE models on RTX PRO 6000 / RTX 5090 (sm_120): DeepSeek-V4.1-Flash on 4x RTX PRO 6000, as a GPU+CPU hybrid on 4x RTX 5090, and 2-bit experts on vLLM v0.24.0

kacper-daftcode
โญ 580 Sass ๐Ÿ‘ค kacper-daftcode User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2026-06-15
120
raids-lab/crater

Crater is a cloud-native AI training & inference platform.

raids-lab
โญ 578 Go ๐Ÿ‘ค raids-lab Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2025-06-08
โญ 568 Python ๐Ÿ‘ค pipecat-ai Organization ๐Ÿ• Updated 2026-06-03 ๐Ÿ“… Created 2026-01-01
122
fufankeji/DeepSeek-OCR-Web

Out-of-the-box DeepSeek OCR document parsing Web Studio

fufankeji
โญ 568 TypeScript ๐Ÿ‘ค fufankeji User ๐Ÿ• Updated 2025-10-27 ๐Ÿ“… Created 2025-10-23
123
spark-arena/sparkrun

sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems. Live support: https://discord.com/invite/GH5kRgv6ZD

spark-arena
โญ 567 Python ๐Ÿ‘ค spark-arena Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2026-02-16
124
FlagOpen/RoboBrain

[CVPR 2025] RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete. Official Repository.

FlagOpen
โญ 564 Python ๐Ÿ‘ค FlagOpen Organization ๐Ÿ• Updated 2025-10-13 ๐Ÿ“… Created 2025-03-27
125
llumnix-project/llumnix-ray

Efficient and easy multi-instance LLM serving

llumnix-project
โญ 561 Python ๐Ÿ‘ค llumnix-project Organization ๐Ÿ• Updated 2026-03-12 ๐Ÿ“… Created 2024-05-20
126
01-ai/Yi-1.5

Yi-1.5 is an upgraded version of Yi, delivering stronger performance in coding, math, reasoning, and instruction-following capability.

01-ai
โญ 557 ๐Ÿ‘ค 01-ai Organization ๐Ÿ• Updated 2024-11-11 ๐Ÿ“… Created 2024-05-10
โญ 557 Python ๐Ÿ‘ค Snowflake-Labs Organization ๐Ÿ• Updated 2024-08-16 ๐Ÿ“… Created 2024-04-22
128
micytao/vllm-playground

A modern web interface for managing and interacting with vLLM servers (www.github.com/vllm-project/vllm). Supports both GPU and CPU modes, with special optimizations for macOS Apple Silicon and enterprise deployment on OpenShift/Kubernetes.

micytao
โญ 548 JavaScript ๐Ÿ‘ค micytao User ๐Ÿ• Updated 2026-11-07 ๐Ÿ“… Created 2025-11-01
129
Yzw202011/OmniSpace

A local-first, full-stack AI creative workstation: five modulesโ€”AI chat, AI comics, comic drama creation, writing desk, and knowledge learning; multi-engine local inference with vLLM / llama.cpp / Transformers, intelligent VRAM scheduling, bulkhead isolation and self-healing. For learning purposes only; feedback and discussion are welcome. (The latest progress, excluding the plugin system, cannot currently be committed to the repository due to protocol restrictions.)

Yzw202011
โญ 536 Python ๐Ÿ‘ค Yzw202011 User ๐Ÿ• Updated 2026-09-20 ๐Ÿ“… Created 2026-08-23
130
ome-projects/ome

Open Model Engine (OME) โ€” Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

ome-projects
โญ 529 Go ๐Ÿ‘ค ome-projects Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-05-19
131
huawei-csl/KVarN

KVarN is a native vLLM KV-cache quantization backend for your agents: 3โ€“5ร— more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.

huawei-csl
โญ 521 Python ๐Ÿ‘ค huawei-csl Organization ๐Ÿ• Updated 2026-06-22 ๐Ÿ“… Created 2026-05-29
132
adventists-ai/sophon-realtime

Open full-duplex voice models that hear your tone and speak with one. One GPU with vLLM-Omni, or fully offline on a Mac with MLX. Zero-shot voice cloning, OpenAI Realtime / GPT-Live compatible, free hosted API trial. Apache-2.0.

adventists-ai
โญ 510 Python ๐Ÿ‘ค adventists-ai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-11-04
133
Chen-zexi/vllm-cli

A command-line interface tool for serving LLM using vLLM.

Chen-zexi
โญ 507 Python ๐Ÿ‘ค Chen-zexi User ๐Ÿ• Updated 2026-01-25 ๐Ÿ“… Created 2025-08-14
134
AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-DFlash

Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/aeon-7/aeon-vllm-ultimate:latest container, tuned for long-context draft acceptance on DGX Spark. 6 HF variants (BF16/NVFP4/MTP/MTP-XS), docker-compose, and QuickStart.

AEON-7
โญ 501 Python ๐Ÿ‘ค AEON-7 User ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2026-04-25