Git Stars Git Stars

#vllm Repositories

133
Total Repositories
554,542
Total Stars
4,169
Avg Stars/Repo
Programming Languages: Python Go HTML C++
51
allenai/Olmo-core

PyTorch building blocks for the OLMo ecosystem

allenai
โญ 1,810 Python ๐Ÿ‘ค allenai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-03-23
52
vllm-project/guidellm

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

vllm-project
โญ 1,800 Python ๐Ÿ‘ค vllm-project Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-05-29
53
StartLuxLabs/StartLux-Decision

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.

StartLuxLabs
โญ 1,780 Python ๐Ÿ‘ค StartLuxLabs Organization ๐Ÿ• Updated 2026-10-23 ๐Ÿ“… Created 2026-09-29
54
intel/auto-round

PTQ-Level Efficiency, QAT-Level Accuracy

intel
โญ 1,706 Python ๐Ÿ‘ค intel Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-01-04
โญ 1,677 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-11-18 ๐Ÿ“… Created 2025-09-29
56
waybarrios/vllm-mlx

High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

waybarrios
โญ 1,657 Python ๐Ÿ‘ค waybarrios User ๐Ÿ• Updated 2026-11-07 ๐Ÿ“… Created 2025-12-06
57
nokia-applied-research/AnyJev

Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating, welcome any issue and PR request)

nokia-applied-research
โญ 1,632 Python ๐Ÿ‘ค nokia-applied-research Organization ๐Ÿ• Updated 2026-10-21 ๐Ÿ“… Created 2026-09-21
58
autonomous-ai/Personal-AI-Computer

Open-source AI computer builds, from 2-GPU desktops to 8-GPU servers: CAD, BOM, BIOS, and build guides. Get the case kit with PCIe risers, or ask us for a free fit check.

autonomous-ai
โญ 1,601 ๐Ÿ‘ค autonomous-ai Organization ๐Ÿ• Updated 2026-11-05 ๐Ÿ“… Created 2025-08-22
โญ 1,599 Python ๐Ÿ‘ค databricks Organization ๐Ÿ• Updated 2026-03-25 ๐Ÿ“… Created 2023-01-26
โญ 1,588 Python ๐Ÿ‘ค Tencent-Hunyuan Organization ๐Ÿ• Updated 2024-12-06 ๐Ÿ“… Created 2024-10-23
61
tencent-ailab/persona-hub

Official repo for the paper "Scaling Synthetic Data Creation with 1,000,000,000 Personas"

tencent-ailab
โญ 1,585 Python ๐Ÿ‘ค tencent-ailab User ๐Ÿ• Updated 2025-02-19 ๐Ÿ“… Created 2024-06-22
62
verl-project/verl-omni

Multimodal RL training framework for diffusion & omni models

verl-project
โญ 1,560 Python ๐Ÿ‘ค verl-project Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-04-21
โญ 1,434 Python ๐Ÿ‘ค QwenLM Organization ๐Ÿ• Updated 2026-06-23 ๐Ÿ“… Created 2026-01-08
64
BICLab/SpikingBrain-7B

Spiking Brain-inspired Large Models, integrating hybrid efficient attention, MoE modules and spike encoding into its architecture

BICLab
โญ 1,432 Python ๐Ÿ‘ค BICLab User ๐Ÿ• Updated 2026-05-14 ๐Ÿ“… Created 2025-09-03
65
MoonshotAI/Kimi-Dev

open-source coding LLM for software engineering tasks

MoonshotAI
โญ 1,357 Python ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2025-09-30 ๐Ÿ“… Created 2025-06-16
66
ModelCloud/GPTQModel

LLM model quantization (compression) toolkit with hardware acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU, Tenstorrent, Huawei Ascend NPU, and Intel/AMD/Apple CPUs via HF, vLLM, and SGLang.

ModelCloud
โญ 1,294 Python ๐Ÿ‘ค ModelCloud Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-06-17
67
kubeai-project/kubeai

AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.

kubeai-project
โญ 1,294 Go ๐Ÿ‘ค kubeai-project Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2023-10-21
โญ 1,293 Python ๐Ÿ‘ค Visual-Agent Organization ๐Ÿ• Updated 2025-11-20 ๐Ÿ“… Created 2025-03-15
69
OpenBuddy/OpenBuddy

Open Multilingual Chatbot for Everyone

OpenBuddy
โญ 1,274 ๐Ÿ‘ค OpenBuddy Organization ๐Ÿ• Updated 2025-06-08 ๐Ÿ“… Created 2023-04-26
70
patchy631/time-to-first-token

A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchmarking.

patchy631
โญ 1,245 HTML ๐Ÿ‘ค patchy631 User ๐Ÿ• Updated 2026-08-14 ๐Ÿ“… Created 2026-08-02
71
Ksuriuri/index-tts-vllm

Added vLLM support to IndexTTS for faster inference.

Ksuriuri
โญ 1,244 Python ๐Ÿ‘ค Ksuriuri User ๐Ÿ• Updated 2026-04-13 ๐Ÿ“… Created 2025-05-03
72
eagle-hq/EAGLE-4

Official Inference Code of EAGLE-4

eagle-hq
โญ 1,220 Python ๐Ÿ‘ค eagle-hq Organization ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2026-10-20
73
bricks-cloud/BricksLLM

โ˜๏ธ Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.

bricks-cloud
โญ 1,210 Go ๐Ÿ‘ค bricks-cloud Organization ๐Ÿ• Updated 2025-01-05 ๐Ÿ“… Created 2023-07-18
74
jmaczan/tiny-vllm

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

jmaczan
โญ 1,207 C++ ๐Ÿ‘ค jmaczan User ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2026-02-09
75
MoonshotAI/Kimi-VL

Kimi-VL: Mixture-of-Experts Vision-Language Model for Multimodal Reasoning, Long-Context Understanding, and Strong Agent Capabilities

MoonshotAI
โญ 1,203 ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2025-07-15 ๐Ÿ“… Created 2025-04-09
76
BytedTsinghua-SIA/MemAgent

A MemAgent framework that can be extrapolated to 3.5M, along with a training framework for RL training of any agent workflow.

BytedTsinghua-SIA
โญ 1,118 Python ๐Ÿ‘ค BytedTsinghua-SIA Organization ๐Ÿ• Updated 2026-05-12 ๐Ÿ“… Created 2025-06-24
โญ 1,090 Python ๐Ÿ‘ค thinking-machines-lab Organization ๐Ÿ• Updated 2025-11-04 ๐Ÿ“… Created 2025-09-10
78
prometheus-eval/prometheus-eval

Evaluate your LLM's response with Prometheus and GPT4 ๐Ÿ’ฏ

prometheus-eval
โญ 1,087 Python ๐Ÿ‘ค prometheus-eval Organization ๐Ÿ• Updated 2025-04-25 ๐Ÿ“… Created 2024-04-18
79
GAIR-NLP/LIMO

[COLM 2025] LIMO: Less is More for Reasoning

GAIR-NLP
โญ 1,081 Python ๐Ÿ‘ค GAIR-NLP Organization ๐Ÿ• Updated 2025-07-30 ๐Ÿ“… Created 2025-02-04
80
Pokee-AI/PokeeResearchOSS

Pokee Deep Research Model Open Source Repo

Pokee-AI
โญ 1,075 Python ๐Ÿ‘ค Pokee-AI Organization ๐Ÿ• Updated 2025-10-22 ๐Ÿ“… Created 2025-10-17
81
Tencent-Hunyuan/UniRL

UniRL is a Framework for Unified Multimodal Model Reinforcement Learning

Tencent-Hunyuan
โญ 1,051 Python ๐Ÿ‘ค Tencent-Hunyuan Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2026-06-08
83
ZJUI-AI4H/Hulu-Med

A Transparent Generalist Model towards Holistic Medical Vision-Language Understanding

ZJUI-AI4H
โญ 1,031 Python ๐Ÿ‘ค ZJUI-AI4H User ๐Ÿ• Updated 2026-08-30 ๐Ÿ“… Created 2025-10-08
84
convergence-ai/proxy-lite

A mini, open-weights, version of our Proxy assistant.

convergence-ai
โญ 989 Python ๐Ÿ‘ค convergence-ai Organization ๐Ÿ• Updated 2025-02-26 ๐Ÿ“… Created 2025-02-10
โญ 970 Python ๐Ÿ‘ค zhentingqi User ๐Ÿ• Updated 2025-01-23 ๐Ÿ“… Created 2024-08-07
86
alvarobartt/hf-mem

A CLI to estimate inference memory requirements for Hugging Face models, written in Python.

alvarobartt
โญ 956 Python ๐Ÿ‘ค alvarobartt User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2025-12-22
87
Valen-Team/Valen

Open multimodal decision models, 0.8B to 27B, for text, images and video. Decide with probabilities over your own candidates and locate with bounding boxes, in place of generated text. Free API, ONNX, GGUF, vLLM and SGLang; fine-tuning code included.

Valen-Team
โญ 932 Python ๐Ÿ‘ค Valen-Team Organization ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2026-09-23
88
XHToken/Spark-X2.6

Spark-x2.6 open model series. Pushing the Limits of Agentic Capabilities in On-Device Models

XHToken
โญ 927 ๐Ÿ‘ค XHToken Organization ๐Ÿ• Updated 2026-11-04 ๐Ÿ“… Created 2026-08-24
89
harleyszhang/llm_note

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

harleyszhang
โญ 897 Python ๐Ÿ‘ค harleyszhang User ๐Ÿ• Updated 2026-08-19 ๐Ÿ“… Created 2024-09-18
90
th1nhhdk/local_ai_ocr

A local, offline (after initial setup), portable OCR software that can process images and PDF files, using DeepSeek-OCR-2 AI (running directly on your machine).

th1nhhdk
โญ 892 Python ๐Ÿ‘ค th1nhhdk User ๐Ÿ• Updated 2026-07-05 ๐Ÿ“… Created 2025-11-21
91
triton-inference-server/tutorials

This repository contains tutorials and examples for Triton Inference Server

triton-inference-server
โญ 869 Python ๐Ÿ‘ค triton-inference-server Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2022-12-16
92
whitecircle/halo

Halo is an open-source framework built by White Circle for training large language and multimodal models

whitecircle
โญ 868 Python ๐Ÿ‘ค whitecircle Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-08-26
93
varunvasudeva1/llm-server-docs

End-to-end documentation to set up your own local & fully private LLM server on Debian. Equipped with chat, web search, RAG, model management, MCP servers, image generation, and TTS.

varunvasudeva1
โญ 853 ๐Ÿ‘ค varunvasudeva1 User ๐Ÿ• Updated 2026-06-30 ๐Ÿ“… Created 2024-03-26
94
pgalko/BambooAI

AI-powered data analysis workbench built for transparency, reproducibility and user control. Inspect the code and evidence behind results, verify calculations through replay, and choose from a wide range of models, including local open-weight models.

pgalko
โญ 848 Python ๐Ÿ‘ค pgalko User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-05-07
95
TideDra/lmm-r1

Extend OpenRLHF to support LMM RL training for reproduction of DeepSeek-R1 on multimodal tasks.

TideDra
โญ 843 Python ๐Ÿ‘ค TideDra User ๐Ÿ• Updated 2025-05-14 ๐Ÿ“… Created 2025-02-13
96
jakobdylanc/llmcord

Make Discord your LLM frontend

jakobdylanc
โญ 843 Python ๐Ÿ‘ค jakobdylanc User ๐Ÿ• Updated 2026-08-14 ๐Ÿ“… Created 2023-05-08
97
OpenBMB/AgentCPM

An End-to-End Infrastructure for Training and Evaluating Various LLM Agents.

OpenBMB
โญ 835 Python ๐Ÿ‘ค OpenBMB Organization ๐Ÿ• Updated 2026-10-22 ๐Ÿ“… Created 2026-01-08
98
Tencent-Hunyuan/Hunyuan-A13B

Tencent Hunyuan A13B (short as Hunyuan-A13B), an innovative and open-source LLM built on a fine-grained MoE architecture.

Tencent-Hunyuan
โญ 815 Python ๐Ÿ‘ค Tencent-Hunyuan Organization ๐Ÿ• Updated 2025-07-08 ๐Ÿ“… Created 2025-06-27
99
ARahim3/kaggle-tpu-lab

Frontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Works with Claude Code, Codex, opencode and pi.

ARahim3
โญ 812 Python ๐Ÿ‘ค ARahim3 User ๐Ÿ• Updated 2026-10-06 ๐Ÿ“… Created 2026-08-31
100
ByteDance-Seed/Seed-Coder

Seed-Coder is a family of lightweight open-source code LLMs comprising base, instruct and reasoning models, developed by ByteDance Seed.

ByteDance-Seed
โญ 755 ๐Ÿ‘ค ByteDance-Seed Organization ๐Ÿ• Updated 2025-06-06 ๐Ÿ“… Created 2025-04-21