Git Stars Git Stars

#inference-optimization Repositories

Discover 41 active open source GitHub repositories tagged with #inference-optimization, accumulating 86,309 total stars (averaging 2,105 stars per project). Leading this topic is lllyasviel/stable-diffusion-webui-forge with 12,902 stars.

41
Total Repositories
86,309
Total Stars
2,105
Avg Stars/Repo
2
THUDM/slime

slime is an LLM training framework for RL Scaling.

THUDM
โญ 8,856 Python ๐Ÿ‘ค THUDM Organization ๐Ÿ• Updated 2026-11-05 ๐Ÿ“… Created 2025-06-18
3
deepseek-ai/open-infra-index

Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation

deepseek-ai
โญ 8,007 ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-05-15 ๐Ÿ“… Created 2025-02-21
4
microsoft/LLMLingua

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

microsoft
โญ 6,846 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2023-07-07
5
QwenLM/Qwen3.8

Qwen3.8 is the large language model series developed by the Qwen team at Alibaba Group.

QwenLM
โญ 4,459 ๐Ÿ‘ค QwenLM Organization ๐Ÿ• Updated 2026-08-17 ๐Ÿ“… Created 2025-09-11
โญ 2,917 C ๐Ÿ‘ค airockchip User ๐Ÿ• Updated 2026-10-20 ๐Ÿ“… Created 2022-02-15
7
syv-ai/HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

syv-ai
โญ 2,778 Python ๐Ÿ‘ค syv-ai Organization ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2026-08-15
8
google/XNNPACK

High-efficiency floating-point neural network inference operators for mobile, server, and Web

google
โญ 2,506 C ๐Ÿ‘ค google Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2019-09-13
9
XiaomiMiMo/MiMo

MiMo: Unlocking the Reasoning Potential of Language Model โ€“ From Pretraining to Posttraining

XiaomiMiMo
โญ 2,411 Python ๐Ÿ‘ค XiaomiMiMo Organization ๐Ÿ• Updated 2026-10-29 ๐Ÿ“… Created 2025-04-26
10
AmberLJC/LLMSys-PaperList

Large Language Model (LLM) Systems Paper List

AmberLJC
โญ 2,332 Python ๐Ÿ‘ค AmberLJC User ๐Ÿ• Updated 2026-10-20 ๐Ÿ“… Created 2023-06-06
11
facebookresearch/blt

Code for BLT research paper

facebookresearch
โญ 2,070 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2025-11-03 ๐Ÿ“… Created 2024-12-09
12
NUS-HPC-AI-Lab/VideoSys

VideoSys: An easy and efficient system for video generation

NUS-HPC-AI-Lab
โญ 2,022 Python ๐Ÿ‘ค NUS-HPC-AI-Lab Organization ๐Ÿ• Updated 2025-08-27 ๐Ÿ“… Created 2024-02-17
13
NVIDIA/Stable-Diffusion-WebUI-TensorRT

TensorRT Extension for Stable Diffusion Web UI

NVIDIA
โญ 1,994 Python ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2024-06-14 ๐Ÿ“… Created 2023-10-10
14
ChenyangSi/FreeU

FreeU: Free Lunch in Diffusion U-Net (CVPR2024 Oral)

ChenyangSi
โญ 1,892 ๐Ÿ‘ค ChenyangSi User ๐Ÿ• Updated 2024-12-24 ๐Ÿ“… Created 2023-09-14
15
llmgenai/LLMInterviewQuestions

This repository contains LLM (Large language model) interview question asked in top companies like Google, Nvidia , Meta , Microsoft & fortune 500 companies.

llmgenai
โญ 1,781 ๐Ÿ‘ค llmgenai User ๐Ÿ• Updated 2025-02-12 ๐Ÿ“… Created 2024-12-26
16
chengzeyi/Comfy-WaveSpeed

https://wavespeed.ai/ [WIP] The all in one inference optimization solution for ComfyUI, universal, flexible, and fast.

chengzeyi
โญ 1,235 Python ๐Ÿ‘ค chengzeyi User ๐Ÿ• Updated 2026-08-21 ๐Ÿ“… Created 2024-11-27
17
EdjeElectronics/TensorFlow-Object-Detection-on-the-Raspberry-Pi

A tutorial showing how to set up TensorFlow's Object Detection API on the Raspberry Pi

EdjeElectronics
โญ 1,156 Python ๐Ÿ‘ค EdjeElectronics User ๐Ÿ• Updated 2024-08-10 ๐Ÿ“… Created 2018-06-05
18
gpu-mode/awesomeMLSys

An ML Systems Onboarding list

gpu-mode
โญ 1,154 ๐Ÿ‘ค gpu-mode Organization ๐Ÿ• Updated 2026-02-19 ๐Ÿ“… Created 2024-05-06
19
huggingface/search-and-learn

Recipes to scale inference-time compute of open models

huggingface
โญ 1,134 Python ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2026-09-23 ๐Ÿ“… Created 2024-12-09
21
allenai/OLMoE

OLMoE: Open Mixture-of-Experts Language Models

allenai
โญ 1,087 Jupyter Notebook ๐Ÿ‘ค allenai Organization ๐Ÿ• Updated 2025-09-23 ๐Ÿ“… Created 2024-06-22
22
onnx/onnx-mlir

Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure

onnx
โญ 1,072 MLIR ๐Ÿ‘ค onnx Organization ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2020-02-18
23
mryab/efficient-dl-systems

Efficient Deep Learning Systems course materials

mryab
โญ 1,052 Jupyter Notebook ๐Ÿ‘ค mryab User ๐Ÿ• Updated 2026-05-28 ๐Ÿ“… Created 2021-12-06
24
hzwer/Practical-RIFE

More practical frame interpolation approach.

hzwer
โญ 1,050 Python ๐Ÿ‘ค hzwer User ๐Ÿ• Updated 2026-08-27 ๐Ÿ“… Created 2021-06-16
25
daixiangzi/Awesome-Token-Compress

A paper list of some recent works about Token Compress for Vit and VLM

daixiangzi
โญ 966 ๐Ÿ‘ค daixiangzi User ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2024-09-06
26
FoundationVision/Waver

Industry-level video foundation model for unified Text-to-Video (T2V) and Image-to-Video (I2V) generation.

FoundationVision
โญ 952 ๐Ÿ‘ค FoundationVision Organization ๐Ÿ• Updated 2025-08-27 ๐Ÿ“… Created 2025-08-05
27
alibaba/BladeDISC

BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.

alibaba
โญ 911 C++ ๐Ÿ‘ค alibaba Organization ๐Ÿ• Updated 2024-12-30 ๐Ÿ“… Created 2021-12-20
28
lifeiteng/OmniSenseVoice

Omni SenseVoice: High-Speed Speech Recognition with words timestamps ๐Ÿ—ฃ๏ธ๐ŸŽฏ

lifeiteng
โญ 897 Python ๐Ÿ‘ค lifeiteng User ๐Ÿ• Updated 2025-12-10 ๐Ÿ“… Created 2024-09-22
29
KohakuBlueleaf/a1111-sd-webui-lycoris

An extension for stable-diffusion-webui to load lycoris models.

KohakuBlueleaf
โญ 894 Python ๐Ÿ‘ค KohakuBlueleaf User ๐Ÿ• Updated 2024-02-16 ๐Ÿ“… Created 2023-04-09
30
aiola-lab/whisper-medusa

Whisper with Medusa heads

aiola-lab
โญ 860 Python ๐Ÿ‘ค aiola-lab Organization ๐Ÿ• Updated 2026-07-02 ๐Ÿ“… Created 2024-07-17
31
datawhalechina/llm-algo-leetcode

Hands-on LLM algorithms and systems with PyTorch, Triton & CUDA. โ€œLLM Algorithms and Systems Tutorialโ€: model architectures and training, post-training, inference acceleration, performance and operator optimization, with companion Notebook exercises, answer explanations, tests, and experiments.

datawhalechina
โญ 811 Jupyter Notebook ๐Ÿ‘ค datawhalechina Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2026-04-01
32
tensorflow/runtime

A performant and modular runtime for TensorFlow

tensorflow
โญ 754 C++ ๐Ÿ‘ค tensorflow Organization ๐Ÿ• Updated 2025-09-04 ๐Ÿ“… Created 2020-04-24
33
IEIT-Yuan/Yuan-2.0

Yuan 2.0 Large Language Model

IEIT-Yuan
โญ 689 Python ๐Ÿ‘ค IEIT-Yuan Organization ๐Ÿ• Updated 2024-07-11 ๐Ÿ“… Created 2023-11-26
34
microsoft/VPTQ

VPTQ, A Flexible and Extreme low-bit quantization algorithm

microsoft
โญ 683 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-08-04 ๐Ÿ“… Created 2024-08-13
36
HybridTalentComputing/cline-chinese

Cline Chinese localized version, Cline is an autonomous programming assistant that runs in your IDE and can create/edit files, run commands, use the browser, and other functions with your permission.

HybridTalentComputing
โญ 660 TypeScript ๐Ÿ‘ค HybridTalentComputing User ๐Ÿ• Updated 2026-07-14 ๐Ÿ“… Created 2025-02-07
โญ 603 Python ๐Ÿ‘ค google-research Organization ๐Ÿ• Updated 2026-03-12 ๐Ÿ“… Created 2019-10-31
38
HazyResearch/aisys-building-blocks

Building blocks for foundation models.

HazyResearch
โญ 587 ๐Ÿ‘ค HazyResearch Organization ๐Ÿ• Updated 2024-01-03 ๐Ÿ“… Created 2023-11-30
39
brontoguana/krasis

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

brontoguana
โญ 530 C++ ๐Ÿ‘ค brontoguana User ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2026-02-09
40
amitshekhariitbhu/llm-inference-engineering

Learn LLM Inference Engineering step by stepโ€”from KV cache, PagedAttention, and continuous batching to vLLM, SGLang, and GPUs.

amitshekhariitbhu
โญ 523 Markdown ๐Ÿ‘ค amitshekhariitbhu User ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2026-08-18
41
YingqingHe/ScaleCrafter

[ICLR 2024 Spotlight] Official implementation of ScaleCrafter for higher-resolution visual generation at inference time.

YingqingHe
โญ 508 Python ๐Ÿ‘ค YingqingHe User ๐Ÿ• Updated 2024-03-07 ๐Ÿ“… Created 2023-10-07

Frequently Asked Questions

1

What is the most popular #inference-optimization repository on GitHub?

The top ranked open source #inference-optimization project on GitHub is lllyasviel/stable-diffusion-webui-forge with 12,902 stars.

2

Which programming languages are most used in #inference-optimization repositories?

Among the top ranked #inference-optimization projects on GitHub, the most common programming languages are Python, Jupyter Notebook, C++, C.

3

How is the #inference-optimization repository ranking calculated?

We track 41 active open source repositories tagged with #inference-optimization (500+ stars), totaling 86,309 stars with an average of 2,105 stars per repository. Rankings update daily from the GitHub API.