Git Stars Git Stars

#inference-acceleration Repositories

Discover 14 active open source GitHub repositories tagged with #inference-acceleration, accumulating 24,811 total stars (averaging 1,772 stars per project). Leading this topic is openai/consistency_models with 6,461 stars.

14
Total Repositories
24,811
Total Stars
1,772
Avg Stars/Repo
Programming Languages: Python Cuda
1
openai/consistency_models

Official repo for consistency models.

openai
โญ 6,461 Python ๐Ÿ‘ค openai Organization ๐Ÿ• Updated 2024-03-22 ๐Ÿ“… Created 2023-02-26
2
thu-ml/SageAttention

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

thu-ml
โญ 4,017 Cuda ๐Ÿ‘ค thu-ml Organization ๐Ÿ• Updated 2026-01-17 ๐Ÿ“… Created 2024-10-03
3
thu-ml/TurboDiffusion

TurboDiffusion: 100โ€“200ร— Acceleration for Video Diffusion Models

thu-ml
โญ 3,870 Python ๐Ÿ‘ค thu-ml Organization ๐Ÿ• Updated 2026-11-04 ๐Ÿ“… Created 2025-12-06
4
ali-vilab/TeaCache

Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model

ali-vilab
โญ 1,333 Python ๐Ÿ‘ค ali-vilab Organization ๐Ÿ• Updated 2025-06-08 ๐Ÿ“… Created 2024-11-27
5
meta-pytorch/segment-anything-fast

A batched offline inference oriented version of segment-anything

meta-pytorch
โญ 1,321 Python ๐Ÿ‘ค meta-pytorch Organization ๐Ÿ• Updated 2025-08-22 ๐Ÿ“… Created 2023-08-30
6
thu-ml/SpargeAttn

[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.

thu-ml
โญ 1,269 Cuda ๐Ÿ‘ค thu-ml Organization ๐Ÿ• Updated 2026-02-25 ๐Ÿ“… Created 2025-02-25
7
microsoft/MInference

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

microsoft
โญ 1,230 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-05-22
8
NVlabs/Fast-dLLM

Official implementation of "Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding"

NVlabs
โญ 1,116 Python ๐Ÿ‘ค NVlabs Organization ๐Ÿ• Updated 2026-05-30 ๐Ÿ“… Created 2025-05-27
9
Alpha-VLLM/Lumina-mGPT-2.0

Lumina-mGPT 2.0: Stand-Alone AutoRegressive Image Modeling

Alpha-VLLM
โญ 1,081 Python ๐Ÿ‘ค Alpha-VLLM Organization ๐Ÿ• Updated 2025-11-03 ๐Ÿ“… Created 2025-04-03
10
AlibabaResearch/SparkDiffusion

SparkDiffusion: Accelerating DiT video generation by 265ร— via joint sparsity, distillation, quantization, etc.

AlibabaResearch
โญ 750 Python ๐Ÿ‘ค AlibabaResearch Organization ๐Ÿ• Updated 2026-10-26 ๐Ÿ“… Created 2026-09-18
11
OpenGVLab/DCNv4

[CVPR 2024] Deformable Convolution v4

OpenGVLab
โญ 695 Python ๐Ÿ‘ค OpenGVLab Organization ๐Ÿ• Updated 2024-05-17 ๐Ÿ“… Created 2023-12-25
12
pkunlp-icler/FastV

[ECCV 2024 Oral] Code for paper: An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models

pkunlp-icler
โญ 579 Python ๐Ÿ‘ค pkunlp-icler Organization ๐Ÿ• Updated 2025-01-04 ๐Ÿ“… Created 2024-03-11
13
thu-ml/SLA

SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparseโ€“Linear Attention

thu-ml
โญ 553 Python ๐Ÿ‘ค thu-ml Organization ๐Ÿ• Updated 2026-10-18 ๐Ÿ“… Created 2025-09-25
14
Generated-Computers/Locals

Run many AI agents at once on your Mac, up to 3x faster.

Generated-Computers
โญ 538 Python ๐Ÿ‘ค Generated-Computers Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-12

Frequently Asked Questions

1

What is the most popular #inference-acceleration repository on GitHub?

The top ranked open source #inference-acceleration project on GitHub is openai/consistency_models with 6,461 stars. Official repo for consistency models.

2

Which programming languages are most used in #inference-acceleration repositories?

Among the top ranked #inference-acceleration projects on GitHub, the most common programming languages are Python, Cuda.

3

How is the #inference-acceleration repository ranking calculated?

We track 14 active open source repositories tagged with #inference-acceleration (500+ stars), totaling 24,811 stars with an average of 1,772 stars per repository. Rankings update daily from the GitHub API.