Git Stars Git Stars

#sparse-attention Repositories

Discover 18 active open source GitHub repositories tagged with #sparse-attention, accumulating 62,491 total stars (averaging 3,471 stars per project). Leading this topic is deepseek-ai/FlashMLA with 13,275 stars.

18
Total Repositories
62,491
Total Stars
3,471
Avg Stars/Repo
1
deepseek-ai/FlashMLA

FlashMLA: Efficient Multi-head Latent Attention Kernels

deepseek-ai
โญ 13,275 C++ ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2026-10-27 ๐Ÿ“… Created 2025-02-21
2
PKU-YuanGroup/Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

PKU-YuanGroup
โญ 12,201 Python ๐Ÿ‘ค PKU-YuanGroup Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-02-20
3
OpenBMB/MiniCPM

MiniCPM5: SOTA on-device LLMs, small yet powerful.

OpenBMB
โญ 11,898 Jupyter Notebook ๐Ÿ‘ค OpenBMB Organization ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2024-01-29
โญ 9,620 Python ๐Ÿ‘ค meituan-longcat Organization ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2025-10-25
โญ 2,509 Python ๐Ÿ‘ค HiDream-ai Organization ๐Ÿ• Updated 2025-07-16 ๐Ÿ“… Created 2025-04-06
7
Tencent-Hunyuan/Prism

We propose Prism, a dynamic sparse attention framework for natively training joint video-audio generation models at 2K.

Tencent-Hunyuan
โญ 1,518 Python ๐Ÿ‘ค Tencent-Hunyuan Organization ๐Ÿ• Updated 2026-10-31 ๐Ÿ“… Created 2026-09-21
8
thu-ml/SpargeAttn

[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.

thu-ml
โญ 1,269 Cuda ๐Ÿ‘ค thu-ml Organization ๐Ÿ• Updated 2026-02-25 ๐Ÿ“… Created 2025-02-25
9
microsoft/MInference

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

microsoft
โญ 1,230 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-05-22
10
Tencent/hpc-ops

High Performance LLM Inference Operator Library

Tencent
โญ 1,193 C++ ๐Ÿ‘ค Tencent Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2026-01-20
11
fla-org/native-sparse-attention

๐Ÿณ Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"

fla-org
โญ 1,030 Python ๐Ÿ‘ค fla-org Organization ๐Ÿ• Updated 2026-02-05 ๐Ÿ“… Created 2025-02-21
12
mit-han-lab/omniserve

[MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention

mit-han-lab
โญ 840 C++ ๐Ÿ‘ค mit-han-lab Organization ๐Ÿ• Updated 2025-03-06 ๐Ÿ“… Created 2024-05-06
13
SHI-Labs/NATTEN

Fast Multi-dimensional Sparse Attention

SHI-Labs
โญ 815 C++ ๐Ÿ‘ค SHI-Labs Organization ๐Ÿ• Updated 2026-07-26 ๐Ÿ“… Created 2022-10-08
14
lucidrains/native-sparse-attention-pytorch

Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper

lucidrains
โญ 811 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2025-08-15 ๐Ÿ“… Created 2025-02-19
15
HKUSTDial/flash-sparse-attention

Trainable fast and memory-efficient sparse attention

HKUSTDial
โญ 750 Python ๐Ÿ‘ค HKUSTDial Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2025-05-12
16
svg-project/Sparse-VideoGen

[ICML2025, NeurIPS2025 Spotlight] Sparse VideoGen 1 & 2: Accelerating Video Diffusion Transformers with Sparse Attention

svg-project
โญ 712 Python ๐Ÿ‘ค svg-project User ๐Ÿ• Updated 2026-07-04 ๐Ÿ“… Created 2025-03-09
17
mit-han-lab/radial-attention

[NeurIPS 2025] Radial Attention: O(nlogn) Sparse Attention with Energy Decay for Long Video Generation

mit-han-lab
โญ 609 Python ๐Ÿ‘ค mit-han-lab Organization ๐Ÿ• Updated 2025-11-11 ๐Ÿ“… Created 2025-06-19
18
thu-ml/SLA

SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparseโ€“Linear Attention

thu-ml
โญ 553 Python ๐Ÿ‘ค thu-ml Organization ๐Ÿ• Updated 2026-10-18 ๐Ÿ“… Created 2025-09-25

Frequently Asked Questions

1

What is the most popular #sparse-attention repository on GitHub?

The top ranked open source #sparse-attention project on GitHub is deepseek-ai/FlashMLA with 13,275 stars. FlashMLA: Efficient Multi-head Latent Attention Kernels

2

Which programming languages are most used in #sparse-attention repositories?

Among the top ranked #sparse-attention projects on GitHub, the most common programming languages are Python, C++, Jupyter Notebook, Cuda.

3

How is the #sparse-attention repository ranking calculated?

We track 18 active open source repositories tagged with #sparse-attention (500+ stars), totaling 62,491 stars with an average of 3,471 stars per repository. Rankings update daily from the GitHub API.