Git Stars Git Stars

#cuda-kernels Repositories

Discover 34 active open source GitHub repositories tagged with #cuda-kernels, accumulating 129,157 total stars (averaging 3,798 stars per project). Leading this topic is Dao-AILab/flash-attention with 25,330 stars.

34
Total Repositories
129,157
Total Stars
3,798
Avg Stars/Repo
1
Dao-AILab/flash-attention

Fast and memory-efficient exact attention

Dao-AILab
โญ 25,330 Python ๐Ÿ‘ค Dao-AILab Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2022-05-19
2
deepseek-ai/FlashMLA

FlashMLA: Efficient Multi-head Latent Attention Kernels

deepseek-ai
โญ 13,275 C++ ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2026-10-27 ๐Ÿ“… Created 2025-02-21
3
xlite-dev/LeetCUDA

An Open-source book with Modern CUDA learning notes for beginners โ€” BF16/FP8/FP4, HGEMM, CuTe, Flash-Attention, etc.

xlite-dev
โญ 12,208 TeX ๐Ÿ‘ค xlite-dev Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2022-12-17
4
facebookresearch/xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.

facebookresearch
โญ 10,579 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2021-10-13
5
deepseek-ai/DeepEP

DeepEP: an efficient expert-parallel communication library

deepseek-ai
โญ 10,462 Cuda ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2025-02-17
6
NVIDIA/cuda-samples

Samples for CUDA Developers which demonstrates features in CUDA Toolkit

NVIDIA
โญ 9,784 C++ ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2026-11-04 ๐Ÿ“… Created 2018-03-27
7
InternLM/lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

InternLM
โญ 8,153 Python ๐Ÿ‘ค InternLM Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-06-15
8
Rust-GPU/rust-cuda

Ecosystem of libraries and tools for writing and executing fast GPU code fully in Rust.

Rust-GPU
โญ 5,432 Rust ๐Ÿ‘ค Rust-GPU Organization ๐Ÿ• Updated 2026-09-30 ๐Ÿ“… Created 2021-10-17
9
mit-han-lab/llm-awq

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

mit-han-lab
โญ 3,590 Python ๐Ÿ‘ค mit-han-lab Organization ๐Ÿ• Updated 2025-07-17 ๐Ÿ“… Created 2023-06-01
10
crowsonkb/k-diffusion

Karras et al. (2022) diffusion models for PyTorch

crowsonkb
โญ 2,605 Python ๐Ÿ‘ค crowsonkb User ๐Ÿ• Updated 2026-02-12 ๐Ÿ“… Created 2022-06-21
11
NVIDIA/cccl

CUDA Core Compute Libraries

NVIDIA
โญ 2,580 C++ ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-09-17
12
NVlabs/nvdiffrast

Nvdiffrast - Modular Primitives for High-Performance Differentiable Rendering

NVlabs
โญ 1,940 C++ ๐Ÿ‘ค NVlabs Organization ๐Ÿ• Updated 2025-12-08 ๐Ÿ“… Created 2020-11-02
13
NVIDIA-AI-IOT/Lidar_AI_Solution

A project demonstrating Lidar related AI solutions, including three GPU accelerated Lidar/camera DL networks (PointPillars, CenterPoint, BEVFusion) and the related libs (cuPCL, 3D SparseConvolution, YUV2RGB, cuOSD).

NVIDIA-AI-IOT
โญ 1,914 C++ ๐Ÿ‘ค NVIDIA-AI-IOT Organization ๐Ÿ• Updated 2026-07-28 ๐Ÿ“… Created 2023-05-15
14
chelsea0x3b/dfdx

Deep learning in Rust, with shape checked tensors and neural networks

chelsea0x3b
โญ 1,881 Rust ๐Ÿ‘ค chelsea0x3b User ๐Ÿ• Updated 2024-07-23 ๐Ÿ“… Created 2021-10-12
โญ 1,677 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-11-18 ๐Ÿ“… Created 2025-09-29
16
zju3dv/street_gaussians

[ECCV 2024] Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splatting

zju3dv
โญ 1,419 Python ๐Ÿ‘ค zju3dv Organization ๐Ÿ• Updated 2026-09-15 ๐Ÿ“… Created 2023-12-27
17
mapillary/inplace_abn

In-Place Activated BatchNorm for Memory-Optimized Training of DNNs

mapillary
โญ 1,331 Python ๐Ÿ‘ค mapillary Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2017-11-23
18
Vahe1994/AQLM

Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf and PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression https://arxiv.org/abs/2405.14852

Vahe1994
โญ 1,330 Python ๐Ÿ‘ค Vahe1994 User ๐Ÿ• Updated 2026-02-26 ๐Ÿ“… Created 2024-01-12
19
chelsea0x3b/cudarc

Safe rust wrapper around CUDA toolkit

chelsea0x3b
โญ 1,248 Rust ๐Ÿ‘ค chelsea0x3b User ๐Ÿ• Updated 2026-09-24 ๐Ÿ“… Created 2022-09-16
20
pytorch/extension-cpp

C++ extensions in PyTorch

pytorch
โญ 1,203 Python ๐Ÿ‘ค pytorch Organization ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2018-03-03
21
zhuzilin/ring-flash-attention

Ring attention implementation with flash attention

zhuzilin
โญ 1,054 Python ๐Ÿ‘ค zhuzilin User ๐Ÿ• Updated 2025-09-10 ๐Ÿ“… Created 2024-02-21
22
mni-ml/framework

A machine learning library with a TypeScript API and Rust backend. CUDA and WebGPU compatibility. Built to understand how ML frameworks and models work internally.

mni-ml
โญ 1,023 Rust ๐Ÿ‘ค mni-ml Organization ๐Ÿ• Updated 2026-04-20 ๐Ÿ“… Created 2026-01-08
23
NVIDIA/cudnn-frontend

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

NVIDIA
โญ 1,008 Cuda ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2021-01-22
24
SandAI-org/MagiAttention

A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training

SandAI-org
โญ 975 Python ๐Ÿ‘ค SandAI-org Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-04-20
25
NVIDIA/nvbench

CUDA Kernel Benchmarking Library

NVIDIA
โญ 952 Cuda ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2021-03-03
26
h5i-dev/h5i

An Axum-based Rust web framework with Lean 4-verifiable application kernels that run on servers, bare-metal embedded devices, and GPUs, plus integrated security testing.

h5i-dev
โญ 936 Rust ๐Ÿ‘ค h5i-dev Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-03-11
27
mit-han-lab/omniserve

[MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention

mit-han-lab
โญ 840 C++ ๐Ÿ‘ค mit-han-lab Organization ๐Ÿ• Updated 2025-03-06 ๐Ÿ“… Created 2024-05-06
28
flashrt-project/FlashRT

FlashRT is a high-performance realtime inference engine for small-batch, latency-sensitive AI workloads. The flagship integration is production VLA control for Pi0, Pi0.5, GROOT N1.6, and Pi0-FAST. Also support llm e.g, qwen3.6-27B

flashrt-project
โญ 817 C++ ๐Ÿ‘ค flashrt-project Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-04-28
29
pegainfer-project/pegainfer

Pure Rust + CUDA LLM inference engine โ€” no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

pegainfer-project
โญ 748 Rust ๐Ÿ‘ค pegainfer-project Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-02-17
30
perplexityai/pplx-kernels

Perplexity GPU Kernels

perplexityai
โญ 614 C++ ๐Ÿ‘ค perplexityai Organization ๐Ÿ• Updated 2025-11-07 ๐Ÿ“… Created 2025-03-24
31
facebookresearch/mvdust3r

Open source impl of **MV-DUSt3R+ Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds** from Meta Reality Labs. Project page https://mv-dust3rp.github.io/

facebookresearch
โญ 608 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2024-12-19
33
deepreinforce-ai/CUDA-L2

CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning

ornith-ai
โญ 563 Cuda ๐Ÿ‘ค ornith-ai Organization ๐Ÿ• Updated 2026-03-30 ๐Ÿ“… Created 2025-12-02
34
Inception3D/Easi3R

[ICCV 2025] A simple training-free approach adapting DUSt3R for dynamic scenes.

Inception3D
โญ 529 Python ๐Ÿ‘ค Inception3D Organization ๐Ÿ• Updated 2025-04-01 ๐Ÿ“… Created 2025-03-15

Frequently Asked Questions

1

What is the most popular #cuda-kernels repository on GitHub?

The top ranked open source #cuda-kernels project on GitHub is Dao-AILab/flash-attention with 25,330 stars. Fast and memory-efficient exact attention

2

Which programming languages are most used in #cuda-kernels repositories?

Among the top ranked #cuda-kernels projects on GitHub, the most common programming languages are Python, C++, Rust, Cuda.

3

How is the #cuda-kernels repository ranking calculated?

We track 34 active open source repositories tagged with #cuda-kernels (500+ stars), totaling 129,157 stars with an average of 3,798 stars per repository. Rankings update daily from the GitHub API.