Git Stars Git Stars

#gpu-optimization Repositories

Discover 55 active open source GitHub repositories tagged with #gpu-optimization, accumulating 216,210 total stars (averaging 3,931 stars per project). Leading this topic is Dao-AILab/flash-attention with 25,330 stars.

55
Total Repositories
216,210
Total Stars
3,931
Avg Stars/Repo
1
Dao-AILab/flash-attention

Fast and memory-efficient exact attention

Dao-AILab
โญ 25,330 Python ๐Ÿ‘ค Dao-AILab Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2022-05-19
2
verl-project/verl

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

verl-project
โญ 24,302 Python ๐Ÿ‘ค verl-project Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-10-31
3
openai/gpt-oss

gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI

openai
โญ 20,557 Python ๐Ÿ‘ค openai Organization ๐Ÿ• Updated 2026-10-29 ๐Ÿ“… Created 2025-06-23
4
lllyasviel/FramePack

Lets make video diffusion practical!

lllyasviel
โญ 17,293 Python ๐Ÿ‘ค lllyasviel User ๐Ÿ• Updated 2025-10-16 ๐Ÿ“… Created 2025-04-12
โญ 12,176 Python ๐Ÿ‘ค Tongyi-MAI Organization ๐Ÿ• Updated 2026-02-09 ๐Ÿ“… Created 2025-11-26
6
NVIDIA/apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in PyTorch

NVIDIA
โญ 9,010 Python ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2018-04-23
7
lltcggie/waifu2x-caffe

waifu2x Caffe Version

lltcggie
โญ 8,210 C++ ๐Ÿ‘ค lltcggie User ๐Ÿ• Updated 2026-03-25 ๐Ÿ“… Created 2015-05-28
8
zhaochenyang20/Awesome-ML-SYS-Tutorial

My learning notes for ML SYS.

zhaochenyang20
โญ 7,695 HTML ๐Ÿ‘ค zhaochenyang20 User ๐Ÿ• Updated 2026-10-21 ๐Ÿ“… Created 2024-11-09
9
deepspeedai/DeepSpeedExamples

Example models using DeepSpeed

deepspeedai
โญ 6,841 Python ๐Ÿ‘ค deepspeedai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2020-01-29
10
Lightning-AI/lit-llama

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

Lightning-AI
โญ 6,084 Python ๐Ÿ‘ค Lightning-AI Organization ๐Ÿ• Updated 2025-07-01 ๐Ÿ“… Created 2023-03-22
11
bojone/bert4keras

Keras implementation of Transformers for humans

bojone
โญ 5,422 Python ๐Ÿ‘ค bojone User ๐Ÿ• Updated 2024-11-11 ๐Ÿ“… Created 2019-08-26
12
city96/ComfyUI-GGUF

GGUF Quantization support for native ComfyUI models

city96
โญ 4,284 Python ๐Ÿ‘ค city96 User ๐Ÿ• Updated 2026-01-12 ๐Ÿ“… Created 2024-08-15
14
fireice-uk/xmr-stak

Free Monero RandomX Miner and unified CryptoNight miner

fireice-uk
โญ 4,080 C++ ๐Ÿ‘ค fireice-uk User ๐Ÿ• Updated 2024-06-20 ๐Ÿ“… Created 2017-10-03
15
HazyResearch/ThunderKittens

Tile primitives for speedy kernels

HazyResearch
โญ 3,805 Cuda ๐Ÿ‘ค HazyResearch Organization ๐Ÿ• Updated 2026-09-12 ๐Ÿ“… Created 2024-03-04
โญ 3,090 Python ๐Ÿ‘ค mistralai Organization ๐Ÿ• Updated 2026-06-12 ๐Ÿ“… Created 2024-05-24
17
yuyuyzl/EasyVtuber

Based on Talking-head-anime 3, works like Vtube Studio.

yuyuyzl
โญ 3,076 Python ๐Ÿ‘ค yuyuyzl User ๐Ÿ• Updated 2026-02-12 ๐Ÿ“… Created 2022-04-23
18
deepseek-ai/DualPipe

A bidirectional pipeline parallelism algorithm for computation-communication overlap in DeepSeek V3/R1 training.

deepseek-ai
โญ 3,036 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2026-01-14 ๐Ÿ“… Created 2025-02-26
19
huggingface/nanotron

Minimalistic large language model 3D-parallelism training

huggingface
โญ 2,869 Python ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2026-10-08 ๐Ÿ“… Created 2023-09-11
โญ 2,804 Python ๐Ÿ‘ค Mega4alik User ๐Ÿ• Updated 2026-07-27 ๐Ÿ“… Created 2025-08-16
21
gpu-mode/resource-stream

GPU programming related news and material links

gpu-mode
โญ 2,415 ๐Ÿ‘ค gpu-mode Organization ๐Ÿ• Updated 2026-06-15 ๐Ÿ“… Created 2023-12-27
22
aigc-apps/VideoX-Fun

๐Ÿ“น A more flexible framework that can generate videos at any resolution and creates videos from images.

aigc-apps
โญ 2,337 Python ๐Ÿ‘ค aigc-apps Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2024-09-10
23
kijai/ComfyUI-SUPIR

SUPIR upscaling wrapper for ComfyUI

kijai
โญ 2,314 Python ๐Ÿ‘ค kijai User ๐Ÿ• Updated 2026-04-29 ๐Ÿ“… Created 2024-02-28
24
IST-DASLab/gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

IST-DASLab
โญ 2,241 Python ๐Ÿ‘ค IST-DASLab Organization ๐Ÿ• Updated 2024-03-27 ๐Ÿ“… Created 2022-10-19
25
cfregly/ai-performance-engineering

Code, labs, and resources for O'Reilly AI Systems Performance Engineering: GPU optimization, distributed training, inference scaling, and full-stack tuning.

cfregly
โญ 2,233 Python ๐Ÿ‘ค cfregly User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2025-04-21
26
NVIDIA-AI-IOT/Lidar_AI_Solution

A project demonstrating Lidar related AI solutions, including three GPU accelerated Lidar/camera DL networks (PointPillars, CenterPoint, BEVFusion) and the related libs (cuPCL, 3D SparseConvolution, YUV2RGB, cuOSD).

NVIDIA-AI-IOT
โญ 1,914 C++ ๐Ÿ‘ค NVIDIA-AI-IOT Organization ๐Ÿ• Updated 2026-07-28 ๐Ÿ“… Created 2023-05-15
27
NVIDIA/DLSS

NVIDIA DLSS is a new and improved deep learning neural network that boosts frame rates and generates beautiful, sharp images for your games

NVIDIA
โญ 1,684 C ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2026-10-16 ๐Ÿ“… Created 2021-07-27
28
NVIDIA/trt-samples-for-hackathon-cn

Simple samples for TensorRT programming

NVIDIA
โญ 1,676 Python ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2026-11-06 ๐Ÿ“… Created 2021-03-19
29
pytorch/FBGEMM

FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/

pytorch
โญ 1,604 C++ ๐Ÿ‘ค pytorch Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-09-24
โญ 1,599 Python ๐Ÿ‘ค databricks Organization ๐Ÿ• Updated 2026-03-25 ๐Ÿ“… Created 2023-01-26
31
graphdeco-inria/hierarchical-3d-gaussians

Official implementation of the SIGGRAPH 2024 paper "A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets"

graphdeco-inria
โญ 1,427 Python ๐Ÿ‘ค graphdeco-inria Organization ๐Ÿ• Updated 2025-06-10 ๐Ÿ“… Created 2024-07-12
32
deepseek-ai/EPLB

Expert Parallelism Load Balancer

deepseek-ai
โญ 1,384 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-03-24 ๐Ÿ“… Created 2025-02-26
33
vblanco20-1/vulkan-guide

Introductory guide to vulkan.

vblanco20-1
โญ 1,355 SCSS ๐Ÿ‘ค vblanco20-1 User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2020-06-02
34
Ksuriuri/index-tts-vllm

Added vLLM support to IndexTTS for faster inference.

Ksuriuri
โญ 1,244 Python ๐Ÿ‘ค Ksuriuri User ๐Ÿ• Updated 2026-04-13 ๐Ÿ“… Created 2025-05-03
35
deepseek-ai/profile-data

Analyze computation-communication overlap in V3/R1.

deepseek-ai
โญ 1,159 ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-03-21 ๐Ÿ“… Created 2025-02-26
36
NVIDIA/cuda-tile

CUDA Tile IR is an MLIR-based intermediate representation and compiler infrastructure for CUDA kernel optimization, focusing on tile-based computation patterns and optimizations targeting NVIDIA tensor core units.

NVIDIA
โญ 1,061 MLIR ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2026-11-07 ๐Ÿ“… Created 2025-11-05
37
tspeterkim/flash-attention-minimal

Flash Attention in ~100 lines of CUDA (forward pass only)

tspeterkim
โญ 1,036 Cuda ๐Ÿ‘ค tspeterkim User ๐Ÿ• Updated 2024-12-30 ๐Ÿ“… Created 2024-03-07
38
MoonshotAI/checkpoint-engine

Checkpoint-engine is a simple middleware to update model weights in LLM inference engines

MoonshotAI
โญ 1,026 Python ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2026-10-22 ๐Ÿ“… Created 2025-09-08
39
alternbits/awesome-cuda-books

A curated list of best CUDA programming books

alternbits
โญ 1,005 ๐Ÿ‘ค alternbits Organization ๐Ÿ• Updated 2026-05-19 ๐Ÿ“… Created 2026-01-31
40
SandAI-org/MagiAttention

A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training

SandAI-org
โญ 975 Python ๐Ÿ‘ค SandAI-org Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-04-20
41
triton-inference-server/tensorrtllm_backend

The Triton TensorRT-LLM Backend

triton-inference-server
โญ 944 ๐Ÿ‘ค triton-inference-server Organization ๐Ÿ• Updated 2026-11-02 ๐Ÿ“… Created 2023-09-12
42
NVIDIA/multi-gpu-programming-models

Examples demonstrating available options to program multiple GPUs in a single node or a cluster

NVIDIA
โญ 919 Cuda ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2026-11-06 ๐Ÿ“… Created 2018-05-11
43
databricks-academy/large-language-models

Notebooks for Large Language Models (LLMs) Specialization

databricks-academy
โญ 824 Python ๐Ÿ‘ค databricks-academy Organization ๐Ÿ• Updated 2024-05-23 ๐Ÿ“… Created 2023-06-02
44
lihaoyun6/ComfyUI-FlashVSR_Ultra_Fast

Real-Time Diffusion-Based Streaming Video Super-Resolution / Real-Time Video Streaming Super-Resolution Model Based on Diffusion Architecture

lihaoyun6
โญ 811 Python ๐Ÿ‘ค lihaoyun6 User ๐Ÿ• Updated 2025-11-20 ๐Ÿ“… Created 2025-10-21
45
powervr-graphics/Native_SDK

C++ cross-platform 3D graphics SDK. Includes demos & helper code (resource loading etc.) to speed up development of Vulkan, OpenGL ES 2.0 & 3.x applications

powervr-graphics
โญ 783 C++ ๐Ÿ‘ค powervr-graphics Organization ๐Ÿ• Updated 2026-09-22 ๐Ÿ“… Created 2014-10-29
โญ 752 Python ๐Ÿ‘ค Open-Magic-Video Organization ๐Ÿ• Updated 2025-02-18 ๐Ÿ“… Created 2025-02-10
47
balazik/ComfyUI-PuLID-Flux

PuLID-Flux ComfyUI implementation

balazik
โญ 705 Python ๐Ÿ‘ค balazik User ๐Ÿ• Updated 2024-10-03 ๐Ÿ“… Created 2024-10-01
48
modal-labs/llm-finetuning

Guide for fine-tuning Llama/Mistral/CodeLlama models and more

modal-labs
โญ 650 Python ๐Ÿ‘ค modal-labs Organization ๐Ÿ• Updated 2025-10-15 ๐Ÿ“… Created 2023-09-14
50
Jittor/JNeRF

JNeRF is a NeRF benchmark based on Jittor. JNeRF re-implemented instant-ngp and achieved same performance with original paper.

Jittor
โญ 643 C++ ๐Ÿ‘ค Jittor User ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2022-05-21

Frequently Asked Questions

1

What is the most popular #gpu-optimization repository on GitHub?

The top ranked open source #gpu-optimization project on GitHub is Dao-AILab/flash-attention with 25,330 stars. Fast and memory-efficient exact attention

2

Which programming languages are most used in #gpu-optimization repositories?

Among the top ranked #gpu-optimization projects on GitHub, the most common programming languages are Python, C++, Cuda, HTML.

3

How is the #gpu-optimization repository ranking calculated?

We track 55 active open source repositories tagged with #gpu-optimization (500+ stars), totaling 216,210 stars with an average of 3,931 stars per repository. Rankings update daily from the GitHub API.