Git Stars Git Stars

#attention Repositories

Discover 29 active open source GitHub repositories tagged with #attention, accumulating 203,649 total stars (averaging 7,022 stars per project). Leading this topic is labmlai/annotated_deep_learning_paper_implementations with 67,780 stars.

29
Total Repositories
203,649
Total Stars
7,022
Avg Stars/Repo
1
labmlai/annotated_deep_learning_paper_implementations

๐Ÿง‘โ€๐Ÿซ 60+ Implementations/tutorials of deep learning papers with side-by-side notes ๐Ÿ“; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), ๐ŸŽฎ reinforcement learning (ppo, dqn), capsnet, distillation, ... ๐Ÿง 

labmlai
โญ 67,780 Python ๐Ÿ‘ค labmlai Organization ๐Ÿ• Updated 2026-11-02 ๐Ÿ“… Created 2020-08-25
2
sgl-project/sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

sgl-project
โญ 38,899 Python ๐Ÿ‘ค sgl-project Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-01-08
3
kyegomez/OpenMythos

A theoretical reconstruction of the Claude Mythos architecture, built from first principles using the available research literature.

kyegomez
โญ 14,932 Python ๐Ÿ‘ค kyegomez User ๐Ÿ• Updated 2026-05-23 ๐Ÿ“… Created 2026-04-18
4
graykode/nlp-tutorial

Natural Language Processing Tutorial for Deep Learning Researchers

graykode
โญ 14,826 Jupyter Notebook ๐Ÿ‘ค graykode User ๐Ÿ• Updated 2024-02-21 ๐Ÿ“… Created 2019-01-09
5
xmu-xiaoma666/External-Attention-pytorch

๐Ÿ€ Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.โญโญโญ

xmu-xiaoma666
โญ 12,182 Python ๐Ÿ‘ค xmu-xiaoma666 User ๐Ÿ• Updated 2026-03-16 ๐Ÿ“… Created 2021-05-08
6
jadore801120/attention-is-all-you-need-pytorch

A PyTorch implementation of the Transformer model in "Attention is All You Need".

jadore801120
โญ 9,576 Python ๐Ÿ‘ค jadore801120 User ๐Ÿ• Updated 2024-04-16 ๐Ÿ“… Created 2017-06-14
7
flashinfer-ai/flashinfer

FlashInfer: Kernel Library for LLM Serving

flashinfer-ai
โญ 6,982 Cuda ๐Ÿ‘ค flashinfer-ai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-07-22
8
bentrevett/pytorch-seq2seq

Tutorials on implementing a few sequence-to-sequence (seq2seq) models with PyTorch and TorchText.

bentrevett
โญ 5,664 Jupyter Notebook ๐Ÿ‘ค bentrevett User ๐Ÿ• Updated 2024-01-20 ๐Ÿ“… Created 2018-07-17
9
hyunwoongko/transformer

Transformer: PyTorch Implementation of "Attention Is All You Need"

hyunwoongko
โญ 4,609 Python ๐Ÿ‘ค hyunwoongko User ๐Ÿ• Updated 2025-07-15 ๐Ÿ“… Created 2019-10-15
10
km1994/nlp_paper_study

This repository primarily contains notes on studying top-conference papers related to NLP algorithm engineers.

km1994
โญ 4,035 C++ ๐Ÿ‘ค km1994 User ๐Ÿ• Updated 2026-10-11 ๐Ÿ“… Created 2019-05-18
11
thu-ml/SageAttention

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

thu-ml
โญ 4,017 Cuda ๐Ÿ‘ค thu-ml Organization ๐Ÿ• Updated 2026-01-17 ๐Ÿ“… Created 2024-10-03
12
google-research/scenic

Scenic: A Jax Library for Computer Vision Research and Beyond

google-research
โญ 3,848 Python ๐Ÿ‘ค google-research Organization ๐Ÿ• Updated 2026-10-31 ๐Ÿ“… Created 2021-07-12
13
kyegomez/awesome-multi-agent-papers

A compilation of the best multi-agent papers

kyegomez
โญ 1,720 TeX ๐Ÿ‘ค kyegomez User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-02-21
14
argusswift/YOLOv4-pytorch

This is a pytorch repository of YOLOv4, attentive YOLOv4 and mobilenet YOLOv4 with PASCAL VOC and COCO

argusswift
โญ 1,673 Python ๐Ÿ‘ค argusswift User ๐Ÿ• Updated 2024-05-03 ๐Ÿ“… Created 2020-07-19
15
lucidrains/transfusion-pytorch

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

lucidrains
โญ 1,412 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2026-09-04 ๐Ÿ“… Created 2024-08-23
16
thu-ml/SpargeAttn

[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.

thu-ml
โญ 1,269 Cuda ๐Ÿ‘ค thu-ml Organization ๐Ÿ• Updated 2026-02-25 ๐Ÿ“… Created 2025-02-25
17
pprp/SimpleCVReproduction

Replication of simple CV Projects including attention, classification, detection, keypoint detection, etc.

pprp
โญ 1,261 Jupyter Notebook ๐Ÿ‘ค pprp User ๐Ÿ• Updated 2025-08-14 ๐Ÿ“… Created 2019-12-11
18
jmaczan/tiny-vllm

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

jmaczan
โญ 1,207 C++ ๐Ÿ‘ค jmaczan User ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2026-02-09
19
NVIDIA/cudnn-frontend

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

NVIDIA
โญ 1,008 Cuda ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2021-01-22
20
ai4co/rl4co

A PyTorch library for all things Reinforcement Learning (RL) for Combinatorial Optimization (CO)

ai4co
โญ 924 Python ๐Ÿ‘ค ai4co Organization ๐Ÿ• Updated 2026-09-14 ๐Ÿ“… Created 2023-02-23
21
lucidrains/native-sparse-attention-pytorch

Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper

lucidrains
โญ 811 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2025-08-15 ๐Ÿ“… Created 2025-02-19
22
hellonlp/classifier-multi-label

Multi-label text classification, multi-label classification, text classification, multi-label, classifier, text classification, BERT, seq2seq, attention, multi-label-classification

hellonlp
โญ 800 Python ๐Ÿ‘ค hellonlp User ๐Ÿ• Updated 2024-12-11 ๐Ÿ“… Created 2020-07-13
23
kyegomez/LongNet

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

kyegomez
โญ 715 Python ๐Ÿ‘ค kyegomez User ๐Ÿ• Updated 2024-01-07 ๐Ÿ“… Created 2023-07-06
24
eeyhsong/EEG-Conformer

EEG Transformer 2.0. i. Convolutional Transformer for EEG Decoding. ii. Novel visualization - Class Activation Topography.

eeyhsong
โญ 674 Python ๐Ÿ‘ค eeyhsong User ๐Ÿ• Updated 2024-07-24 ๐Ÿ“… Created 2022-07-26
25
therealoliver/Deepdive-llama3-from-scratch

Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.

therealoliver
โญ 629 Jupyter Notebook ๐Ÿ‘ค therealoliver User ๐Ÿ• Updated 2025-02-24 ๐Ÿ“… Created 2025-02-19
26
leondgarse/keras_cv_attention_models

Keras beit,caformer,CMT,CoAtNet,convnext,davit,dino,efficientdet,edgenext,efficientformer,efficientnet,eva,fasternet,fastervit,fastvit,flexivit,gcvit,ghostnet,gpvit,hornet,hiera,iformer,inceptionnext,lcnet,levit,maxvit,mobilevit,moganet,nat,nfnets,pvt,swin,tinynet,tinyvit,uniformer,volo,vanillanet,yolor,yolov7,yolov8,yolox,gpt2,llama2, alias kecam

leondgarse
โญ 624 Python ๐Ÿ‘ค leondgarse User ๐Ÿ• Updated 2026-02-15 ๐Ÿ“… Created 2021-08-02
27
chengchingwen/Transformers.jl

Julia Implementation of Transformer models

chengchingwen
โญ 574 Julia ๐Ÿ‘ค chengchingwen User ๐Ÿ• Updated 2026-10-24 ๐Ÿ“… Created 2018-12-27
28
WenjieDu/SAITS

The official PyTorch implementation of the paper "SAITS: Self-Attention-based Imputation for Time Series". A fast and state-of-the-art (SOTA) deep-learning neural network model for efficient time-series imputation (impute multivariate incomplete time series containing NaN missing data/values with machine learning). https://arxiv.org/abs/2202.08516

WenjieDu
โญ 521 Python ๐Ÿ‘ค WenjieDu User ๐Ÿ• Updated 2026-08-25 ๐Ÿ“… Created 2021-12-07
29
MuLabPKU/TransArch

Design hardware-friendly model architectures and migrate existing LLMs with minimal performance loss

MuLabPKU
โญ 520 Python ๐Ÿ‘ค MuLabPKU Organization ๐Ÿ• Updated 2026-11-11 ๐Ÿ“… Created 2025-01-02

Frequently Asked Questions

1

What is the most popular #attention repository on GitHub?

The top ranked open source #attention project on GitHub is labmlai/annotated_deep_learning_paper_implementations with 67,780 stars. ๐Ÿง‘โ€๐Ÿซ 60+ Implementations/tutorials of deep learning papers with side-by-side notes ๐Ÿ“; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), ๐ŸŽฎ reinforcement learning (ppo, dqn), capsnet, distillation, ... ๐Ÿง 

2

Which programming languages are most used in #attention repositories?

Among the top ranked #attention projects on GitHub, the most common programming languages are Python, Jupyter Notebook, Cuda, C++.

3

How is the #attention repository ranking calculated?

We track 29 active open source repositories tagged with #attention (500+ stars), totaling 203,649 stars with an average of 7,022 stars per repository. Rankings update daily from the GitHub API.