Git Stars Git Stars

#attention-mechanisms Repositories

Discover 39 active open source GitHub repositories tagged with #attention-mechanisms, accumulating 77,773 total stars (averaging 1,994 stars per project). Leading this topic is lucidrains/PaLM-rlhf-pytorch with 7,858 stars.

39
Total Repositories
77,773
Total Stars
1,994
Avg Stars/Repo
1
lucidrains/PaLM-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

lucidrains
โญ 7,858 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2026-09-20 ๐Ÿ“… Created 2022-12-09
2
lllyasviel/Omost

Your image is almost there!

lllyasviel
โญ 7,661 Python ๐Ÿ‘ค lllyasviel User ๐Ÿ• Updated 2024-07-26 ๐Ÿ“… Created 2024-05-29
3
mit-han-lab/streaming-llm

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

mit-han-lab
โญ 7,165 Python ๐Ÿ‘ค mit-han-lab Organization ๐Ÿ• Updated 2024-07-11 ๐Ÿ“… Created 2023-09-29
5
cmhungsteve/Awesome-Transformer-Attention

An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites

cmhungsteve
โญ 4,987 ๐Ÿ‘ค cmhungsteve User ๐Ÿ• Updated 2024-07-30 ๐Ÿ“… Created 2021-09-15
6
fundamentalvision/Deformable-DETR

Deformable DETR: Deformable Transformers for End-to-End Object Detection.

fundamentalvision
โญ 3,846 Python ๐Ÿ‘ค fundamentalvision Organization ๐Ÿ• Updated 2024-05-16 ๐Ÿ“… Created 2020-11-30
7
Everlyn-Labs/Everlyn-1

The first open autoregressive foundational video AI model.

Everlyn-Labs
โญ 2,891 ๐Ÿ‘ค Everlyn-Labs Organization ๐Ÿ• Updated 2026-10-07 ๐Ÿ“… Created 2024-10-13
8
MenghaoGuo/Awesome-Vision-Attentions

Summary of related papers on visual attention. Related code will be released based on Jittor gradually.

MenghaoGuo
โญ 2,844 Python ๐Ÿ‘ค MenghaoGuo User ๐Ÿ• Updated 2024-10-20 ๐Ÿ“… Created 2021-09-01
9
lucidrains/audiolm-pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

lucidrains
โญ 2,620 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2025-01-12 ๐Ÿ“… Created 2022-09-09
10
heykeetae/Self-Attention-GAN

Pytorch implementation of Self-Attention Generative Adversarial Networks (SAGAN)

heykeetae
โญ 2,603 Python ๐Ÿ‘ค heykeetae User ๐Ÿ• Updated 2024-04-22 ๐Ÿ“… Created 2018-05-28
11
lucidrains/toolformer-pytorch

Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI

lucidrains
โญ 2,055 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2024-07-22 ๐Ÿ“… Created 2023-02-10
12
lucidrains/make-a-video-pytorch

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

lucidrains
โญ 1,983 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2024-05-03 ๐Ÿ“… Created 2022-09-29
13
lehaifeng/T-GCN

Temporal Graph Convolutional Network for Urban Traffic Flow Prediction Method

lehaifeng
โญ 1,782 Jupyter Notebook ๐Ÿ‘ค lehaifeng User ๐Ÿ• Updated 2025-09-01 ๐Ÿ“… Created 2018-11-12
14
lucidrains/alphafold3-pytorch

Implementation of Alphafold 3 from Google Deepmind in Pytorch

lucidrains
โญ 1,700 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2026-08-10 ๐Ÿ“… Created 2024-05-08
15
franciszzj/Leffa

[CVPR 2025] Learning Flow Fields in Attention for Controllable Person Image Generation

franciszzj
โญ 1,675 Python ๐Ÿ‘ค franciszzj User ๐Ÿ• Updated 2025-09-12 ๐Ÿ“… Created 2024-12-05
16
AIFengheshu/Plug-play-modules

The most comprehensive plug-and-play modules for the entire network in 2025, shared for free! CVPR2025, AAAI2025, ICLR2025, TNNLS2025, arXiv2025...... Covers all fields of artificial intelligence (machine learning, deep learning, etc.), suitable for image classification, object detection, instance segmentation, semantic segmentation, panoptic segmentation, pose estimation, medical image segmentation, video object segmentation, image matting, image editing, single object tracking, multiple object tracking, person re-identification, RGBT, image denoising, rain removal, fog removal, shadow removal, blur removal, super-resolution, reflection removal, moirรฉ pattern removal, image restoration, image inpainting, hyperspectral image restoration, image fusion, image colorization, high dynamic range imaging, video and image compression, 3D point clouds, 3D object detection, 3D semantic segmentation, 3D pose estimation, and various other computer vision and image processing tasks, as well as natural language processing, large language models, multimodal, and other AI-related tasks. Continuously updating......

AIFengheshu
โญ 1,547 Python ๐Ÿ‘ค AIFengheshu User ๐Ÿ• Updated 2025-05-24 ๐Ÿ“… Created 2024-10-09
17
XPixelGroup/HAT

CVPR2023 - Activating More Pixels in Image Super-Resolution Transformer TPAMI - HAT: Hybrid Attention Transformer for Image Restoration

XPixelGroup
โญ 1,455 Python ๐Ÿ‘ค XPixelGroup Organization ๐Ÿ• Updated 2024-06-02 ๐Ÿ“… Created 2022-04-27
18
facebookresearch/llm-transparency-tool

LLM Transparency Tool (LLM-TT), an open-source interactive toolkit for analyzing internal workings of Transformer-based language models. *Check out demo at* https://huggingface.co/spaces/facebook/llm-transparency-tool-demo

facebookresearch
โญ 1,228 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2024-12-03 ๐Ÿ“… Created 2023-12-21
19
DAMO-NLP-SG/VideoLLaMA3

Frontier Multimodal Foundation Models for Image and Video Understanding

DAMO-NLP-SG
โญ 1,174 Jupyter Notebook ๐Ÿ‘ค DAMO-NLP-SG Organization ๐Ÿ• Updated 2025-08-14 ๐Ÿ“… Created 2025-01-19
20
gpu-mode/awesomeMLSys

An ML Systems Onboarding list

gpu-mode
โญ 1,154 ๐Ÿ‘ค gpu-mode Organization ๐Ÿ• Updated 2026-02-19 ๐Ÿ“… Created 2024-05-06
21
zhuzilin/ring-flash-attention

Ring attention implementation with flash attention

zhuzilin
โญ 1,054 Python ๐Ÿ‘ค zhuzilin User ๐Ÿ• Updated 2025-09-10 ๐Ÿ“… Created 2024-02-21
22
lucidrains/BS-RoFormer

Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs

lucidrains
โญ 1,030 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2026-11-04 ๐Ÿ“… Created 2023-09-09
23
microsoft/Samba

[ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling

microsoft
โญ 981 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2025-11-16 ๐Ÿ“… Created 2024-06-05
24
daixiangzi/Awesome-Token-Compress

A paper list of some recent works about Token Compress for Vit and VLM

daixiangzi
โญ 966 ๐Ÿ‘ค daixiangzi User ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2024-09-06
25
lucidrains/muse-maskgit-pytorch

Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch

lucidrains
โญ 920 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2024-02-29 ๐Ÿ“… Created 2023-01-03
26
lucidrains/meshgpt-pytorch

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

lucidrains
โญ 858 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2024-12-24 ๐Ÿ“… Created 2023-11-29
27
mystorm16/FastVGGT

[ICLR 2026] FastVGGT: Fast Visual Geometry Transformer

mystorm16
โญ 826 Python ๐Ÿ‘ค mystorm16 User ๐Ÿ• Updated 2026-01-28 ๐Ÿ“… Created 2025-09-01
28
lucidrains/phenaki-pytorch

Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch

lucidrains
โญ 792 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2024-07-29 ๐Ÿ“… Created 2022-09-29
29
D-Keqi/mtla

MTLA: Multi-head Temporal Latent Attention

D-Keqi
โญ 761 Python ๐Ÿ‘ค D-Keqi User ๐Ÿ• Updated 2025-10-06 ๐Ÿ“… Created 2025-05-19
30
labmlai/inspectus

LLM Analytics

labmlai
โญ 718 TypeScript ๐Ÿ‘ค labmlai Organization ๐Ÿ• Updated 2026-07-08 ๐Ÿ“… Created 2024-05-20
31
kyegomez/LongNet

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

kyegomez
โญ 715 Python ๐Ÿ‘ค kyegomez User ๐Ÿ• Updated 2024-01-07 ๐Ÿ“… Created 2023-07-06
32
JulesBelveze/time-series-autoencoder

PyTorch Dual-Attention LSTM-Autoencoder For Multivariate Time Series

JulesBelveze
โญ 715 Python ๐Ÿ‘ค JulesBelveze User ๐Ÿ• Updated 2025-11-11 ๐Ÿ“… Created 2020-06-05
33
lucidrains/magvit2-pytorch

Implementation of MagViT2 Tokenizer in Pytorch

lucidrains
โญ 664 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2025-01-12 ๐Ÿ“… Created 2023-10-10
34
LeapLabTHU/Agent-Attention

Official repository of Agent Attention (ECCV2024)

LeapLabTHU
โญ 657 Python ๐Ÿ‘ค LeapLabTHU User ๐Ÿ• Updated 2024-11-17 ๐Ÿ“… Created 2023-12-12
35
lucidrains/MEGABYTE-pytorch

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

lucidrains
โญ 655 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2024-12-27 ๐Ÿ“… Created 2023-05-15
36
lucidrains/mmdit

Implementation of a single layer of the MMDiT, proposed in Stable Diffusion 3, in Pytorch

lucidrains
โญ 564 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2026-01-18 ๐Ÿ“… Created 2024-05-04
37
lucidrains/iTransformer

Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks, out of Tsinghua / Ant group

lucidrains
โญ 535 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2025-06-12 ๐Ÿ“… Created 2023-10-11
38
changzy00/pytorch-attention

๐Ÿฆ–Pytorch implementation of popular Attention Mechanisms, Vision Transformers, MLP-Like models and CNNs.๐Ÿ”ฅ๐Ÿ”ฅ๐Ÿ”ฅ

changzy00
โญ 530 Python ๐Ÿ‘ค changzy00 User ๐Ÿ• Updated 2024-01-11 ๐Ÿ“… Created 2023-05-26
39
lucidrains/local-attention

An implementation of local windowed attention for language modeling

lucidrains
โญ 502 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2025-07-16 ๐Ÿ“… Created 2020-07-05

Frequently Asked Questions

1

What is the most popular #attention-mechanisms repository on GitHub?

The top ranked open source #attention-mechanisms project on GitHub is lucidrains/PaLM-rlhf-pytorch with 7,858 stars. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

2

Which programming languages are most used in #attention-mechanisms repositories?

Among the top ranked #attention-mechanisms projects on GitHub, the most common programming languages are Python, Jupyter Notebook, TypeScript.

3

How is the #attention-mechanisms repository ranking calculated?

We track 39 active open source repositories tagged with #attention-mechanisms (500+ stars), totaling 77,773 stars with an average of 1,994 stars per repository. Rankings update daily from the GitHub API.