#attention-mechanism Repositories
Discover 50 active open source GitHub repositories tagged with #attention-mechanism, accumulating 274,514 total stars (averaging 5,490 stars per project). Leading this topic is rasbt/LLMs-from-scratch with 108,745 stars.
50 repositories total
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
Qiu Xipengโs Neural Networks and Deep Learning, Second Edition and General Education Edition: E-book, Table of Contents, Learning Resources, and Errata
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
Hackable and optimized Transformers building blocks, supporting a composable construction.
A concise but complete full-attention transformer with a set of promising experimental features from various papers
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites
RuVector provides high-performance, real-time decision-making and agent memory, self-learning AI, and a vector GNN database built in Rust.
Build a modern LLM from scratch. Every line commented. Explained like we are five.
Multilingual Automatic Speech Recognition with word-level timestamps and confidence
This is the official repository for The Hundred-Page Language Models Book by Andriy Burkov
Learn LLM internals step by step - from tokenization to attention to inference optimization.
Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch
PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"
Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch
Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch
A Deep Learning library for EEG Tasks (Signals) Classification, based on TensorFlow.
A curated list of NLP resources focused on Transformer networks, attention mechanism, GPT, BERT, ChatGPT, LLMs, and transfer learning.
Implementation of TabTransformer, attention network for tabular data, in Pytorch
The official implementation for [NeurIPS2025 Oral] Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
Implementation of Attention-based Deep Multiple Instance Learning in PyTorch
Transformer based on a variant of attention that is linear complexity in respect to sequence length
[IJCV] FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention
[ECCV 2020] Learning Enriched Features for Real Image Restoration and Enhancement. SOTA results for image denoising, super-resolution, and image enhancement.
based on yolo-high-level project (detect\pose\classify\segment\):include yolov5\yolov7\yolov8\ core ,improvement research ,SwintransformV2 and Attention Series. training skills, business customization, engineering deployment C
[CVPR'26] ObjectClear: Precise Object and Effect Removal with Adaptive Target-Aware Attention
Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.
[CVPR 2021, Oral] PREDATOR: Registration of 3D Point Clouds with Low Overlap.
[ICLR2025 Spotlight๐ฅ] Official Implementation of TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters
Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch
Implementation of ๐ Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
Democratization of RT-2 "RT-2: New model translates vision and language into action"
The official PyTorch implementation of the paper "SAITS: Self-Attention-based Imputation for Time Series". A fast and state-of-the-art (SOTA) deep-learning neural network model for efficient time-series imputation (impute multivariate incomplete time series containing NaN missing data/values with machine learning). https://arxiv.org/abs/2202.08516
Frequently Asked Questions
What is the most popular #attention-mechanism repository on GitHub?
The top ranked open source #attention-mechanism project on GitHub is rasbt/LLMs-from-scratch with 108,745 stars. Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
Which programming languages are most used in #attention-mechanism repositories?
Among the top ranked #attention-mechanism projects on GitHub, the most common programming languages are Python, Jupyter Notebook, Rust, Swift.
How is the #attention-mechanism repository ranking calculated?
We track 50 active open source repositories tagged with #attention-mechanism (500+ stars), totaling 274,514 stars with an average of 5,490 stars per repository. Rankings update daily from the GitHub API.