#attention-mechanisms Repositories
Discover 39 active open source GitHub repositories tagged with #attention-mechanisms, accumulating 77,773 total stars (averaging 1,994 stars per project). Leading this topic is lucidrains/PaLM-rlhf-pytorch with 7,858 stars.
39 repositories total
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM
An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites
Deformable DETR: Deformable Transformers for End-to-End Object Detection.
Summary of related papers on visual attention. Related code will be released based on Jittor gradually.
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch
Pytorch implementation of Self-Attention Generative Adversarial Networks (SAGAN)
Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI
Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch
[CVPR 2025] Learning Flow Fields in Attention for Controllable Person Image Generation
The most comprehensive plug-and-play modules for the entire network in 2025, shared for free! CVPR2025, AAAI2025, ICLR2025, TNNLS2025, arXiv2025...... Covers all fields of artificial intelligence (machine learning, deep learning, etc.), suitable for image classification, object detection, instance segmentation, semantic segmentation, panoptic segmentation, pose estimation, medical image segmentation, video object segmentation, image matting, image editing, single object tracking, multiple object tracking, person re-identification, RGBT, image denoising, rain removal, fog removal, shadow removal, blur removal, super-resolution, reflection removal, moirรฉ pattern removal, image restoration, image inpainting, hyperspectral image restoration, image fusion, image colorization, high dynamic range imaging, video and image compression, 3D point clouds, 3D object detection, 3D semantic segmentation, 3D pose estimation, and various other computer vision and image processing tasks, as well as natural language processing, large language models, multimodal, and other AI-related tasks. Continuously updating......
CVPR2023 - Activating More Pixels in Image Super-Resolution Transformer TPAMI - HAT: Hybrid Attention Transformer for Image Restoration
LLM Transparency Tool (LLM-TT), an open-source interactive toolkit for analyzing internal workings of Transformer-based language models. *Check out demo at* https://huggingface.co/spaces/facebook/llm-transparency-tool-demo
Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs
[ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
A paper list of some recent works about Token Compress for Vit and VLM
Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch
Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch
Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch
Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
PyTorch Dual-Attention LSTM-Autoencoder For Multivariate Time Series
Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch
Implementation of a single layer of the MMDiT, proposed in Stable Diffusion 3, in Pytorch
Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks, out of Tsinghua / Ant group
๐ฆPytorch implementation of popular Attention Mechanisms, Vision Transformers, MLP-Like models and CNNs.๐ฅ๐ฅ๐ฅ
Frequently Asked Questions
What is the most popular #attention-mechanisms repository on GitHub?
The top ranked open source #attention-mechanisms project on GitHub is lucidrains/PaLM-rlhf-pytorch with 7,858 stars. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM
Which programming languages are most used in #attention-mechanisms repositories?
Among the top ranked #attention-mechanisms projects on GitHub, the most common programming languages are Python, Jupyter Notebook, TypeScript.
How is the #attention-mechanisms repository ranking calculated?
We track 39 active open source repositories tagged with #attention-mechanisms (500+ stars), totaling 77,773 stars with an average of 1,994 stars per repository. Rankings update daily from the GitHub API.