#transformer-architecture Repositories
147 repositories total
PixArt-ฮฃ: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation
The official repo of Qwen-Audio (้ไนๅ้ฎ-Audio) chat & pretrained large audio language model proposed by Alibaba Cloud.
Information hub for our project training the largest possible historical LLMs.
This repository contains LLM (Large language model) interview question asked in top companies like Google, Nvidia , Meta , Microsoft & fortune 500 companies.
MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Source code and dataset for ACL 2019 paper "ERNIE: Enhanced Language Representation with Informative Entities"
(CVPR 2025) From Slow Bidirectional to Fast Autoregressive Video Diffusion Models
Minimal and annotated implementations of key ideas from modern deep learning research.
[CVPR 2023] An academic alternative to Tesla's occupancy network for autonomous driving.
ProtTrans is providing state of the art pretrained language models for proteins. ProtTrans was trained on thousands of GPUs from Summit and hundreds of Google TPUs using Transformers Models.
๐ฐ Must-read papers and blogs on Speculative Decoding โก๏ธ
Official PyTorch Implementation of "SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers"
Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch
Awesome papers about Multi-Camera 3D Object Detection and Segmentation in Bird's-Eye-View, such as DETR3D, BEVDet, BEVFormer, BEVDepth, UniAD
It is said that, Ilya Sutskever gave John Carmack this reading list of ~ 30 research papers on deep learning.
Source code analysis of the lightweight large language model MiniMind, including the complete processes for tokenizer, RoPE, MoE, KV Cache, pretraining, SFT, LoRA, and DPO.
Public repo for the NeurIPS 2023 paper "Unlimiformer: Long-Range Transformers with Unlimited Length Input"
๐ณ Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"
The official implementation for [NeurIPS2025 Oral] Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
Inference code for the paper "Spirit-LM Interleaved Spoken and Written Language Model".