Git Stars Git Stars

#transformer-architecture Repositories

147
Total Repositories
594,368
Total Stars
4,043
Avg Stars/Repo
51
PixArt-alpha/PixArt-sigma

PixArt-ฮฃ: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation

PixArt-alpha
โญ 1,884 Python ๐Ÿ‘ค PixArt-alpha Organization ๐Ÿ• Updated 2024-10-31 ๐Ÿ“… Created 2024-02-29
52
QwenLM/Qwen-Audio

The official repo of Qwen-Audio (้€šไน‰ๅƒ้—ฎ-Audio) chat & pretrained large audio language model proposed by Alibaba Cloud.

QwenLM
โญ 1,857 Python ๐Ÿ‘ค QwenLM Organization ๐Ÿ• Updated 2024-07-05 ๐Ÿ“… Created 2023-11-07
53
nlpodyssey/spago

Self-contained Machine Learning and Natural Language Processing library in Go

nlpodyssey
โญ 1,850 Go ๐Ÿ‘ค nlpodyssey Organization ๐Ÿ• Updated 2025-04-01 ๐Ÿ“… Created 2020-01-05
54
DGoettlich/history-llms

Information hub for our project training the largest possible historical LLMs.

DGoettlich
โญ 1,808 ๐Ÿ‘ค DGoettlich User ๐Ÿ• Updated 2025-12-22 ๐Ÿ“… Created 2025-12-14
55
llmgenai/LLMInterviewQuestions

This repository contains LLM (Large language model) interview question asked in top companies like Google, Nvidia , Meta , Microsoft & fortune 500 companies.

llmgenai
โญ 1,781 ๐Ÿ‘ค llmgenai User ๐Ÿ• Updated 2025-02-12 ๐Ÿ“… Created 2024-12-26
โญ 1,677 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-11-18 ๐Ÿ“… Created 2025-09-29
โญ 1,634 ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2025-11-17 ๐Ÿ“… Created 2025-10-29
59
baaivision/Emu3.5

Native Multimodal Models are World Learners

baaivision
โญ 1,567 Python ๐Ÿ‘ค baaivision Organization ๐Ÿ• Updated 2025-12-30 ๐Ÿ“… Created 2025-10-29
60
ZeyueT/AudioX

[ICLR 2026] Repository of AudioX

ZeyueT
โญ 1,559 Python ๐Ÿ‘ค ZeyueT User ๐Ÿ• Updated 2026-03-10 ๐Ÿ“… Created 2025-03-13
โญ 1,503 Python ๐Ÿ‘ค Stability-AI Organization ๐Ÿ• Updated 2025-01-08 ๐Ÿ“… Created 2024-10-09
โญ 1,472 Python ๐Ÿ‘ค Om-Alve User ๐Ÿ• Updated 2025-02-15 ๐Ÿ“… Created 2025-01-12
63
facebookresearch/MobileLLM

MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.

facebookresearch
โญ 1,471 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-04-30 ๐Ÿ“… Created 2024-07-09
64
FireRedTeam/FireRedTTS2

Long-form streaming TTS system for multi-speaker dialogue generation

FireRedTeam
โญ 1,433 Python ๐Ÿ‘ค FireRedTeam Organization ๐Ÿ• Updated 2025-10-26 ๐Ÿ“… Created 2025-09-02
65
bigscience-workshop/Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

bigscience-workshop
โญ 1,426 Python ๐Ÿ‘ค bigscience-workshop Organization ๐Ÿ• Updated 2024-03-20 ๐Ÿ“… Created 2021-07-02
โญ 1,423 Jupyter Notebook ๐Ÿ‘ค ARENA-education Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-10-08
67
thunlp/ERNIE

Source code and dataset for ACL 2019 paper "ERNIE: Enhanced Language Representation with Informative Entities"

thunlp
โญ 1,418 Python ๐Ÿ‘ค thunlp Organization ๐Ÿ• Updated 2024-01-10 ๐Ÿ“… Created 2019-05-17
68
tianweiy/CausVid

(CVPR 2025) From Slow Bidirectional to Fast Autoregressive Video Diffusion Models

tianweiy
โญ 1,414 Python ๐Ÿ‘ค tianweiy User ๐Ÿ• Updated 2025-08-07 ๐Ÿ“… Created 2025-03-14
69
PKU-VCL-3DV/SLAM3R

[CVPR 2025 Highlight] Real-time dense scene reconstruction with SLAM3R

PKU-VCL-3DV
โญ 1,378 Python ๐Ÿ‘ค PKU-VCL-3DV Organization ๐Ÿ• Updated 2025-10-18 ๐Ÿ“… Created 2024-12-11
70
tanishqkumar/beyond-nanogpt

Minimal and annotated implementations of key ideas from modern deep learning research.

tanishqkumar
โญ 1,361 Python ๐Ÿ‘ค tanishqkumar User ๐Ÿ• Updated 2026-01-29 ๐Ÿ“… Created 2025-04-16
71
wzzheng/TPVFormer

[CVPR 2023] An academic alternative to Tesla's occupancy network for autonomous driving.

wzzheng
โญ 1,314 Python ๐Ÿ‘ค wzzheng User ๐Ÿ• Updated 2024-09-07 ๐Ÿ“… Created 2022-11-19
72
agemagician/ProtTrans

ProtTrans is providing state of the art pretrained language models for proteins. ProtTrans was trained on thousands of GPUs from Summit and hundreds of Google TPUs using Transformers Models.

agemagician
โญ 1,310 Jupyter Notebook ๐Ÿ‘ค agemagician User ๐Ÿ• Updated 2025-05-22 ๐Ÿ“… Created 2020-05-11
73
hemingkx/SpeculativeDecodingPapers

๐Ÿ“ฐ Must-read papers and blogs on Speculative Decoding โšก๏ธ

hemingkx
โญ 1,309 ๐Ÿ‘ค hemingkx User ๐Ÿ• Updated 2026-06-27 ๐Ÿ“… Created 2023-09-20
74
glouppe/info8010-deep-learning

Lectures for INFO8010 Deep Learning, ULiรจge

glouppe
โญ 1,294 Jupyter Notebook ๐Ÿ‘ค glouppe User ๐Ÿ• Updated 2026-07-13 ๐Ÿ“… Created 2018-03-14
75
Ma-Lab-Berkeley/CRATE

Code for CRATE (Coding RAte reduction TransformEr).

Ma-Lab-Berkeley
โญ 1,261 Python ๐Ÿ‘ค Ma-Lab-Berkeley Organization ๐Ÿ• Updated 2024-10-23 ๐Ÿ“… Created 2023-05-29
76
ga642381/speech-trident

Awesome speech/audio LLMs, representation learning, and codec models

ga642381
โญ 1,249 ๐Ÿ‘ค ga642381 User ๐Ÿ• Updated 2026-07-10 ๐Ÿ“… Created 2024-04-08
77
willisma/SiT

Official PyTorch Implementation of "SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers"

willisma
โญ 1,228 Python ๐Ÿ‘ค willisma User ๐Ÿ• Updated 2025-12-22 ๐Ÿ“… Created 2023-11-28
78
awslabs/sockeye

Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch

awslabs
โญ 1,217 Python ๐Ÿ‘ค awslabs Organization ๐Ÿ• Updated 2024-10-24 ๐Ÿ“… Created 2017-06-08
โญ 1,209 Python ๐Ÿ‘ค XiangLi1999 User ๐Ÿ• Updated 2024-08-08 ๐Ÿ“… Created 2022-05-25
80
gpu-mode/awesomeMLSys

An ML Systems Onboarding list

gpu-mode
โญ 1,154 ๐Ÿ‘ค gpu-mode Organization ๐Ÿ• Updated 2026-02-19 ๐Ÿ“… Created 2024-05-06
82
HITsz-TMG/Uni-MoE

Uni-MoE: Lychee's Large Multimodal Model Family.

HITsz-TMG
โญ 1,120 Python ๐Ÿ‘ค HITsz-TMG Organization ๐Ÿ• Updated 2026-08-06 ๐Ÿ“… Created 2024-03-04
83
XiaomiMiMo/MiMo-Audio

MiMo-Audio: Audio Language Models are Few-Shot Learners

XiaomiMiMo
โญ 1,095 Python ๐Ÿ‘ค XiaomiMiMo Organization ๐Ÿ• Updated 2026-10-29 ๐Ÿ“… Created 2025-09-19
84
chaytonmin/Awesome-BEV-Perception-Multi-Cameras

Awesome papers about Multi-Camera 3D Object Detection and Segmentation in Bird's-Eye-View, such as DETR3D, BEVDet, BEVFormer, BEVDepth, UniAD

chaytonmin
โญ 1,088 ๐Ÿ‘ค chaytonmin User ๐Ÿ• Updated 2024-04-26 ๐Ÿ“… Created 2022-04-24
85
Alpha-VLLM/Lumina-mGPT-2.0

Lumina-mGPT 2.0: Stand-Alone AutoRegressive Image Modeling

Alpha-VLLM
โญ 1,081 Python ๐Ÿ‘ค Alpha-VLLM Organization ๐Ÿ• Updated 2025-11-03 ๐Ÿ“… Created 2025-04-03
86
dzyim/ilya-sutskever-recommended-reading

It is said that, Ilya Sutskever gave John Carmack this reading list of ~ 30 research papers on deep learning.

dzyim
โญ 1,080 ๐Ÿ‘ค dzyim User ๐Ÿ• Updated 2024-06-04 ๐Ÿ“… Created 2024-05-14
87
gangweix/pixel-perfect-depth

[TPAMI 2026 & NeurIPS 2025] Pixel-Perfect Depth

gangweix
โญ 1,076 Python ๐Ÿ‘ค gangweix User ๐Ÿ• Updated 2026-02-13 ๐Ÿ“… Created 2025-05-26
88
hans0809/MiniMind-in-Depth

Source code analysis of the lightweight large language model MiniMind, including the complete processes for tokenizer, RoPE, MoE, KV Cache, pretraining, SFT, LoRA, and DPO.

hans0809
โญ 1,072 ๐Ÿ‘ค hans0809 User ๐Ÿ• Updated 2025-06-16 ๐Ÿ“… Created 2025-06-16
89
OpenGVLab/SAM-Med2D

Official implementation of SAM-Med2D

OpenGVLab
โญ 1,071 Jupyter Notebook ๐Ÿ‘ค OpenGVLab Organization ๐Ÿ• Updated 2024-06-18 ๐Ÿ“… Created 2023-08-26
90
abertsch72/unlimiformer

Public repo for the NeurIPS 2023 paper "Unlimiformer: Long-Range Transformers with Unlimited Length Input"

abertsch72
โญ 1,063 Python ๐Ÿ‘ค abertsch72 User ๐Ÿ• Updated 2024-03-07 ๐Ÿ“… Created 2023-05-03
91
tspeterkim/flash-attention-minimal

Flash Attention in ~100 lines of CUDA (forward pass only)

tspeterkim
โญ 1,036 Cuda ๐Ÿ‘ค tspeterkim User ๐Ÿ• Updated 2024-12-30 ๐Ÿ“… Created 2024-03-07
92
fla-org/native-sparse-attention

๐Ÿณ Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"

fla-org
โญ 1,030 Python ๐Ÿ‘ค fla-org Organization ๐Ÿ• Updated 2026-02-05 ๐Ÿ“… Created 2025-02-21
93
Ligo-Biosciences/AlphaFold3

Open source implementation of AlphaFold3

Ligo-Biosciences
โญ 1,024 Python ๐Ÿ‘ค Ligo-Biosciences Organization ๐Ÿ• Updated 2024-10-07 ๐Ÿ“… Created 2023-11-11
94
qiuzh20/gated_attention

The official implementation for [NeurIPS2025 Oral] Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free

qiuzh20
โญ 994 Jupyter Notebook ๐Ÿ‘ค qiuzh20 User ๐Ÿ• Updated 2025-12-20 ๐Ÿ“… Created 2025-05-09
95
likejazz/llama3.np

llama3.np is a pure NumPy implementation for Llama 3 model.

likejazz
โญ 991 Python ๐Ÿ‘ค likejazz User ๐Ÿ• Updated 2025-04-27 ๐Ÿ“… Created 2024-05-13
96
wzzheng/StreamVGGT

[ICLR 2026] Streaming 4D Visual Geometry Transformer

wzzheng
โญ 988 Python ๐Ÿ‘ค wzzheng User ๐Ÿ• Updated 2025-10-27 ๐Ÿ“… Created 2025-07-14
97
myshell-ai/JetMoE

Reaching LLaMA2 Performance with 0.1M Dollars

myshell-ai
โญ 988 Python ๐Ÿ‘ค myshell-ai Organization ๐Ÿ• Updated 2024-07-23 ๐Ÿ“… Created 2024-04-02
98
kebijuelun/Awesome-LLM-Learning

Learning Large Language Model (LLM๏ผ‰(ๅคง่ฏญ่จ€ๆจกๅž‹ๅญฆไน )

kebijuelun
โญ 984 TypeScript ๐Ÿ‘ค kebijuelun User ๐Ÿ• Updated 2026-01-05 ๐Ÿ“… Created 2023-08-06
99
nathanrs/tiny-diffusion

A character-level language diffusion model trained on Tiny Shakespeare

nathanrs
โญ 935 Python ๐Ÿ‘ค nathanrs User ๐Ÿ• Updated 2026-01-16 ๐Ÿ“… Created 2025-10-21
100
facebookresearch/spiritlm

Inference code for the paper "Spirit-LM Interleaved Spoken and Written Language Model".

facebookresearch
โญ 926 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2024-10-28 ๐Ÿ“… Created 2024-09-06