Git Stars Git Stars

#transformer-architecture Repositories

Discover 147 active open source GitHub repositories tagged with #transformer-architecture, accumulating 594,368 total stars (averaging 4,043 stars per project). Leading this topic is karpathy/nanoGPT with 64,617 stars.

147
Total Repositories
594,368
Total Stars
4,043
Avg Stars/Repo
1
karpathy/nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

karpathy
โญ 64,617 Python ๐Ÿ‘ค karpathy User ๐Ÿ• Updated 2025-11-12 ๐Ÿ“… Created 2022-12-28
2
shiyu-coder/Kronos

Kronos: A Foundation Model for the Language of Financial Markets

shiyu-coder
โญ 48,042 Python ๐Ÿ‘ค shiyu-coder User ๐Ÿ• Updated 2026-04-13 ๐Ÿ“… Created 2025-07-01
3
google-research/timesfm

TimesFM (Time Series Foundation Model) is a pretrained time-series foundation model developed by Google Research for time-series forecasting.

google-research
โญ 35,653 Python ๐Ÿ‘ค google-research Organization ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2024-04-29
4
Hannibal046/Awesome-LLM

Awesome-LLM: a curated list of Large Language Model

Hannibal046
โญ 27,212 ๐Ÿ‘ค Hannibal046 User ๐Ÿ• Updated 2025-07-31 ๐Ÿ“… Created 2023-02-17
5
Dao-AILab/flash-attention

Fast and memory-efficient exact attention

Dao-AILab
โญ 25,330 Python ๐Ÿ‘ค Dao-AILab Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2022-05-19
6
facebookresearch/sam2

The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

facebookresearch
โญ 20,204 Jupyter Notebook ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-05-30 ๐Ÿ“… Created 2024-07-29
7
ZJU-LLMs/Foundations-of-LLMs

A book for Learning the Foundations of LLMs

ZJU-LLMs
โญ 18,631 ๐Ÿ‘ค ZJU-LLMs User ๐Ÿ• Updated 2025-12-12 ๐Ÿ“… Created 2024-06-30
8
Stability-AI/StableLM

StableLM: Stability AI Language Models

Stability-AI
โญ 15,784 Jupyter Notebook ๐Ÿ‘ค Stability-AI Organization ๐Ÿ• Updated 2024-04-08 ๐Ÿ“… Created 2023-04-19
9
naklecha/llama3-from-scratch

llama3 implementation one matrix multiplication at a time

naklecha
โญ 15,215 Jupyter Notebook ๐Ÿ‘ค naklecha User ๐Ÿ• Updated 2024-05-23 ๐Ÿ“… Created 2024-05-19
10
facebookresearch/vggt

[CVPR 2025 Best Paper Award] VGGT: Visual Geometry Grounded Transformer

facebookresearch
โญ 14,654 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-05-19 ๐Ÿ“… Created 2025-02-18
โญ 12,779 Jupyter Notebook ๐Ÿ‘ค google-research Organization ๐Ÿ• Updated 2026-10-01 ๐Ÿ“… Created 2020-10-21
12
facebookresearch/sam3

The repository provides code for running inference and finetuning with the Meta Segment Anything Model 3 (SAM 3), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

facebookresearch
โญ 12,369 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2025-07-17
13
PKU-YuanGroup/Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

PKU-YuanGroup
โญ 12,201 Python ๐Ÿ‘ค PKU-YuanGroup Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-02-20
14
MoonshotAI/Kimi-K2

Kimi K2 is the large language model series developed by Moonshot AI team

MoonshotAI
โญ 11,121 ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2026-01-21 ๐Ÿ“… Created 2025-07-03
15
dair-ai/ML-Papers-Explained

Explanation to key concepts in ML

dair-ai
โญ 8,575 ๐Ÿ‘ค dair-ai Organization ๐Ÿ• Updated 2025-06-30 ๐Ÿ“… Created 2023-01-12
16
QwenLM/Qwen-Image

Qwen-Image is a powerful image generation foundation model capable of complex text rendering and precise image editing.

QwenLM
โญ 8,489 Python ๐Ÿ‘ค QwenLM Organization ๐Ÿ• Updated 2026-02-10 ๐Ÿ“… Created 2025-08-03
17
datawhalechina/so-large-lm

Large Model Fundamentals: Understand Large Model Fundamentals in One Article

datawhalechina
โญ 7,791 ๐Ÿ‘ค datawhalechina Organization ๐Ÿ• Updated 2026-06-22 ๐Ÿ“… Created 2023-07-17
18
LargeWorldModel/LWM

Large World Model -- Modeling Text and Video with Millions Context

LargeWorldModel
โญ 7,391 Python ๐Ÿ‘ค LargeWorldModel Organization ๐Ÿ• Updated 2024-10-19 ๐Ÿ“… Created 2024-02-08
19
mit-han-lab/streaming-llm

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

mit-han-lab
โญ 7,165 Python ๐Ÿ‘ค mit-han-lab Organization ๐Ÿ• Updated 2024-07-11 ๐Ÿ“… Created 2023-09-29
20
karminski/one-small-step

This is a simple technical็ง‘ๆ™ฎ tutorial project, mainly focusing on explaining some interesting, cutting-edge technical concepts and principles. Each article strives to be completed within 5 minutes of reading.

karminski
โญ 7,006 Python ๐Ÿ‘ค karminski User ๐Ÿ• Updated 2026-03-08 ๐Ÿ“… Created 2025-01-16
โญ 6,721 Python ๐Ÿ‘ค ByteDance-Seed Organization ๐Ÿ• Updated 2026-07-27 ๐Ÿ“… Created 2025-11-12
22
deepseek-ai/DeepSeek-LLM

DeepSeek LLM: Let there be answers

deepseek-ai
โญ 6,688 Makefile ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2024-02-04 ๐Ÿ“… Created 2023-11-29
23
meta-pytorch/gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

meta-pytorch
โญ 6,244 Python ๐Ÿ‘ค meta-pytorch Organization ๐Ÿ• Updated 2025-08-22 ๐Ÿ“… Created 2023-10-17
24
KellerJordan/modded-nanogpt

NanoGPT (124M) in 90 seconds

KellerJordan
โญ 6,058 Python ๐Ÿ‘ค KellerJordan User ๐Ÿ• Updated 2026-09-28 ๐Ÿ“… Created 2024-06-01
25
OpenGVLab/LLaMA-Adapter

[ICLR 2024] Fine-tuning LLaMA to follow Instructions within 1 Hour and 1.2M Parameters

OpenGVLab
โญ 5,931 Python ๐Ÿ‘ค OpenGVLab Organization ๐Ÿ• Updated 2024-03-14 ๐Ÿ“… Created 2023-03-19
26
bbycroft/llm-viz

3D Visualization of an GPT-style LLM

bbycroft
โญ 5,599 TypeScript ๐Ÿ‘ค bbycroft User ๐Ÿ• Updated 2026-08-10 ๐Ÿ“… Created 2023-03-01
27
bojone/bert4keras

Keras implementation of Transformers for humans

bojone
โญ 5,422 Python ๐Ÿ‘ค bojone User ๐Ÿ• Updated 2024-11-11 ๐Ÿ“… Created 2019-08-26
28
deepseek-ai/DeepSeek-V2

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

deepseek-ai
โญ 4,988 ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2024-09-25 ๐Ÿ“… Created 2024-04-22
29
cmhungsteve/Awesome-Transformer-Attention

An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites

cmhungsteve
โญ 4,987 ๐Ÿ‘ค cmhungsteve User ๐Ÿ• Updated 2024-07-30 ๐Ÿ“… Created 2021-09-15
30
facebookresearch/vjepa2

PyTorch code and models for VJEPA2 self-supervised learning from video.

facebookresearch
โญ 4,911 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-10-29 ๐Ÿ“… Created 2025-04-25
31
karpathy/build-nanogpt

Video+code lecture on building nanoGPT from scratch

karpathy
โญ 4,640 Python ๐Ÿ‘ค karpathy User ๐Ÿ• Updated 2024-08-13 ๐Ÿ“… Created 2024-06-09
33
649453932/Bert-Chinese-Text-Classification-Pytorch

Using Bert, ERNIE for Chinese text classification

649453932
โญ 4,366 Python ๐Ÿ‘ค 649453932 User ๐Ÿ• Updated 2024-06-28 ๐Ÿ“… Created 2019-07-15
34
skindhu/Build-A-Large-Language-Model-CN

"Build a Large Language Model (From Scratch)" is an e-book that delves into the principles and implementation of large language models, suitable for learners who wish to deeply understand the architecture, training process, and application development of large models like GPT. To enable more Chinese readers to access this highly valuable textbook, I have decided to translate it into Chinese and share it as open source via GitHub.

skindhu
โญ 4,188 HTML ๐Ÿ‘ค skindhu User ๐Ÿ• Updated 2026-08-10 ๐Ÿ“… Created 2024-10-23
35
ML-GSAI/LLaDA

Official PyTorch implementation for "Large Language Diffusion Models"

ML-GSAI
โญ 4,020 Python ๐Ÿ‘ค ML-GSAI Organization ๐Ÿ• Updated 2026-07-15 ๐Ÿ“… Created 2025-02-13
36
genmoai/mochi

The best OSS video generation models, created by Genmo

genmoai
โญ 3,766 Python ๐Ÿ‘ค genmoai Organization ๐Ÿ• Updated 2026-10-22 ๐Ÿ“… Created 2024-09-11
37
tekaratzas/RustGPT

An transformer based LLM. Written completely in Rust

tekaratzas
โญ 3,187 Rust ๐Ÿ‘ค tekaratzas User ๐Ÿ• Updated 2025-10-10 ๐Ÿ“… Created 2025-09-13
โญ 3,186 Python ๐Ÿ‘ค stepfun-ai Organization ๐Ÿ• Updated 2025-03-17 ๐Ÿ“… Created 2025-02-08
39
MLNLP-World/LLMs-from-scratch-CN

LLMs-from-scratch project Chinese translation

MLNLP-World
โญ 3,100 Jupyter Notebook ๐Ÿ‘ค MLNLP-World Organization ๐Ÿ• Updated 2026-04-19 ๐Ÿ“… Created 2025-01-20
40
Saiyan-World/goku

[CVPR2025 Highlight] Video Generation Foundation Models: https://saiyan-world.github.io/goku/

Saiyan-World
โญ 2,909 Python ๐Ÿ‘ค Saiyan-World Organization ๐Ÿ• Updated 2025-02-19 ๐Ÿ“… Created 2025-02-10
41
kzl/decision-transformer

Official codebase for Decision Transformer: Reinforcement Learning via Sequence Modeling.

kzl
โญ 2,738 Python ๐Ÿ‘ค kzl User ๐Ÿ• Updated 2024-04-29 ๐Ÿ“… Created 2021-06-02
42
km1994/LLMs_interview_notes

This repository mainly records interview questions related to Large Language Model (LLM) Algorithm Engineers.

km1994
โญ 2,670 ๐Ÿ‘ค km1994 User ๐Ÿ• Updated 2026-10-11 ๐Ÿ“… Created 2023-09-16
44
yuqinie98/PatchTST

An official implementation of PatchTST: "A Time Series is Worth 64 Words: Long-term Forecasting with Transformers." (ICLR 2023) https://arxiv.org/abs/2211.14730

yuqinie98
โญ 2,353 Python ๐Ÿ‘ค yuqinie98 User ๐Ÿ• Updated 2024-08-12 ๐Ÿ“… Created 2022-10-25
45
MineDojo/NitroGen

A Foundation Model for Generalist Gaming Agents

MineDojo
โญ 2,251 Python ๐Ÿ‘ค MineDojo Organization ๐Ÿ• Updated 2026-01-25 ๐Ÿ“… Created 2025-12-17
46
aburkov/theLMbook

This is the official repository for The Hundred-Page Language Models Book by Andriy Burkov

aburkov
โญ 2,194 Jupyter Notebook ๐Ÿ‘ค aburkov User ๐Ÿ• Updated 2026-02-08 ๐Ÿ“… Created 2024-06-28
47
dabochen/spreadsheet-is-all-you-need

A nanoGPT pipeline packed in a spreadsheet

dabochen
โญ 2,142 ๐Ÿ‘ค dabochen User ๐Ÿ• Updated 2024-06-17 ๐Ÿ“… Created 2024-06-08
48
tairov/llama2.mojo

Inference Llama 2 in one file of pure ๐Ÿ”ฅ

tairov
โญ 2,126 Mojo ๐Ÿ‘ค tairov User ๐Ÿ• Updated 2026-09-30 ๐Ÿ“… Created 2023-09-10
49
facebookresearch/blt

Code for BLT research paper

facebookresearch
โญ 2,070 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2025-11-03 ๐Ÿ“… Created 2024-12-09
50
haykgrigo3/TimeCapsuleLLM

A LLM trained only on data from certain time periods to reduce modern bias

haykgrigo3
โญ 2,014 Python ๐Ÿ‘ค haykgrigo3 User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2025-07-02

Frequently Asked Questions

1

What is the most popular #transformer-architecture repository on GitHub?

The top ranked open source #transformer-architecture project on GitHub is karpathy/nanoGPT with 64,617 stars. The simplest, fastest repository for training/finetuning medium-sized GPTs.

2

Which programming languages are most used in #transformer-architecture repositories?

Among the top ranked #transformer-architecture projects on GitHub, the most common programming languages are Python, Jupyter Notebook, Makefile, TypeScript.

3

How is the #transformer-architecture repository ranking calculated?

We track 147 active open source repositories tagged with #transformer-architecture (500+ stars), totaling 594,368 stars with an average of 4,043 stars per repository. Rankings update daily from the GitHub API.