Git Stars Git Stars

#llm-training Repositories

Discover 58 active open source GitHub repositories tagged with #llm-training, accumulating 316,620 total stars (averaging 5,458 stars per project). Leading this topic is karpathy/nanochat with 60,746 stars.

58
Total Repositories
316,620
Total Stars
5,458
Avg Stars/Repo
1
karpathy/nanochat

The best ChatGPT that $100 can buy.

karpathy
โญ 60,746 Python ๐Ÿ‘ค karpathy User ๐Ÿ• Updated 2026-09-07 ๐Ÿ“… Created 2025-10-13
2
gitleaks/gitleaks

Find secrets with Gitleaks ๐Ÿ”‘

gitleaks
โญ 31,234 Go ๐Ÿ‘ค gitleaks Organization ๐Ÿ• Updated 2026-10-28 ๐Ÿ“… Created 2018-01-27
3
karpathy/llm.c

LLM training in simple, raw C/CUDA

karpathy
โญ 30,323 Cuda ๐Ÿ‘ค karpathy User ๐Ÿ• Updated 2025-06-26 ๐Ÿ“… Created 2024-04-08
4
Hannibal046/Awesome-LLM

Awesome-LLM: a curated list of Large Language Model

Hannibal046
โญ 27,212 ๐Ÿ‘ค Hannibal046 User ๐Ÿ• Updated 2025-07-31 ๐Ÿ“… Created 2023-02-17
5
liguodongiot/llm-action

This project aims to share the technical principles and practical experience related to large models (large model engineering, practical application of large models).

liguodongiot
โญ 25,297 HTML ๐Ÿ‘ค liguodongiot User ๐Ÿ• Updated 2026-10-14 ๐Ÿ“… Created 2023-05-23
6
ludwig-ai/ludwig

Low-code framework for building custom LLMs, neural networks, and other AI models

ludwig-ai
โญ 11,793 Python ๐Ÿ‘ค ludwig-ai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2018-12-27
7
skypilot-org/skypilot

The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.

skypilot-org
โญ 10,869 Python ๐Ÿ‘ค skypilot-org Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2021-08-11
8
karpathy/minbpe

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

karpathy
โญ 10,241 Python ๐Ÿ‘ค karpathy User ๐Ÿ• Updated 2024-07-01 ๐Ÿ“… Created 2024-02-16
9
THUDM/slime

slime is an LLM training framework for RL Scaling.

THUDM
โญ 8,856 Python ๐Ÿ‘ค THUDM Organization ๐Ÿ• Updated 2026-11-05 ๐Ÿ“… Created 2025-06-18
10
bojieli/ai-infra-book

ใ€ŠIn-Depth Understanding of AI Infrastructure: Quantitative Analysis and System Designใ€‹ (by Li Bojie), an open-source book manuscript: Starting from hardware constraints and model architectures, it quantitatively derives the design of LLM inference and training systems. Includes the full text, PDF, accompanying computational tools, and experiments.

bojieli
โญ 8,179 Python ๐Ÿ‘ค bojieli User ๐Ÿ• Updated 2026-10-01 ๐Ÿ“… Created 2026-08-22
11
Human-Agent-Society/reef

Infrastructure for continually self-improving agents

Human-Agent-Society
โญ 7,652 Python ๐Ÿ‘ค Human-Agent-Society Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-08-31
12
linkedin/Liger-Kernel

Efficient Triton Kernels for LLM Training

linkedin
โญ 6,707 Python ๐Ÿ‘ค linkedin Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-08-06
13
KellerJordan/modded-nanogpt

NanoGPT (124M) in 90 seconds

KellerJordan
โญ 6,058 Python ๐Ÿ‘ค KellerJordan User ๐Ÿ• Updated 2026-09-28 ๐Ÿ“… Created 2024-06-01
14
rllm-org/rllm

Democratizing Reinforcement Learning for LLMs

rllm-org
โญ 5,927 Python ๐Ÿ‘ค rllm-org Organization ๐Ÿ• Updated 2026-10-16 ๐Ÿ“… Created 2025-01-26
15
h2oai/h2o-llmstudio

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/

h2oai
โญ 5,199 Python ๐Ÿ‘ค h2oai Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2023-04-17
16
facebookresearch/lingua

Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

facebookresearch
โญ 4,758 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2025-07-18 ๐Ÿ“… Created 2024-10-16
17
radixark/miles

Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.

radixark
โญ 3,386 Python ๐Ÿ‘ค radixark Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-10-09
18
openlake-project/openlake

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

openlake-project
โญ 3,089 Rust ๐Ÿ‘ค openlake-project Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2026-04-27
19
openenv/OpenEnv

An interface library for RL post training with environments.

openenv
โญ 2,872 Python ๐Ÿ‘ค openenv Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-10-01
20
qibin0506/Cortex

Building Large Models from Scratch: A Complete Practice from Pretraining to RLHF

qibin0506
โญ 2,705 Python ๐Ÿ‘ค qibin0506 User ๐Ÿ• Updated 2026-08-21 ๐Ÿ“… Created 2025-06-24
21
databricks/dbrx

Code examples and resources for DBRX, a large language model developed by Databricks

databricks
โญ 2,578 Python ๐Ÿ‘ค databricks Organization ๐Ÿ• Updated 2024-05-01 ๐Ÿ“… Created 2024-03-26
22
langfengQ/verl-agent

verl-agent is an extension of veRL, designed for training LLM/VLM agents via RL. verl-agent is also the official code for paper "Group-in-Group Policy Optimization for LLM Agent Training"

langfengQ
โญ 2,457 Python ๐Ÿ‘ค langfengQ User ๐Ÿ• Updated 2026-06-09 ๐Ÿ“… Created 2025-03-23
23
MoonshotAI/MoBA

MoBA: Mixture of Block Attention for Long-Context LLMs

MoonshotAI
โญ 2,122 Python ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2025-04-03 ๐Ÿ“… Created 2025-02-17
24
Open-Reasoner-Zero/Open-Reasoner-Zero

Official Repo for Open-Reasoner-Zero

Open-Reasoner-Zero
โญ 2,094 Python ๐Ÿ‘ค Open-Reasoner-Zero Organization ๐Ÿ• Updated 2025-06-02 ๐Ÿ“… Created 2025-02-19
25
allenai/Olmo-core

PyTorch building blocks for the OLMo ecosystem

allenai
โญ 1,810 Python ๐Ÿ‘ค allenai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-03-23
26
intelligent-machine-learning/dlrover

DLRover: An Automatic Distributed Deep Learning System

intelligent-machine-learning
โญ 1,695 Python ๐Ÿ‘ค intelligent-machine-learning Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2022-06-24
โญ 1,677 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-11-18 ๐Ÿ“… Created 2025-09-29
28
alibaba/Pai-Megatron-Patch

The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.

alibaba
โญ 1,601 Python ๐Ÿ‘ค alibaba Organization ๐Ÿ• Updated 2025-12-15 ๐Ÿ“… Created 2023-09-04
29
utkuozdemir/nvidia_gpu_exporter

NVIDIA GPU exporter for Prometheus using the nvidia-smi binary OR NVML

utkuozdemir
โญ 1,599 Go ๐Ÿ‘ค utkuozdemir User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2021-06-08
30
MoonshotAI/Moonlight

Muon is Scalable for LLM Training

MoonshotAI
โญ 1,523 ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2025-08-03 ๐Ÿ“… Created 2025-02-22
31
bigscience-workshop/Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

bigscience-workshop
โญ 1,426 Python ๐Ÿ‘ค bigscience-workshop Organization ๐Ÿ• Updated 2024-03-20 ๐Ÿ“… Created 2021-07-02
32
NousResearch/atropos

Atropos is a Language Model Reinforcement Learning Environments framework for collecting and evaluating LLM trajectories through diverse environments

NousResearch
โญ 1,324 Python ๐Ÿ‘ค NousResearch Organization ๐Ÿ• Updated 2026-06-29 ๐Ÿ“… Created 2025-04-29
33
Alibaba-NLP/ZeroSearch

ZeroSearch: Incentivize the Search Capability of LLMs without Searching

Alibaba-NLP
โญ 1,305 Python ๐Ÿ‘ค Alibaba-NLP Organization ๐Ÿ• Updated 2025-08-16 ๐Ÿ“… Created 2025-05-07
34
InternScience/GraphGen

GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation

InternScience
โญ 1,236 Python ๐Ÿ‘ค InternScience Organization ๐Ÿ• Updated 2026-11-08 ๐Ÿ“… Created 2025-01-08
35
open-gigaai/giga-train

GigaTrain: An Efficient and Scalable Training Framework for AI Models

open-gigaai
โญ 1,074 Python ๐Ÿ‘ค open-gigaai Organization ๐Ÿ• Updated 2026-08-25 ๐Ÿ“… Created 2025-09-26
36
volcengine/veScale

Byted PyTorch Distributed for Hyperscale Training of LLMs and RLs

volcengine
โญ 1,051 Python ๐Ÿ‘ค volcengine Organization ๐Ÿ• Updated 2026-11-11 ๐Ÿ“… Created 2024-02-26
37
tingaicompass/AI-Compass

"AI-Compass" will guide the community in navigating the ocean of AI technology. Whether you are a beginner or an advanced developer, you can find a path towards all major AI directions here. It aims to help developers systematically understand AIโ€™s core concepts, mainstream technologies, cuttingโ€‘edge trends, and, through practice, master the entire process from theory to implementation.

tingaicompass
โญ 1,000 Python ๐Ÿ‘ค tingaicompass User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2025-05-27
38
yandex/YaFSDP

YaFSDP: Yet another Fully Sharded Data Parallel

yandex
โญ 992 Python ๐Ÿ‘ค yandex Organization ๐Ÿ• Updated 2026-10-26 ๐Ÿ“… Created 2024-05-27
39
myshell-ai/JetMoE

Reaching LLaMA2 Performance with 0.1M Dollars

myshell-ai
โญ 988 Python ๐Ÿ‘ค myshell-ai Organization ๐Ÿ• Updated 2024-07-23 ๐Ÿ“… Created 2024-04-02
โญ 986 Python ๐Ÿ‘ค multimodal-art-projection Organization ๐Ÿ• Updated 2025-02-07 ๐Ÿ“… Created 2024-05-08
41
yongliang-wu/DFT

[ICLR 2026] On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification.

yongliang-wu
โญ 932 Python ๐Ÿ‘ค yongliang-wu User ๐Ÿ• Updated 2026-08-01 ๐Ÿ“… Created 2025-08-01
42
chrisliu298/awesome-on-policy-distillation

A curated collection of papers, technical reports, frameworks, and tools for on-policy distillation (OPD) of large language models

chrisliu298
โญ 926 ๐Ÿ‘ค chrisliu298 User ๐Ÿ• Updated 2026-10-06 ๐Ÿ“… Created 2026-03-22
43
EleutherAI/cookbook

Deep learning for dummies. All the practical details and useful utilities that go into working with real models.

EleutherAI
โญ 858 Python ๐Ÿ‘ค EleutherAI Organization ๐Ÿ• Updated 2026-10-27 ๐Ÿ“… Created 2023-12-20
44
qiufengqijun/mini_qwen

This is a project to train large language models from scratch, including pre-training, fine-tuning, and direct preference optimization, with the model having 1B parameters and supporting Chinese and English.

qiufengqijun
โญ 849 Python ๐Ÿ‘ค qiufengqijun User ๐Ÿ• Updated 2025-02-18 ๐Ÿ“… Created 2025-01-18
46
sail-sg/Adan

Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models

sail-sg
โญ 818 Python ๐Ÿ‘ค sail-sg Organization ๐Ÿ• Updated 2025-06-08 ๐Ÿ“… Created 2022-09-01
47
datawhalechina/llm-algo-leetcode

Hands-on LLM algorithms and systems with PyTorch, Triton & CUDA. โ€œLLM Algorithms and Systems Tutorialโ€: model architectures and training, post-training, inference acceleration, performance and operator optimization, with companion Notebook exercises, answer explanations, tests, and experiments.

datawhalechina
โญ 811 Jupyter Notebook ๐Ÿ‘ค datawhalechina Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2026-04-01
48
SimpleBerry/LLaMA-O1

Large Reasoning Models

SimpleBerry
โญ 806 Python ๐Ÿ‘ค SimpleBerry Organization ๐Ÿ• Updated 2024-12-03 ๐Ÿ“… Created 2024-10-10
49
ghimiresunil/LLM-PowerHouse-A-Curated-Guide-for-Large-Language-Models-with-Custom-Training-and-Inferencing

LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.

ghimiresunil
โญ 734 Jupyter Notebook ๐Ÿ‘ค ghimiresunil User ๐Ÿ• Updated 2026-03-13 ๐Ÿ“… Created 2023-05-28
50
RUCAIBox/R1-Searcher

R1-searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning

RUCAIBox
โญ 722 Python ๐Ÿ‘ค RUCAIBox Organization ๐Ÿ• Updated 2025-08-05 ๐Ÿ“… Created 2025-02-26

Frequently Asked Questions

1

What is the most popular #llm-training repository on GitHub?

The top ranked open source #llm-training project on GitHub is karpathy/nanochat with 60,746 stars. The best ChatGPT that $100 can buy.

2

Which programming languages are most used in #llm-training repositories?

Among the top ranked #llm-training projects on GitHub, the most common programming languages are Python, Jupyter Notebook, Go, Cuda.

3

How is the #llm-training repository ranking calculated?

We track 58 active open source repositories tagged with #llm-training (500+ stars), totaling 316,620 stars with an average of 5,458 stars per repository. Rankings update daily from the GitHub API.