#llm-training Repositories
Discover 58 active open source GitHub repositories tagged with #llm-training, accumulating 316,620 total stars (averaging 5,458 stars per project). Leading this topic is karpathy/nanochat with 60,746 stars.
59 repositories total
This project aims to share the technical principles and practical experience related to large models (large model engineering, practical application of large models).
Low-code framework for building custom LLMs, neural networks, and other AI models
The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
ใIn-Depth Understanding of AI Infrastructure: Quantitative Analysis and System Designใ (by Li Bojie), an open-source book manuscript: Starting from hardware constraints and model architectures, it quantitatively derives the design of LLM inference and training systems. Includes the full text, PDF, accompanying computational tools, and experiments.
H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/
Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.
Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.
OpenLake is a high performance storage engine for efficient LLM inference and GPU Training
Building Large Models from Scratch: A Complete Practice from Pretraining to RLHF
Code examples and resources for DBRX, a large language model developed by Databricks
verl-agent is an extension of veRL, designed for training LLM/VLM agents via RL. verl-agent is also the official code for paper "Group-in-Group Policy Optimization for LLM Agent Training"
The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.
NVIDIA GPU exporter for Prometheus using the nvidia-smi binary OR NVML
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Atropos is a Language Model Reinforcement Learning Environments framework for collecting and evaluating LLM trajectories through diverse environments
GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation
"AI-Compass" will guide the community in navigating the ocean of AI technology. Whether you are a beginner or an advanced developer, you can find a path towards all major AI directions here. It aims to help developers systematically understand AIโs core concepts, mainstream technologies, cuttingโedge trends, and, through practice, master the entire process from theory to implementation.
[ICLR 2026] On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification.
A curated collection of papers, technical reports, frameworks, and tools for on-policy distillation (OPD) of large language models
Deep learning for dummies. All the practical details and useful utilities that go into working with real models.
This is a project to train large language models from scratch, including pre-training, fine-tuning, and direct preference optimization, with the model having 1B parameters and supporting Chinese and English.
Hands-on LLM algorithms and systems with PyTorch, Triton & CUDA. โLLM Algorithms and Systems Tutorialโ: model architectures and training, post-training, inference acceleration, performance and operator optimization, with companion Notebook exercises, answer explanations, tests, and experiments.
LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.
R1-searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning
Frequently Asked Questions
What is the most popular #llm-training repository on GitHub?
The top ranked open source #llm-training project on GitHub is karpathy/nanochat with 60,746 stars. The best ChatGPT that $100 can buy.
Which programming languages are most used in #llm-training repositories?
Among the top ranked #llm-training projects on GitHub, the most common programming languages are Python, Jupyter Notebook, Go, Cuda.
How is the #llm-training repository ranking calculated?
We track 58 active open source repositories tagged with #llm-training (500+ stars), totaling 316,620 stars with an average of 5,458 stars per repository. Rankings update daily from the GitHub API.