Git Stars Git Stars

#distributed-training Repositories

Discover 147 active open source GitHub repositories tagged with #distributed-training, accumulating 615,835 total stars (averaging 4,189 stars per project). Leading this topic is tensorflow/models with 77,631 stars.

147
Total Repositories
615,835
Total Stars
4,189
Avg Stars/Repo
1
tensorflow/models

Models and examples built with TensorFlow

tensorflow
โญ 77,631 Python ๐Ÿ‘ค tensorflow Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2016-02-05
2
GokuMohandas/Made-With-ML

Learn how to develop, deploy and iterate on production-grade ML applications.

GokuMohandas
โญ 50,112 Jupyter Notebook ๐Ÿ‘ค GokuMohandas User ๐Ÿ• Updated 2026-03-04 ๐Ÿ“… Created 2018-11-05
3
huggingface/pytorch-image-models

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

huggingface
โญ 37,299 Python ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2019-02-02
4
hpcaitech/Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

hpcaitech
โญ 29,943 Python ๐Ÿ‘ค hpcaitech Organization ๐Ÿ• Updated 2026-04-09 ๐Ÿ“… Created 2024-02-20
5
verl-project/verl

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

verl-project
โญ 24,302 Python ๐Ÿ‘ค verl-project Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-10-31
6
PaddlePaddle/Paddle

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice (ใ€ŽFeiJiangใ€core framework, Deep Learning & Machine Learning high-performance single-node, distributed training, and cross-platform deployment)

PaddlePaddle
โญ 24,149 C++ ๐Ÿ‘ค PaddlePaddle Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2016-08-15
7
pytorch/examples

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.

pytorch
โญ 24,020 Python ๐Ÿ‘ค pytorch Organization ๐Ÿ• Updated 2025-09-01 ๐Ÿ“… Created 2016-08-24
8
huggingface/trl

Train transformer language models with reinforcement learning.

huggingface
โญ 19,788 Python ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-03-27
9
Jiayi-Pan/TinyZero

Minimal reproduction of DeepSeek R1-Zero

Jiayi-Pan
โญ 13,259 Python ๐Ÿ‘ค Jiayi-Pan User ๐Ÿ• Updated 2026-02-27 ๐Ÿ“… Created 2025-01-21
10
PaddlePaddle/PaddleNLP

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

PaddlePaddle
โญ 13,000 Python ๐Ÿ‘ค PaddlePaddle Organization ๐Ÿ• Updated 2026-05-23 ๐Ÿ“… Created 2021-02-05
11
skypilot-org/skypilot

The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.

skypilot-org
โญ 10,869 Python ๐Ÿ‘ค skypilot-org Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2021-08-11
12
Netflix/metaflow

Build, Manage and Deploy AI/ML Systems

Netflix
โญ 10,355 Python ๐Ÿ‘ค Netflix Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2019-09-17
13
THUDM/slime

slime is an LLM training framework for RL Scaling.

THUDM
โญ 8,856 Python ๐Ÿ‘ค THUDM Organization ๐Ÿ• Updated 2026-11-05 ๐Ÿ“… Created 2025-06-18
14
datawhalechina/so-large-lm

Large Model Fundamentals: Understand Large Model Fundamentals in One Article

datawhalechina
โญ 7,791 ๐Ÿ‘ค datawhalechina Organization ๐Ÿ• Updated 2026-06-22 ๐Ÿ“… Created 2023-07-17
15
zhaochenyang20/Awesome-ML-SYS-Tutorial

My learning notes for ML SYS.

zhaochenyang20
โญ 7,695 HTML ๐Ÿ‘ค zhaochenyang20 User ๐Ÿ• Updated 2026-10-21 ๐Ÿ“… Created 2024-11-09
16
deepspeedai/DeepSpeedExamples

Example models using DeepSpeed

deepspeedai
โญ 6,841 Python ๐Ÿ‘ค deepspeedai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2020-01-29
17
allenai/OLMo

Modeling, training, eval, and inference code for OLMo

allenai
โญ 6,739 Python ๐Ÿ‘ค allenai Organization ๐Ÿ• Updated 2025-11-24 ๐Ÿ“… Created 2023-02-20
18
KellerJordan/modded-nanogpt

NanoGPT (124M) in 90 seconds

KellerJordan
โญ 6,058 Python ๐Ÿ‘ค KellerJordan User ๐Ÿ• Updated 2026-09-28 ๐Ÿ“… Created 2024-06-01
19
pytorch/torchtitan

A PyTorch native platform for training generative AI models

pytorch
โญ 5,936 Python ๐Ÿ‘ค pytorch Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-12-13
20
rllm-org/rllm

Democratizing Reinforcement Learning for LLMs

rllm-org
โญ 5,927 Python ๐Ÿ‘ค rllm-org Organization ๐Ÿ• Updated 2026-10-16 ๐Ÿ“… Created 2025-01-26
21
meta-pytorch/torchtune

PyTorch native post-training library

meta-pytorch
โญ 5,820 Python ๐Ÿ‘ค meta-pytorch Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2023-10-20
22
NVIDIA/tacotron2

Tacotron 2 - PyTorch implementation with faster-than-realtime inference

NVIDIA
โญ 5,299 Jupyter Notebook ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2024-06-12 ๐Ÿ“… Created 2018-05-03
23
facebookresearch/moco

PyTorch implementation of MoCo: https://arxiv.org/abs/1911.05722

facebookresearch
โญ 5,139 ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-02-03 ๐Ÿ“… Created 2020-03-17
24
facebookresearch/lingua

Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

facebookresearch
โญ 4,758 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2025-07-18 ๐Ÿ“… Created 2024-10-16
25
mindspore-ai/mindspore

MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.

mindspore-ai
โญ 4,729 C++ ๐Ÿ‘ค mindspore-ai Organization ๐Ÿ• Updated 2026-10-28 ๐Ÿ“… Created 2020-02-11
26
CalvinXKY/InfraTech

Sharing AI Infra knowledge & code practice: PyTorch, vLLM/SGLang, slime/vime framework introduction โšก๏ธ, performance acceleration ๐Ÿš€, large model basics ๐Ÿง , AI software and hardware ๐Ÿ”ง, etc.

CalvinXKY
โญ 4,411 Jupyter Notebook ๐Ÿ‘ค CalvinXKY User ๐Ÿ• Updated 2026-09-20 ๐Ÿ“… Created 2025-11-14
27
alibaba/x-deeplearning

An industrial deep learning framework for high-dimension sparse data

alibaba
โญ 4,306 PureBasic ๐Ÿ‘ค alibaba Organization ๐Ÿ• Updated 2024-09-25 ๐Ÿ“… Created 2018-12-12
28
thinking-machines-lab/tinker-cookbook

Post-training with Tinker

thinking-machines-lab
โญ 4,291 Python ๐Ÿ‘ค thinking-machines-lab Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2025-07-14
29
microsoft/AI-System

System for AI Education Resource.

microsoft
โญ 4,185 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2024-10-25 ๐Ÿ“… Created 2020-09-02
30
IDEA-CCNL/Fengshenbang-LM

Fengshenbang-LM (ๅฐ็ฅžๆฆœๅคงๆจกๅž‹) is an open-source large model system led by the Research Center for Cognitive Computing and Natural Language Processing of the IDEA Institute, serving as the infrastructure for Chinese AIGC and cognitive intelligence.

IDEA-CCNL
โญ 4,116 Python ๐Ÿ‘ค IDEA-CCNL Organization ๐Ÿ• Updated 2026-06-08 ๐Ÿ“… Created 2021-10-28
31
facebookresearch/dlrm

An implementation of a deep learning recommendation model (DLRM)

facebookresearch
โญ 4,076 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-01-12 ๐Ÿ“… Created 2019-05-14
32
FedML-AI/FedML

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

FedML-AI
โญ 4,068 Python ๐Ÿ‘ค FedML-AI Organization ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2020-07-21
33
Stability-AI/stable-audio-tools

Generative models for conditional audio generation

Stability-AI
โญ 3,921 Python ๐Ÿ‘ค Stability-AI Organization ๐Ÿ• Updated 2026-11-01 ๐Ÿ“… Created 2023-05-23
34
FlagAI-Open/FlagAI

FlagAI (Fast LArge-scale General AI models) is a fast, easy-to-use and extensible toolkit for large-scale model.

FlagAI-Open
โญ 3,864 Python ๐Ÿ‘ค FlagAI-Open Organization ๐Ÿ• Updated 2026-07-13 ๐Ÿ“… Created 2022-05-16
35
fundamentalvision/Deformable-DETR

Deformable DETR: Deformable Transformers for End-to-End Object Detection.

fundamentalvision
โญ 3,846 Python ๐Ÿ‘ค fundamentalvision Organization ๐Ÿ• Updated 2024-05-16 ๐Ÿ“… Created 2020-11-30
36
facebookresearch/ReAgent

A platform for Reasoning systems (Reinforcement Learning, Contextual Bandits, etc.)

facebookresearch
โญ 3,726 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2017-07-27
37
facebookresearch/fairscale

PyTorch extensions for high performance and large scale training.

facebookresearch
โญ 3,408 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2025-04-26 ๐Ÿ“… Created 2020-07-07
38
radixark/miles

Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.

radixark
โญ 3,386 Python ๐Ÿ‘ค radixark Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-10-09
โญ 3,258 Python ๐Ÿ‘ค google Organization ๐Ÿ• Updated 2024-07-30 ๐Ÿ“… Created 2021-01-19
40
determined-ai/determined

Determined is an open-source machine learning platform that simplifies distributed training, hyperparameter tuning, experiment tracking, and resource management. Works with PyTorch and TensorFlow.

determined-ai
โญ 3,225 Go ๐Ÿ‘ค determined-ai Organization ๐Ÿ• Updated 2025-03-20 ๐Ÿ“… Created 2020-04-07
โญ 3,090 Python ๐Ÿ‘ค mistralai Organization ๐Ÿ• Updated 2026-06-12 ๐Ÿ“… Created 2024-05-24
42
deepseek-ai/DualPipe

A bidirectional pipeline parallelism algorithm for computation-communication overlap in DeepSeek V3/R1 training.

deepseek-ai
โญ 3,036 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2026-01-14 ๐Ÿ“… Created 2025-02-26
โญ 3,007 Python ๐Ÿ‘ค google-research Organization ๐Ÿ• Updated 2026-08-03 ๐Ÿ“… Created 2021-11-01
44
huggingface/nanotron

Minimalistic large language model 3D-parallelism training

huggingface
โญ 2,869 Python ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2026-10-08 ๐Ÿ“… Created 2023-09-11
45
rail-berkeley/rlkit

Collection of reinforcement learning algorithms

rail-berkeley
โญ 2,841 Python ๐Ÿ‘ค rail-berkeley Organization ๐Ÿ• Updated 2024-06-17 ๐Ÿ“… Created 2018-01-25
46
eric-mitchell/direct-preference-optimization

Reference implementation for DPO (Direct Preference Optimization)

eric-mitchell
โญ 2,826 Python ๐Ÿ‘ค eric-mitchell User ๐Ÿ• Updated 2024-08-11 ๐Ÿ“… Created 2023-06-22
47
km1994/LLMs_interview_notes

This repository mainly records interview questions related to Large Language Model (LLM) Algorithm Engineers.

km1994
โญ 2,670 ๐Ÿ‘ค km1994 User ๐Ÿ• Updated 2026-10-11 ๐Ÿ“… Created 2023-09-16
48
romeokienzler/TensorFlow

Project containing related material for my TensorFlow articles

romeokienzler
โญ 2,597 Jupyter Notebook ๐Ÿ‘ค romeokienzler User ๐Ÿ• Updated 2024-06-17 ๐Ÿ“… Created 2019-05-15
49
LTH14/JiT

PyTorch implementation of JiT https://arxiv.org/abs/2511.13720

LTH14
โญ 2,570 Python ๐Ÿ‘ค LTH14 User ๐Ÿ• Updated 2025-12-08 ๐Ÿ“… Created 2025-11-10
50
learning-at-home/hivemind

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

learning-at-home
โญ 2,554 Python ๐Ÿ‘ค learning-at-home Organization ๐Ÿ• Updated 2026-01-11 ๐Ÿ“… Created 2020-02-27

Frequently Asked Questions

1

What is the most popular #distributed-training repository on GitHub?

The top ranked open source #distributed-training project on GitHub is tensorflow/models with 77,631 stars. Models and examples built with TensorFlow

2

Which programming languages are most used in #distributed-training repositories?

Among the top ranked #distributed-training projects on GitHub, the most common programming languages are Python, Jupyter Notebook, C++, HTML.

3

How is the #distributed-training repository ranking calculated?

We track 147 active open source repositories tagged with #distributed-training (500+ stars), totaling 615,835 stars with an average of 4,189 stars per repository. Rankings update daily from the GitHub API.