Git Stars Git Stars

#model-parallelism Repositories

Discover 16 active open source GitHub repositories tagged with #model-parallelism, accumulating 115,629 total stars (averaging 7,226 stars per project). Leading this topic is deepspeedai/DeepSpeed with 43,348 stars.

16
Total Repositories
115,629
Total Stars
7,226
Avg Stars/Repo
Programming Languages: Python TypeScript Cuda
1
deepspeedai/DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

deepspeedai
โญ 43,348 Python ๐Ÿ‘ค deepspeedai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-01-23
2
hpcaitech/ColossalAI

Making large AI models cheaper, faster and more accessible

hpcaitech
โญ 41,421 Python ๐Ÿ‘ค hpcaitech Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2021-10-28
3
huggingface/accelerate

๐Ÿš€ A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

huggingface
โญ 9,952 Python ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2020-10-30
4
facebookresearch/lingua

Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

facebookresearch
โญ 4,758 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2025-07-18 ๐Ÿ“… Created 2024-10-16
5
facebookresearch/fairscale

PyTorch extensions for high performance and large scale training.

facebookresearch
โญ 3,408 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2025-04-26 ๐Ÿ“… Created 2020-07-07
6
ByteDance-Seed/VeOmni

VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo

ByteDance-Seed
โญ 2,299 Python ๐Ÿ‘ค ByteDance-Seed Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-03-28
7
huggingface/picotron

Minimalistic 4D-parallelism distributed training framework for education purpose

huggingface
โญ 2,286 Python ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2025-08-26 ๐Ÿ“… Created 2024-09-18
8
laekov/fastmoe

A fast MoE impl for PyTorch

laekov
โญ 1,853 Python ๐Ÿ‘ค laekov User ๐Ÿ• Updated 2025-02-10 ๐Ÿ“… Created 2021-01-25
9
open-gigaai/giga-train

GigaTrain: An Efficient and Scalable Training Framework for AI Models

open-gigaai
โญ 1,074 Python ๐Ÿ‘ค open-gigaai Organization ๐Ÿ• Updated 2026-08-25 ๐Ÿ“… Created 2025-09-26
10
kebijuelun/Awesome-LLM-Learning

Learning Large Language Model (LLM๏ผ‰(ๅคง่ฏญ่จ€ๆจกๅž‹ๅญฆไน )

kebijuelun
โญ 984 TypeScript ๐Ÿ‘ค kebijuelun User ๐Ÿ• Updated 2026-01-05 ๐Ÿ“… Created 2023-08-06
11
whitecircle/halo

Halo is an open-source framework built by White Circle for training large language and multimodal models

whitecircle
โญ 868 Python ๐Ÿ‘ค whitecircle Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-08-26
12
kakaobrain/torchgpipe

A GPipe implementation in PyTorch

kakaobrain
โญ 861 Python ๐Ÿ‘ค kakaobrain Organization ๐Ÿ• Updated 2024-07-25 ๐Ÿ“… Created 2019-05-10
13
PrimeIntellect-ai/prime-diloco

prime is a framework for efficient, globally distributed training of AI models over the internet.

PrimeIntellect-ai
โญ 857 Python ๐Ÿ‘ค PrimeIntellect-ai Organization ๐Ÿ• Updated 2026-07-17 ๐Ÿ“… Created 2024-09-11
14
nihalsid/mesh-gpt

MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers

nihalsid
โญ 632 ๐Ÿ‘ค nihalsid User ๐Ÿ• Updated 2024-10-31 ๐Ÿ“… Created 2023-11-26
15
XuezheMax/megalodon

Reference implementation of Megalodon 7B model

XuezheMax
โญ 525 Cuda ๐Ÿ‘ค XuezheMax User ๐Ÿ• Updated 2025-05-17 ๐Ÿ“… Created 2024-04-01
16
NVIDIA/NeMo-Framework-Launcher

Provides end-to-end model development pipelines for LLMs and Multimodal models that can be launched on-prem or cloud-native.

NVIDIA
โญ 508 Python ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2025-04-18 ๐Ÿ“… Created 2023-01-05

Frequently Asked Questions

1

What is the most popular #model-parallelism repository on GitHub?

The top ranked open source #model-parallelism project on GitHub is deepspeedai/DeepSpeed with 43,348 stars. DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

2

Which programming languages are most used in #model-parallelism repositories?

Among the top ranked #model-parallelism projects on GitHub, the most common programming languages are Python, TypeScript, Cuda.

3

How is the #model-parallelism repository ranking calculated?

We track 16 active open source repositories tagged with #model-parallelism (500+ stars), totaling 115,629 stars with an average of 7,226 stars per repository. Rankings update daily from the GitHub API.