Git Stars Git Stars

#mixture-of-experts Repositories

Discover 78 active open source GitHub repositories tagged with #mixture-of-experts, accumulating 392,641 total stars (averaging 5,033 stars per project). Leading this topic is deepseek-ai/DeepSeek-V3 with 104,417 stars.

78
Total Repositories
392,641
Total Stars
5,033
Avg Stars/Repo
โญ 104,417 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-08-28 ๐Ÿ“… Created 2024-12-26
2
xai-org/grok-1

Grok open release

xai-org
โญ 50,567 Python ๐Ÿ‘ค xai-org Organization ๐Ÿ• Updated 2024-08-30 ๐Ÿ“… Created 2024-03-17
3
deepspeedai/DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

deepspeedai
โญ 43,348 Python ๐Ÿ‘ค deepspeedai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-01-23
4
openai/gpt-oss

gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI

openai
โญ 20,557 Python ๐Ÿ‘ค openai Organization ๐Ÿ• Updated 2026-10-29 ๐Ÿ“… Created 2025-06-23
5
FareedKhan-dev/kimi-k3-in-c

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

FareedKhan-dev
โญ 12,523 C ๐Ÿ‘ค FareedKhan-dev User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2026-08-01
6
MoonshotAI/Kimi-K2

Kimi K2 is the large language model series developed by Moonshot AI team

MoonshotAI
โญ 11,121 ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2026-01-21 ๐Ÿ“… Created 2025-07-03
7
deepseek-ai/DeepEP

DeepEP: an efficient expert-parallel communication library

deepseek-ai
โญ 10,462 Cuda ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2025-02-17
8
bojieli/ai-infra-book

ใ€ŠIn-Depth Understanding of AI Infrastructure: Quantitative Analysis and System Designใ€‹ (by Li Bojie), an open-source book manuscript: Starting from hardware constraints and model architectures, it quantitatively derives the design of LLM inference and training systems. Includes the full text, PDF, accompanying computational tools, and experiments.

bojieli
โญ 8,179 Python ๐Ÿ‘ค bojieli User ๐Ÿ• Updated 2026-10-01 ๐Ÿ“… Created 2026-08-22
9
deepseek-ai/open-infra-index

Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation

deepseek-ai
โญ 8,007 ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-05-15 ๐Ÿ“… Created 2025-02-21
10
deepseek-ai/DeepSeek-Coder-V2

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

deepseek-ai
โญ 7,030 ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-11-11 ๐Ÿ“… Created 2024-06-14
11
ByteDance-Seed/Bagel

Open-source unified multimodal model

ByteDance-Seed
โญ 6,223 Python ๐Ÿ‘ค ByteDance-Seed Organization ๐Ÿ• Updated 2026-05-04 ๐Ÿ“… Created 2025-04-17
12
deepseek-ai/DeepSeek-VL2

DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding

deepseek-ai
โญ 5,297 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-02-26 ๐Ÿ“… Created 2024-12-13
13
ztxz16/fastllm

fastllm is a high-performance large language model inference library with no backend dependencies, helping you run inference efficiently on most devices!

ztxz16
โญ 5,133 C++ ๐Ÿ‘ค ztxz16 User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-05-13
14
deepseek-ai/DeepSeek-V2

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

deepseek-ai
โญ 4,988 ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2024-09-25 ๐Ÿ“… Created 2024-04-22
15
deepseek-ai/Engram

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models

deepseek-ai
โญ 4,835 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2026-01-14 ๐Ÿ“… Created 2026-01-12
16
QwenLM/Qwen3.8

Qwen3.8 is the large language model series developed by the Qwen team at Alibaba Group.

QwenLM
โญ 4,459 ๐Ÿ‘ค QwenLM Organization ๐Ÿ• Updated 2026-08-17 ๐Ÿ“… Created 2025-09-11
17
algorithmicsuperintelligence/optillm

Optimizing inference proxy for LLMs

algorithmicsuperintelligence
โญ 4,343 Python ๐Ÿ‘ค algorithmicsuperintelligence Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2024-08-22
18
ML-GSAI/LLaDA

Official PyTorch implementation for "Large Language Diffusion Models"

ML-GSAI
โญ 4,020 Python ๐Ÿ‘ค ML-GSAI Organization ๐Ÿ• Updated 2026-07-15 ๐Ÿ“… Created 2025-02-13
19
radixark/miles

Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.

radixark
โญ 3,386 Python ๐Ÿ‘ค radixark Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-10-09
20
mw00/project-maya

Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B), and Hy4-Flash (220B) on your own GPUs โ€” starting from 8 GB of VRAM, with NVIDIA and AMD support, one dashboard, and an OpenAI- and Anthropic-compatible API.

mw00
โญ 3,140 C++ ๐Ÿ‘ค mw00 User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-06
21
huggingface/nanotron

Minimalistic large language model 3D-parallelism training

huggingface
โญ 2,869 Python ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2026-10-08 ๐Ÿ“… Created 2023-09-11
22
qibin0506/Cortex

Building Large Models from Scratch: A Complete Practice from Pretraining to RLHF

qibin0506
โญ 2,705 Python ๐Ÿ‘ค qibin0506 User ๐Ÿ• Updated 2026-08-21 ๐Ÿ“… Created 2025-06-24
23
learning-at-home/hivemind

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

learning-at-home
โญ 2,554 Python ๐Ÿ‘ค learning-at-home Organization ๐Ÿ• Updated 2026-01-11 ๐Ÿ“… Created 2020-02-27
24
dvmazur/mixtral-offloading

Run Mixtral-8x7B models in Colab or consumer desktops

dvmazur
โญ 2,327 Python ๐Ÿ‘ค dvmazur User ๐Ÿ• Updated 2024-04-08 ๐Ÿ“… Created 2023-12-15
25
MoonshotAI/Kimi-K2.5

Open Visual Agentic Intelligence

MoonshotAI
โญ 2,321 ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2026-08-06 ๐Ÿ“… Created 2026-01-30
26
PKU-YuanGroup/MoE-LLaVA

ใ€TMM 2025๐Ÿ”ฅใ€‘ Mixture-of-Experts for Large Vision-Language Models

PKU-YuanGroup
โญ 2,321 Python ๐Ÿ‘ค PKU-YuanGroup Organization ๐Ÿ• Updated 2025-07-15 ๐Ÿ“… Created 2023-12-14
27
Yuliang-Liu/Monkey

Monkey (LMM): Image Resolution and Text Label Are Important Things for Large Multi-modal Models (CVPR 2024 Highlight)

Yuliang-Liu
โญ 1,954 Python ๐Ÿ‘ค Yuliang-Liu User ๐Ÿ• Updated 2026-09-25 ๐Ÿ“… Created 2023-11-09
28
deepseek-ai/DeepSeek-MoE

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

deepseek-ai
โญ 1,862 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2024-01-16 ๐Ÿ“… Created 2024-01-02
29
laekov/fastmoe

A fast MoE impl for PyTorch

laekov
โญ 1,853 Python ๐Ÿ‘ค laekov User ๐Ÿ• Updated 2025-02-10 ๐Ÿ“… Created 2021-01-25
30
allenai/Olmo-core

PyTorch building blocks for the OLMo ecosystem

allenai
โญ 1,810 Python ๐Ÿ‘ค allenai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-03-23
31
XueFuzhao/OpenMoE

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

XueFuzhao
โญ 1,648 Python ๐Ÿ‘ค XueFuzhao User ๐Ÿ• Updated 2024-03-08 ๐Ÿ“… Created 2023-08-08
โญ 1,599 Python ๐Ÿ‘ค databricks Organization ๐Ÿ• Updated 2026-03-25 ๐Ÿ“… Created 2023-01-26
33
MoonshotAI/Moonlight

Muon is Scalable for LLM Training

MoonshotAI
โญ 1,523 ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2025-08-03 ๐Ÿ“… Created 2025-02-22
34
allenzren/open-pi-zero

Re-implementation of pi0 vision-language-action (VLA) model from Physical Intelligence

allenzren
โญ 1,470 Python ๐Ÿ‘ค allenzren User ๐Ÿ• Updated 2025-01-31 ๐Ÿ“… Created 2024-12-03
35
BICLab/SpikingBrain-7B

Spiking Brain-inspired Large Models, integrating hybrid efficient attention, MoE modules and spike encoding into its architecture

BICLab
โญ 1,432 Python ๐Ÿ‘ค BICLab User ๐Ÿ• Updated 2026-05-14 ๐Ÿ“… Created 2025-09-03
36
peonist-ai/halogen-flash-server

Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.

peonist-ai
โญ 1,412 Shell ๐Ÿ‘ค peonist-ai Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2026-08-26
37
XiaomiMiMo/MiMo-V2-Flash

MiMo-V2-Flash: Efficient Reasoning, Coding, and Agentic Foundation Model

XiaomiMiMo
โญ 1,399 ๐Ÿ‘ค XiaomiMiMo Organization ๐Ÿ• Updated 2026-10-29 ๐Ÿ“… Created 2025-12-15
38
synchopate/llama-modchip

Zero-GPU CPU inference for 30Bโ€“284B MoE models (Qwen3.8 176.5B cold-cache certified in 28.26/48.35 GiB Cgroup v2, DeepSeek-V4 284B, Nemotron-30B) + RouteCrack 2.0.

synchopate
โญ 1,398 Python ๐Ÿ‘ค synchopate User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-29
39
deepseek-ai/EPLB

Expert Parallelism Load Balancer

deepseek-ai
โญ 1,384 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-03-24 ๐Ÿ“… Created 2025-02-26
โญ 1,368 ๐Ÿ‘ค meituan-longcat Organization ๐Ÿ• Updated 2026-06-23 ๐Ÿ“… Created 2025-08-30
41
simplexai-labs/PolyStrata

Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder, and MiniCPM-V on a second engine built for PolyStrata. One-click installation for Windows and Linux, with an OpenAI/Anthropic API on localhost.

simplexai-labs
โญ 1,324 C++ ๐Ÿ‘ค simplexai-labs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2026-10-06
42
wici-ai/inferssd

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

wici-ai
โญ 1,277 C++ ๐Ÿ‘ค wici-ai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-10-22
43
XueFuzhao/awesome-mixture-of-experts

A collection of AWESOME things about mixture-of-experts

XueFuzhao
โญ 1,247 ๐Ÿ‘ค XueFuzhao User ๐Ÿ• Updated 2024-12-08 ๐Ÿ“… Created 2022-03-30
44
davidmrau/mixture-of-experts

PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538

davidmrau
โญ 1,215 Python ๐Ÿ‘ค davidmrau User ๐Ÿ• Updated 2024-04-19 ๐Ÿ“… Created 2019-07-19
45
MoonshotAI/Kimi-VL

Kimi-VL: Mixture-of-Experts Vision-Language Model for Multimodal Reasoning, Long-Context Understanding, and Strong Agent Capabilities

MoonshotAI
โญ 1,203 ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2025-07-15 ๐Ÿ“… Created 2025-04-09
46
deepseek-ai/profile-data

Analyze computation-communication overlap in V3/R1.

deepseek-ai
โญ 1,159 ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-03-21 ๐Ÿ“… Created 2025-02-26
48
HITsz-TMG/Uni-MoE

Uni-MoE: Lychee's Large Multimodal Model Family.

HITsz-TMG
โญ 1,120 Python ๐Ÿ‘ค HITsz-TMG Organization ๐Ÿ• Updated 2026-08-06 ๐Ÿ“… Created 2024-03-04
49
rhymes-ai/Aria

Codebase for Aria - an Open Multimodal Native MoE

rhymes-ai
โญ 1,089 Jupyter Notebook ๐Ÿ‘ค rhymes-ai Organization ๐Ÿ• Updated 2025-01-22 ๐Ÿ“… Created 2024-09-29
50
allenai/OLMoE

OLMoE: Open Mixture-of-Experts Language Models

allenai
โญ 1,087 Jupyter Notebook ๐Ÿ‘ค allenai Organization ๐Ÿ• Updated 2025-09-23 ๐Ÿ“… Created 2024-06-22

Frequently Asked Questions

1

What is the most popular #mixture-of-experts repository on GitHub?

The top ranked open source #mixture-of-experts project on GitHub is deepseek-ai/DeepSeek-V3 with 104,417 stars.

2

Which programming languages are most used in #mixture-of-experts repositories?

Among the top ranked #mixture-of-experts projects on GitHub, the most common programming languages are Python, C++, Jupyter Notebook, C.

3

How is the #mixture-of-experts repository ranking calculated?

We track 78 active open source repositories tagged with #mixture-of-experts (500+ stars), totaling 392,641 stars with an average of 5,033 stars per repository. Rankings update daily from the GitHub API.