Git Stars Git Stars

#multimodal-llm Repositories

Discover 27 active open source GitHub repositories tagged with #multimodal-llm, accumulating 63,138 total stars (averaging 2,338 stars per project). Leading this topic is anthropics/courses with 22,833 stars.

27
Total Repositories
63,138
Total Stars
2,338
Avg Stars/Repo
1
anthropics/courses

Anthropic's educational courses

anthropics
โญ 22,833 Jupyter Notebook ๐Ÿ‘ค anthropics Organization ๐Ÿ• Updated 2026-08-28 ๐Ÿ“… Created 2024-05-30
2
MoonshotAI/Kimi-Audio

Kimi-Audio, an open-source audio foundation model excelling in audio understanding, generation, and conversation

MoonshotAI
โญ 4,654 Python ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2025-06-21 ๐Ÿ“… Created 2025-04-25
โญ 4,630 Python ๐Ÿ‘ค stepfun-ai Organization ๐Ÿ• Updated 2026-03-16 ๐Ÿ“… Created 2025-02-11
4
gpt-omni/mini-omni

open-source multimodal large language model that can hear, talk while thinking. Featuring real-time end-to-end speech input and streaming audio output conversational capabilities.

gpt-omni
โญ 3,504 Python ๐Ÿ‘ค gpt-omni Organization ๐Ÿ• Updated 2024-11-05 ๐Ÿ“… Created 2024-08-27
5
Everlyn-Labs/Everlyn-1

The first open autoregressive foundational video AI model.

Everlyn-Labs
โญ 2,891 ๐Ÿ‘ค Everlyn-Labs Organization ๐Ÿ• Updated 2026-10-07 ๐Ÿ“… Created 2024-10-13
6
chatdoc-com/OCRFlux

OCRFlux is a lightweight yet powerful multimodal toolkit that significantly advances PDF-to-Markdown conversion, excelling in complex layout handling, complicated table parsing and cross-page content merging.

chatdoc-com
โญ 2,532 Python ๐Ÿ‘ค chatdoc-com Organization ๐Ÿ• Updated 2026-04-14 ๐Ÿ“… Created 2025-06-04
7
MoonshotAI/Kimi-K2.5

Open Visual Agentic Intelligence

MoonshotAI
โญ 2,321 ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2026-08-06 ๐Ÿ“… Created 2026-01-30
8
FireRedTeam/FireRedASR

Open-source industrial-grade ASR models supporting Mandarin, Chinese dialects and English, achieving a new SOTA on public Mandarin ASR benchmarks, while also offering outstanding singing lyrics recognition capability.

FireRedTeam
โญ 2,014 Python ๐Ÿ‘ค FireRedTeam Organization ๐Ÿ• Updated 2026-02-25 ๐Ÿ“… Created 2025-01-24
9
Tencent/WeMM-Embedding

WeMM-Embedding is a family of universal multimodal embedding models by the WeChat Vision Team at Tencent, supporting multimodal understanding and retrieval.

Tencent
โญ 1,824 Python ๐Ÿ‘ค Tencent Organization ๐Ÿ• Updated 2026-10-29 ๐Ÿ“… Created 2026-08-25
10
OpenBMB/AgentCPM-GUI

AgentCPM-GUI: An on-device GUI agent for operating Android apps, enhancing reasoning ability with reinforcement fine-tuning for efficient task execution.

OpenBMB
โญ 1,431 Python ๐Ÿ‘ค OpenBMB Organization ๐Ÿ• Updated 2026-01-11 ๐Ÿ“… Created 2025-05-13
11
gokayfem/awesome-vlm-architectures

Curated visual catalog of 200+ vision-language model (VLM/MLLM) architectures: papers, diagrams, training recipes, datasets, and a release timeline for multimodal AI agents.

gokayfem
โญ 1,368 Markdown ๐Ÿ‘ค gokayfem User ๐Ÿ• Updated 2026-10-27 ๐Ÿ“… Created 2024-02-15
12
DLYuanGod/TinyGPT-V

TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones

DLYuanGod
โญ 1,313 Python ๐Ÿ‘ค DLYuanGod User ๐Ÿ• Updated 2026-02-05 ๐Ÿ“… Created 2023-12-28
13
DAMO-NLP-SG/VideoLLaMA2

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

DAMO-NLP-SG
โญ 1,298 Python ๐Ÿ‘ค DAMO-NLP-SG Organization ๐Ÿ• Updated 2025-01-23 ๐Ÿ“… Created 2024-05-31
14
HITsz-TMG/Uni-MoE

Uni-MoE: Lychee's Large Multimodal Model Family.

HITsz-TMG
โญ 1,120 Python ๐Ÿ‘ค HITsz-TMG Organization ๐Ÿ• Updated 2026-08-06 ๐Ÿ“… Created 2024-03-04
15
ZJUI-AI4H/Hulu-Med

A Transparent Generalist Model towards Holistic Medical Vision-Language Understanding

ZJUI-AI4H
โญ 1,031 Python ๐Ÿ‘ค ZJUI-AI4H User ๐Ÿ• Updated 2026-08-30 ๐Ÿ“… Created 2025-10-08
16
tulerfeng/Video-R1

Video-R1: Reinforcing Video Reasoning in MLLMs [๐Ÿ”ฅthe first paper to explore R1 for video]

tulerfeng
โญ 900 Python ๐Ÿ‘ค tulerfeng User ๐Ÿ• Updated 2025-12-14 ๐Ÿ“… Created 2025-02-22
17
UCSB-AI/MiniGPT-5

Official implementation of paper "MiniGPT-5: Interleaved Vision-and-Language Generation via Generative Vokens"

UCSB-AI
โญ 865 Python ๐Ÿ‘ค UCSB-AI Organization ๐Ÿ• Updated 2025-05-08 ๐Ÿ“… Created 2023-10-02
18
ngafar/llama-scan

Transcribe PDFs with local LLMs

ngafar
โญ 807 Python ๐Ÿ‘ค ngafar User ๐Ÿ• Updated 2026-10-22 ๐Ÿ“… Created 2025-06-11
19
FireRedTeam/FireRedASR2S

A SOTA Industrial-Grade All-in-One ASR system with ASR, VAD, LID, and Punc modules. FireRedASR2 supports Chinese (Mandarin, 20+ dialects/accents), English, code-switching, and both speech and singing ASR. FireRedVAD supports speech/singing/music in 100+ langs. FireRedLID supports 100+ langs and 20+ zh dialects. FireRedPunc supports zh and en.

FireRedTeam
โญ 745 Python ๐Ÿ‘ค FireRedTeam Organization ๐Ÿ• Updated 2026-06-02 ๐Ÿ“… Created 2026-02-12
โญ 703 Python ๐Ÿ‘ค stepfun-ai Organization ๐Ÿ• Updated 2026-04-29 ๐Ÿ“… Created 2025-11-11
21
showlab/videollm-online

VideoLLM-online: Online Video Large Language Model for Streaming Video (CVPR 2024)

showlab
โญ 690 Python ๐Ÿ‘ค showlab Organization ๐Ÿ• Updated 2025-11-26 ๐Ÿ“… Created 2024-06-17
22
penghao-wu/vstar

PyTorch Implementation of "V* : Guided Visual Search as a Core Mechanism in Multimodal LLMs"

penghao-wu
โญ 684 Python ๐Ÿ‘ค penghao-wu User ๐Ÿ• Updated 2024-01-07 ๐Ÿ“… Created 2023-12-15
23
Zeyi-Lin/LLM-Finetune

Large language model fine-tuning, Qwen2VL, Qwen2, GLM4 instruction fine-tuning

Zeyi-Lin
โญ 644 Jupyter Notebook ๐Ÿ‘ค Zeyi-Lin User ๐Ÿ• Updated 2025-05-26 ๐Ÿ“… Created 2024-06-10
24
ShaohonChen/Qwen3-SmVL

Performed concatenated fine-tuning of the SmolVLM2 visual head and the Qwen3-0.6B model.

ShaohonChen
โญ 608 Python ๐Ÿ‘ค ShaohonChen User ๐Ÿ• Updated 2025-09-08 ๐Ÿ“… Created 2025-07-12
25
jy0205/LaVIT

LaVIT: Empower the Large Language Model to Understand and Generate Visual Content

jy0205
โญ 601 Jupyter Notebook ๐Ÿ‘ค jy0205 User ๐Ÿ• Updated 2024-10-06 ๐Ÿ“… Created 2023-09-09
26
omarcusdev/zap-gpt-free

This project explores the integration of ChatGPT with WhatsApp, transforming the chatbot into a virtual assistant capable of performing tasks such as talking to friends, answering customer questions, and much more, with a touch of humanization in conversations.

omarcusdev
โญ 597 TypeScript ๐Ÿ‘ค omarcusdev User ๐Ÿ• Updated 2024-12-07 ๐Ÿ“… Created 2024-01-09
27
OpenGVLab/VideoChat-Flash

[ICLR2026] VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling

OpenGVLab
โญ 532 Python ๐Ÿ‘ค OpenGVLab Organization ๐Ÿ• Updated 2026-07-19 ๐Ÿ“… Created 2024-03-24

Frequently Asked Questions

1

What is the most popular #multimodal-llm repository on GitHub?

The top ranked open source #multimodal-llm project on GitHub is anthropics/courses with 22,833 stars. Anthropic's educational courses

2

Which programming languages are most used in #multimodal-llm repositories?

Among the top ranked #multimodal-llm projects on GitHub, the most common programming languages are Python, Jupyter Notebook, Markdown, TypeScript.

3

How is the #multimodal-llm repository ranking calculated?

We track 27 active open source repositories tagged with #multimodal-llm (500+ stars), totaling 63,138 stars with an average of 2,338 stars per repository. Rankings update daily from the GitHub API.