Git Stars Git Stars

#multimodal-ai Repositories

191
Total Repositories
589,873
Total Stars
3,088
Avg Stars/Repo
51
ByteDance-Seed/VeOmni

VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo

ByteDance-Seed
โญ 2,299 Python ๐Ÿ‘ค ByteDance-Seed Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-03-28
52
ActiveVisionLab/Awesome-LLM-3D

Awesome-LLM-3D: a curated list of Multi-modal Large Language Model in 3D world Resources

ActiveVisionLab
โญ 2,284 ๐Ÿ‘ค ActiveVisionLab Organization ๐Ÿ• Updated 2026-04-16 ๐Ÿ“… Created 2023-12-15
53
Liuziyu77/Visual-RFT

Official repository of 'Visual-RFT: Visual Reinforcement Fine-Tuning' & 'Visual-ARFT: Visual Agentic Reinforcement Fine-Tuningโ€™

Liuziyu77
โญ 2,283 Jupyter Notebook ๐Ÿ‘ค Liuziyu77 User ๐Ÿ• Updated 2025-10-29 ๐Ÿ“… Created 2025-02-24
54
mrgoonie/claudekit-skills

All powerful skills of ClaudeKit.cc!

mrgoonie
โญ 2,244 Python ๐Ÿ‘ค mrgoonie User ๐Ÿ• Updated 2026-04-03 ๐Ÿ“… Created 2025-10-23
55
microsoft/LLaVA-Med

Large Language-and-Vision Assistant for Biomedicine, built towards multimodal GPT-4 level capabilities.

microsoft
โญ 2,207 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2025-06-04 ๐Ÿ“… Created 2023-05-20
56
LAION-AI/CLAP

Contrastive Language-Audio Pretraining

LAION-AI
โญ 2,167 Python ๐Ÿ‘ค LAION-AI Organization ๐Ÿ• Updated 2025-05-15 ๐Ÿ“… Created 2022-03-06
57
km1994/LLMsNineStoryDemonTower

ใ€LLMs Nine-Layered Demon Towerใ€‘Sharing LLMs practical experiences and knowledge in fields such as Natural Language Processing (ChatGLM, Chinese-LLaMA-Alpaca, Xiaoyangtuo Vicuna, LLaMA, GPT4ALL, etc.), Information Retrieval (langchain), Language Synthesis, Language Recognition, Multimodal (Stable Diffusion, MiniGPT-4, VisualGLM-6B, Ziya-Visual, etc.).

km1994
โญ 2,140 ๐Ÿ‘ค km1994 User ๐Ÿ• Updated 2024-03-30 ๐Ÿ“… Created 2023-05-01
58
shcherbak-ai/contextgem

Extract structured data and insights from documents using LLMs and decision models

shcherbak-ai
โญ 2,095 Python ๐Ÿ‘ค shcherbak-ai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-10-11
59
Tencent-Hunyuan/HunyuanOCR

HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better

Tencent-Hunyuan
โญ 2,091 Python ๐Ÿ‘ค Tencent-Hunyuan Organization ๐Ÿ• Updated 2026-09-24 ๐Ÿ“… Created 2025-11-18
60
QwenLM/Qwen2-Audio

The official repo of Qwen2-Audio chat & pretrained large audio language model proposed by Alibaba Cloud.

QwenLM
โญ 2,072 Python ๐Ÿ‘ค QwenLM Organization ๐Ÿ• Updated 2025-04-21 ๐Ÿ“… Created 2024-06-24
61
merveenoyan/smol-vision

Recipes for shrinking, optimizing, customizing cutting edge vision models. ๐Ÿ’œ

merveenoyan
โญ 2,004 Jupyter Notebook ๐Ÿ‘ค merveenoyan User ๐Ÿ• Updated 2026-08-12 ๐Ÿ“… Created 2024-03-13
62
NVIDIA-AI-Blueprints/video-search-and-summarization

NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.

NVIDIA-AI-Blueprints
โญ 2,003 Python ๐Ÿ‘ค NVIDIA-AI-Blueprints Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-10-22
63
chn-lee-yumi/MaterialSearch

Semantic search. Search local photos and videos through natural language. AI semantic search local material. Search by image, find local material, match scenes by text description, video frame search, search videos by scene description.

chn-lee-yumi
โญ 1,998 HTML ๐Ÿ‘ค chn-lee-yumi User ๐Ÿ• Updated 2026-09-21 ๐Ÿ“… Created 2023-03-16
65
kimjammer/Neuro

A recreation of Neuro-Sama originally created in 7 days.

kimjammer
โญ 1,960 Python ๐Ÿ‘ค kimjammer User ๐Ÿ• Updated 2025-01-17 ๐Ÿ“… Created 2024-01-07
66
microsoft/Magma

[CVPR 2025] Magma: A Foundation Model for Multimodal AI Agents

microsoft
โญ 1,948 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-03-03 ๐Ÿ“… Created 2024-11-22
67
gpt-omni/mini-omni2

Towards Open-source GPT-4o with Vision, Speech and Duplex Capabilities.

gpt-omni
โญ 1,896 Python ๐Ÿ‘ค gpt-omni Organization ๐Ÿ• Updated 2025-01-16 ๐Ÿ“… Created 2024-10-14
68
ivanfioravanti/chatbot-ollama

Chatbot Ollama is an open source chat UI for Ollama.

ivanfioravanti
โญ 1,892 TypeScript ๐Ÿ‘ค ivanfioravanti User ๐Ÿ• Updated 2025-09-05 ๐Ÿ“… Created 2023-10-01
69
RanFeng/clipsketch-ai

Turn Video Moments into Hand-Drawn Stories

RanFeng
โญ 1,883 TypeScript ๐Ÿ‘ค RanFeng User ๐Ÿ• Updated 2026-01-14 ๐Ÿ“… Created 2025-11-27
70
AutoArk/EVA-OS

EVA OS โ€” A real-time multimodal AIOS for next-generation hardware, enabling your devices being โ€œaliveโ€ and as intelligent as a real brain.

AutoArk
โญ 1,875 ๐Ÿ‘ค AutoArk Organization ๐Ÿ• Updated 2026-08-03 ๐Ÿ“… Created 2025-11-26
71
Audio-AGI/AudioSep

Official implementation of "Separate Anything You Describe"

Audio-AGI
โญ 1,857 Python ๐Ÿ‘ค Audio-AGI Organization ๐Ÿ• Updated 2024-11-26 ๐Ÿ“… Created 2023-08-08
72
facebookresearch/MetaCLIP

NeurIPS 2025 Spotlight; ICLR2024 Spotlight; CVPR 2024; EMNLP 2024

facebookresearch
โญ 1,856 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-08-11 ๐Ÿ“… Created 2023-09-28
73
Core-Mate/OpenGUI

OpenGUI is an Android GUI agent framework for phone-use AI that can see, plan, and operate real mobile apps through the GUI.

Core-Mate
โญ 1,814 HTML ๐Ÿ‘ค Core-Mate Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2026-04-18
74
apple-aiml-research/ml-4m

4M: Massively Multimodal Masked Modeling

apple-aiml-research
โญ 1,814 Python ๐Ÿ‘ค apple-aiml-research Organization ๐Ÿ• Updated 2026-09-11 ๐Ÿ“… Created 2024-04-08
โญ 1,776 Python ๐Ÿ‘ค airockchip User ๐Ÿ• Updated 2026-09-29 ๐Ÿ“… Created 2024-03-14
76
kijai/ComfyUI-Florence2

Inference Microsoft Florence2 VLM

kijai
โญ 1,764 Python ๐Ÿ‘ค kijai User ๐Ÿ• Updated 2026-05-06 ๐Ÿ“… Created 2024-06-19
77
facebookresearch/multimodal

TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.

facebookresearch
โญ 1,747 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2022-01-27
79
JiuhaiChen/BLIP3o

Official implementation of BLIP3o-Series

JiuhaiChen
โญ 1,667 Python ๐Ÿ‘ค JiuhaiChen User ๐Ÿ• Updated 2025-11-29 ๐Ÿ“… Created 2025-04-23
80
waybarrios/vllm-mlx

High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

waybarrios
โญ 1,657 Python ๐Ÿ‘ค waybarrios User ๐Ÿ• Updated 2026-11-07 ๐Ÿ“… Created 2025-12-06
โญ 1,651 Jupyter Notebook ๐Ÿ‘ค Google-Health Organization ๐Ÿ• Updated 2026-06-19 ๐Ÿ“… Created 2025-04-30
82
baaivision/Emu3.5

Native Multimodal Models are World Learners

baaivision
โญ 1,567 Python ๐Ÿ‘ค baaivision Organization ๐Ÿ• Updated 2025-12-30 ๐Ÿ“… Created 2025-10-29
83
microsoft/SoM

[arXiv 2023] Set-of-Mark Prompting for GPT-4V and LMMs

microsoft
โญ 1,499 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2024-08-19 ๐Ÿ“… Created 2023-10-16
84
Nutlope/napkins

napkins.dev โ€“ from screenshot to app

Nutlope
โญ 1,484 TypeScript ๐Ÿ‘ค Nutlope User ๐Ÿ• Updated 2026-10-23 ๐Ÿ“… Created 2024-09-18
โญ 1,479 Python ๐Ÿ‘ค ByteDance-Seed Organization ๐Ÿ• Updated 2026-02-12 ๐Ÿ“… Created 2025-07-30
86
zstar1003/ragflow-plus

Ragflow-Plus is a secondary development version of Ragflow, making it more concise and practical

zstar1003
โญ 1,471 Python ๐Ÿ‘ค zstar1003 User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2025-03-24
87
0nutation/SpeechGPT

SpeechGPT Series: Speech Large Language Models

0nutation
โญ 1,397 Python ๐Ÿ‘ค 0nutation User ๐Ÿ• Updated 2024-07-22 ๐Ÿ“… Created 2023-05-16
88
IDEA-Research/awesome-detection-transformer

Collect some papers about transformer for detection and segmentation. Awesome Detection Transformer for Computer Vision (CV)

IDEA-Research
โญ 1,392 ๐Ÿ‘ค IDEA-Research Organization ๐Ÿ• Updated 2024-07-04 ๐Ÿ“… Created 2022-03-09
89
gokayfem/awesome-vlm-architectures

Curated visual catalog of 200+ vision-language model (VLM/MLLM) architectures: papers, diagrams, training recipes, datasets, and a release timeline for multimodal AI agents.

gokayfem
โญ 1,368 Markdown ๐Ÿ‘ค gokayfem User ๐Ÿ• Updated 2026-10-27 ๐Ÿ“… Created 2024-02-15
90
poe-platform/server-bot-quick-start

Tutorial for Poe server bots

poe-platform
โญ 1,347 Python ๐Ÿ‘ค poe-platform Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2023-05-10
91
gavrielc/Nano-PDF

Edit PDF files with Nano Banana

gavrielc
โญ 1,331 Python ๐Ÿ‘ค gavrielc User ๐Ÿ• Updated 2025-12-03 ๐Ÿ“… Created 2025-11-28
92
ConardLi/easy-llm-cli

An open-source AI agent that is compatible with multiple LLM models

ConardLi
โญ 1,319 TypeScript ๐Ÿ‘ค ConardLi User ๐Ÿ• Updated 2026-02-05 ๐Ÿ“… Created 2025-07-11
93
Meituan-AutoML/MobileVLM

Strong and Open Vision Language Assistant for Mobile Devices

Meituan-AutoML
โญ 1,317 Python ๐Ÿ‘ค Meituan-AutoML Organization ๐Ÿ• Updated 2024-04-15 ๐Ÿ“… Created 2023-12-29
94
hemingkx/SpeculativeDecodingPapers

๐Ÿ“ฐ Must-read papers and blogs on Speculative Decoding โšก๏ธ

hemingkx
โญ 1,309 ๐Ÿ‘ค hemingkx User ๐Ÿ• Updated 2026-06-27 ๐Ÿ“… Created 2023-09-20
95
JMAN730/opendroid

Your Open Autonomous Android Agent โ€” A production-ready, self-planning AI assistant powered by local/remote LLMs and accessibility-driven screen automation.

JMAN730
โญ 1,300 Kotlin ๐Ÿ‘ค JMAN730 User ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2026-05-21
96
opendatalab/LabelLLM

The Open-Source Data Annotation Platform

opendatalab
โญ 1,298 TypeScript ๐Ÿ‘ค opendatalab Organization ๐Ÿ• Updated 2026-07-02 ๐Ÿ“… Created 2024-05-07
โญ 1,293 Python ๐Ÿ‘ค Visual-Agent Organization ๐Ÿ• Updated 2025-11-20 ๐Ÿ“… Created 2025-03-15
98
aiming-lab/Agent0

[ICML'26 & COLM'26] Agent0 Series: Self-Evolving Agents from Zero Data

aiming-lab
โญ 1,292 Python ๐Ÿ‘ค aiming-lab Organization ๐Ÿ• Updated 2026-07-10 ๐Ÿ“… Created 2025-10-20
99
OpenTSLM/OpenTSLM

OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Data

OpenTSLM
โญ 1,273 Python ๐Ÿ‘ค OpenTSLM Organization ๐Ÿ• Updated 2026-07-21 ๐Ÿ“… Created 2025-05-14
100
VILA-Lab/Awesome-DLMs

The official GitHub repo for the survey paper "A Survey on Diffusion Language Models".

VILA-Lab
โญ 1,252 ๐Ÿ‘ค VILA-Lab Organization ๐Ÿ• Updated 2026-10-02 ๐Ÿ“… Created 2025-07-14