Git Stars Git Stars

#multi-modal Repositories

Discover 47 active open source GitHub repositories tagged with #multi-modal, accumulating 208,313 total stars (averaging 4,432 stars per project). Leading this topic is agentscope-ai/agentscope with 36,422 stars.

47
Total Repositories
208,313
Total Stars
4,432
Avg Stars/Repo
1
agentscope-ai/agentscope

Build and run agents you can see, understand and trust.

agentscope-ai
โญ 36,422 Python ๐Ÿ‘ค agentscope-ai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-01-12
2
OpenBMB/MiniCPM-V

See, hear, and interact on your local devices.

OpenBMB
โญ 27,174 Python ๐Ÿ‘ค OpenBMB Organization ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2024-01-29
3
TEN-framework/ten-framework

Open-source framework for conversational voice AI agents

TEN-framework
โญ 11,241 Python ๐Ÿ‘ค TEN-framework Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-06-19
4
OpenGVLab/InternVL

[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. An open-source multimodal dialogue model with performance close to GPT-4o.

OpenGVLab
โญ 10,160 Python ๐Ÿ‘ค OpenGVLab Organization ๐Ÿ• Updated 2025-09-22 ๐Ÿ“… Created 2023-11-22
5
modelscope/modelscope

ModelScope: bring the notion of Model-as-a-Service to life.

modelscope
โญ 9,224 Python ๐Ÿ‘ค modelscope Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2022-07-25
6
datajuicer/data-juicer

Data processing for and with foundation models! ๐ŸŽ ๐Ÿ‹ ๐ŸŒฝ โžก๏ธ โžก๏ธ๐Ÿธ ๐Ÿน ๐Ÿท

datajuicer
โญ 7,266 Python ๐Ÿ‘ค datajuicer Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-08-01
7
enricoros/big-AGI

The AI workspace for experts. Switch across 800+ models mid-chat, tune 40+ parameters, Beam models side by side. BYO keys, self-hostable. Built on our native-protocol engine: reliable, hyper-configurable, and fast.

enricoros
โญ 7,167 TypeScript ๐Ÿ‘ค enricoros User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-03-19
8
zai-org/CogVLM

a state-of-the-art-level open visual language model | multimodal pre-trained model

zai-org
โญ 6,712 Python ๐Ÿ‘ค zai-org Organization ๐Ÿ• Updated 2024-05-29 ๐Ÿ“… Created 2023-09-18
9
valhalla/valhalla

Open Source Routing Engine for OpenStreetMap

valhalla
โญ 6,475 C++ ๐Ÿ‘ค valhalla Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2016-01-19
10
OFA-Sys/Chinese-CLIP

Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.

OFA-Sys
โญ 6,056 Jupyter Notebook ๐Ÿ‘ค OFA-Sys Organization ๐Ÿ• Updated 2026-03-31 ๐Ÿ“… Created 2022-07-08
11
google-deepmind/gemma

Gemma open-weight LLM library, from Google DeepMind

google-deepmind
โญ 5,867 Python ๐Ÿ‘ค google-deepmind Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2024-02-20
12
lucidrains/DALLE-pytorch

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

lucidrains
โญ 5,632 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2024-02-17 ๐Ÿ“… Created 2021-01-05
13
marqo-ai/marqo

Ecommerce Search and Discovery - marqo.ai

marqo-ai
โญ 5,024 Python ๐Ÿ‘ค marqo-ai Organization ๐Ÿ• Updated 2026-09-03 ๐Ÿ“… Created 2022-08-01
14
open-compass/VLMEvalKit

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

open-compass
โญ 4,499 Python ๐Ÿ‘ค open-compass Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2023-12-01
15
zjunlp/DeepKE

[EMNLP 2022] An Open Toolkit for Knowledge Graph Extraction and Construction

zjunlp
โญ 4,486 Python ๐Ÿ‘ค zjunlp Organization ๐Ÿ• Updated 2026-07-13 ๐Ÿ“… Created 2018-08-01
16
VectorSpaceLab/OmniGen

OmniGen: Unified Image Generation. https://arxiv.org/pdf/2409.11340

VectorSpaceLab
โญ 4,340 Jupyter Notebook ๐Ÿ‘ค VectorSpaceLab Organization ๐Ÿ• Updated 2025-12-04 ๐Ÿ“… Created 2024-09-16
17
zai-org/VisualGLM-6B

Chinese and English multimodal conversational language model | Multimodal Chinese-English Bilingual Conversational Language Model

zai-org
โญ 4,181 Python ๐Ÿ‘ค zai-org Organization ๐Ÿ• Updated 2024-08-23 ๐Ÿ“… Created 2023-04-23
18
SciSharp/LLamaSharp

A C#/.NET library to run LLM (๐Ÿฆ™LLaMA/LLaVA) on your local device efficiently.

SciSharp
โญ 3,826 C# ๐Ÿ‘ค SciSharp Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-05-09
19
huangjunsen0406/py-xiaozhi

Open-source AI assistant ecosystem with MCP integrations, multimodal workflows, IoT support, and cross-platform voice interaction.

huangjunsen0406
โญ 3,560 Python ๐Ÿ‘ค huangjunsen0406 User ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2025-02-15
20
PKU-YuanGroup/Video-LLaVA

ใ€EMNLP 2024๐Ÿ”ฅใ€‘Video-LLaVA: Learning United Visual Representation by Alignment Before Projection

PKU-YuanGroup
โญ 3,425 Python ๐Ÿ‘ค PKU-YuanGroup Organization ๐Ÿ• Updated 2024-12-03 ๐Ÿ“… Created 2023-10-23
21
docarray/docarray

Represent, send, store and search multimodal data

docarray
โญ 3,123 Python ๐Ÿ‘ค docarray Organization ๐Ÿ• Updated 2026-03-27 ๐Ÿ“… Created 2021-12-14
22
JIA-Lab-research/LISA

Project Page for "LISA: Reasoning Segmentation via Large Language Model"

JIA-Lab-research
โญ 2,643 Python ๐Ÿ‘ค JIA-Lab-research Organization ๐Ÿ• Updated 2025-02-16 ๐Ÿ“… Created 2023-08-01
23
zai-org/CogVLM2

GPT4V-level open-source multi-modal model based on Llama3-8B

zai-org
โญ 2,439 Python ๐Ÿ‘ค zai-org Organization ๐Ÿ• Updated 2025-03-03 ๐Ÿ“… Created 2024-05-10
24
PKU-YuanGroup/MoE-LLaVA

ใ€TMM 2025๐Ÿ”ฅใ€‘ Mixture-of-Experts for Large Vision-Language Models

PKU-YuanGroup
โญ 2,321 Python ๐Ÿ‘ค PKU-YuanGroup Organization ๐Ÿ• Updated 2025-07-15 ๐Ÿ“… Created 2023-12-14
25
tangxyw/RecSysPapers

A collection of industry classics and cutting-edge papers in the field of recommendation/advertising/search. A collection of industry classics and cutting-edge papers in the field of recommendation/advertising/search.

tangxyw
โญ 2,216 Python ๐Ÿ‘ค tangxyw User ๐Ÿ• Updated 2026-06-11 ๐Ÿ“… Created 2022-08-16
26
OpenMotionLab/MotionGPT

[NeurIPS 2023] MotionGPT: Human Motion as a Foreign Language, a unified motion-language generation model using LLMs

OpenMotionLab
โญ 1,943 Python ๐Ÿ‘ค OpenMotionLab Organization ๐Ÿ• Updated 2025-07-01 ๐Ÿ“… Created 2023-06-20
27
Kav-K/GPTDiscord

A robust, all-in-one GPT interface for Discord. ChatGPT-style conversations, image generation, AI-moderation, custom indexes/knowledgebase, youtube summarizer, and more!

Kav-K
โญ 1,852 Python ๐Ÿ‘ค Kav-K User ๐Ÿ• Updated 2026-11-01 ๐Ÿ“… Created 2022-12-08
28
IntelLabs/fastRAG

Efficient Retrieval Augmentation and Generation Framework

IntelLabs
โญ 1,758 Python ๐Ÿ‘ค IntelLabs Organization ๐Ÿ• Updated 2025-01-09 ๐Ÿ“… Created 2023-01-23
29
bytedance/SALMONN

SALMONN family: A suite of advanced multi-modal LLMs

bytedance
โญ 1,576 ๐Ÿ‘ค bytedance Organization ๐Ÿ• Updated 2026-10-16 ๐Ÿ“… Created 2023-08-11
30
lucidrains/transfusion-pytorch

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

lucidrains
โญ 1,412 Python ๐Ÿ‘ค lucidrains User ๐Ÿ• Updated 2026-09-04 ๐Ÿ“… Created 2024-08-23
31
datascale-ai/data_engineering_book

Large Model Data Engineering: Architecture, Algorithms, and Project Practice

datascale-ai
โญ 1,398 Python ๐Ÿ‘ค datascale-ai Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2026-01-31
32
MedMNIST/MedMNIST

[pip install medmnist] 18 Standardized Datasets for 2D and 3D Biomedical Image Classification

MedMNIST
โญ 1,367 Python ๐Ÿ‘ค MedMNIST Organization ๐Ÿ• Updated 2025-01-11 ๐Ÿ“… Created 2020-10-25
33
vercel/modelfusion

The TypeScript library for building AI applications.

vercel
โญ 1,307 TypeScript ๐Ÿ‘ค vercel Organization ๐Ÿ• Updated 2024-07-19 ๐Ÿ“… Created 2023-05-25
34
Tebmer/Awesome-Knowledge-Distillation-of-LLMs

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.

Tebmer
โญ 1,282 ๐Ÿ‘ค Tebmer User ๐Ÿ• Updated 2025-03-09 ๐Ÿ“… Created 2024-02-08
35
OpenBMB/VisRAG

Parsing-free RAG supported by VLMs

OpenBMB
โญ 991 Python ๐Ÿ‘ค OpenBMB Organization ๐Ÿ• Updated 2026-11-02 ๐Ÿ“… Created 2024-10-14
36
microsoft/farmvibes-ai

FarmVibes.AI: Multi-Modal GeoSpatial ML Models for Agriculture and Sustainability

microsoft
โญ 913 Jupyter Notebook ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-11-03 ๐Ÿ“… Created 2022-09-06
37
PKU-YuanGroup/LanguageBind

ใ€ICLR 2024๐Ÿ”ฅใ€‘ Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment

PKU-YuanGroup
โญ 856 Python ๐Ÿ‘ค PKU-YuanGroup Organization ๐Ÿ• Updated 2024-03-25 ๐Ÿ“… Created 2023-10-02
38
AnswerDotAI/byaldi

Use late-interaction multi-modal models such as ColPali in just a few lines of code.

AnswerDotAI
โญ 848 Python ๐Ÿ‘ค AnswerDotAI Organization ๐Ÿ• Updated 2025-01-28 ๐Ÿ“… Created 2024-09-04
39
OpenVGLab/OmniLottie

[CVPR 2026๐Ÿ”ฅ] ๐Ÿง‘โ€๐ŸŽจ OmniLottie, an open-sourced multi-modal instructed vector animation generator that produces Lottie JSONs.

OpenVGLab
โญ 816 Python ๐Ÿ‘ค OpenVGLab Organization ๐Ÿ• Updated 2026-04-06 ๐Ÿ“… Created 2026-01-15
40
raphael-seo/Versatile-OCR-Program

Multi-modal OCR pipeline optimized for ML training (text, figure, math, tables, diagrams)

raphael-seo
โญ 677 Python ๐Ÿ‘ค raphael-seo User ๐Ÿ• Updated 2026-05-13 ๐Ÿ“… Created 2025-04-01
41
salesforce/UniControl

Unified Controllable Visual Generation Model

salesforce
โญ 665 Python ๐Ÿ‘ค salesforce Organization ๐Ÿ• Updated 2026-06-02 ๐Ÿ“… Created 2023-05-18
42
InternRobotics/Aether

[ICCV 2025 & ICCV 2025 RIWM Outstanding Paper] Aether: Geometric-Aware Unified World Modeling

InternRobotics
โญ 614 Python ๐Ÿ‘ค InternRobotics Organization ๐Ÿ• Updated 2025-10-26 ๐Ÿ“… Created 2025-03-24
43
F-R-L/forge-film

Multi-model DAG-driven parallel AI film generation โ€” parallel speedup scales with scene independence; Generate film scenes simultaneously instead of one by one; "Transform the execution graph of audiovisual generation from topological order to critical path optimal scheduling"; The only film generation engine that models scene narrative dependencies as DAG and drives parallel scheduling with CPM algorithm

F-R-L
โญ 540 Python ๐Ÿ‘ค F-R-L User ๐Ÿ• Updated 2026-03-26 ๐Ÿ“… Created 2026-03-24
44
kyegomez/RT-2

Democratization of RT-2 "RT-2: New model translates vision and language into action"

kyegomez
โญ 539 Python ๐Ÿ‘ค kyegomez User ๐Ÿ• Updated 2024-07-26 ๐Ÿ“… Created 2023-07-28
45
worldbench/awesome-vla-for-ad

๐ŸŒ Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future

worldbench
โญ 530 HTML ๐Ÿ‘ค worldbench Organization ๐Ÿ• Updated 2026-11-14 ๐Ÿ“… Created 2025-09-07
46
patrickjohncyh/fashion-clip

FashionCLIP is a CLIP-like model fine-tuned for the fashion domain.

patrickjohncyh
โญ 518 Python ๐Ÿ‘ค patrickjohncyh User ๐Ÿ• Updated 2025-01-30 ๐Ÿ“… Created 2022-05-20
47
jqtangust/Robust-R1

๐Ÿ”ฅ๐Ÿ”ฅ๐Ÿ”ฅ[AAAI 2026 Oral] Official Implementation of Robustโ€‘R1: Degradationโ€‘Aware Reasoning for Robust Visual Understanding

jqtangust
โญ 510 Python ๐Ÿ‘ค jqtangust User ๐Ÿ• Updated 2026-01-20 ๐Ÿ“… Created 2025-11-09

Frequently Asked Questions

1

What is the most popular #multi-modal repository on GitHub?

The top ranked open source #multi-modal project on GitHub is agentscope-ai/agentscope with 36,422 stars. Build and run agents you can see, understand and trust.

2

Which programming languages are most used in #multi-modal repositories?

Among the top ranked #multi-modal projects on GitHub, the most common programming languages are Python, Jupyter Notebook, TypeScript, C++.

3

How is the #multi-modal repository ranking calculated?

We track 47 active open source repositories tagged with #multi-modal (500+ stars), totaling 208,313 stars with an average of 4,432 stars per repository. Rankings update daily from the GitHub API.