Git Stars Git Stars

#multimodal-learning Repositories

Discover 35 active open source GitHub repositories tagged with #multimodal-learning, accumulating 63,371 total stars (averaging 1,810 stars per project). Leading this topic is apple-aiml-research/ml-ferret with 8,646 stars.

35
Total Repositories
63,371
Total Stars
1,810
Avg Stars/Repo
2
dair-ai/ML-Papers-Explained

Explanation to key concepts in ML

dair-ai
โญ 8,575 ๐Ÿ‘ค dair-ai Organization ๐Ÿ• Updated 2025-06-30 ๐Ÿ“… Created 2023-01-12
3
HenryNdubuaku/maths-cs-ai-compendium

Become a cracked AI/ML researcher/engineer with this unconventional textbook covering maths, computing, and ML with intuition.

HenryNdubuaku
โญ 7,730 TypeScript ๐Ÿ‘ค HenryNdubuaku User ๐Ÿ• Updated 2026-07-18 ๐Ÿ“… Created 2026-02-03
4
pliang279/awesome-multimodal-ml

Reading list for research topics in multimodal machine learning

pliang279
โญ 6,773 ๐Ÿ‘ค pliang279 User ๐Ÿ• Updated 2024-08-20 ๐Ÿ“… Created 2019-05-27
5
mlfoundations/open_flamingo

An open-source framework for training large multimodal models.

mlfoundations
โญ 4,056 Python ๐Ÿ‘ค mlfoundations Organization ๐Ÿ• Updated 2024-08-31 ๐Ÿ“… Created 2022-10-20
6
google-research/big_vision

Official codebase used to develop Vision Transformer, SigLIP, MLP-Mixer, LiT and more.

google-research
โญ 3,455 Jupyter Notebook ๐Ÿ‘ค google-research Organization ๐Ÿ• Updated 2025-05-19 ๐Ÿ“… Created 2022-04-04
7
KaiyangZhou/CoOp

Prompt Learning for Vision-Language Models (IJCV'22, CVPR'22)

KaiyangZhou
โญ 2,147 Python ๐Ÿ‘ค KaiyangZhou User ๐Ÿ• Updated 2024-05-20 ๐Ÿ“… Created 2021-09-01
9
X-Square-Robot/wall-x

Building General-Purpose Robots Based on Embodied Foundation Model

X-Square-Robot
โญ 1,337 Python ๐Ÿ‘ค X-Square-Robot Organization ๐Ÿ• Updated 2026-09-18 ๐Ÿ“… Created 2025-09-04
10
DAMO-NLP-SG/VideoLLaMA3

Frontier Multimodal Foundation Models for Image and Video Understanding

DAMO-NLP-SG
โญ 1,174 Jupyter Notebook ๐Ÿ‘ค DAMO-NLP-SG Organization ๐Ÿ• Updated 2025-08-14 ๐Ÿ“… Created 2025-01-19
11
AILab-CVC/UniRepLKNet

[CVPR 2024 & TPAMI 2025] UniRepLKNet

AILab-CVC
โญ 1,072 Python ๐Ÿ‘ค AILab-CVC Organization ๐Ÿ• Updated 2025-08-10 ๐Ÿ“… Created 2023-11-24
12
PreferredAI/cornac

A Comparative Framework for Multimodal Recommender Systems

PreferredAI
โญ 1,065 Python ๐Ÿ‘ค PreferredAI Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2018-07-17
13
ArrowLuo/CLIP4Clip

An official implementation for "CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval"

ArrowLuo
โญ 1,017 Python ๐Ÿ‘ค ArrowLuo User ๐Ÿ• Updated 2024-04-12 ๐Ÿ“… Created 2021-04-13
14
richard-peng-xia/awesome-multimodal-in-medical-imaging

A collection of resources on applications of multi-modal learning in medical imaging.

richard-peng-xia
โญ 986 ๐Ÿ‘ค richard-peng-xia User ๐Ÿ• Updated 2026-10-26 ๐Ÿ“… Created 2022-07-13
15
DmitryRyumin/ICCV-2023-25-Papers

ICCV 2023-2025 Papers: Discover cutting-edge research from ICCV 2023-25, the leading computer vision conference. Stay updated on the latest in computer vision and deep learning, with code included. โญ support visual intelligence development!

DmitryRyumin
โญ 972 Python ๐Ÿ‘ค DmitryRyumin User ๐Ÿ• Updated 2025-11-07 ๐Ÿ“… Created 2023-09-04
16
joonson/syncnet_python

Out of time: automated lip sync in the wild

joonson
โญ 931 Python ๐Ÿ‘ค joonson User ๐Ÿ• Updated 2026-04-17 ๐Ÿ“… Created 2018-04-28
17
facebookresearch/SONAR

SONAR, a new multilingual and multimodal fixed-size sentence embedding space, with a full suite of speech and text encoders and decoders.

facebookresearch
โญ 912 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2025-10-10 ๐Ÿ“… Created 2023-08-01
18
Yuliang-Liu/MultimodalOCR

On the Hidden Mystery of OCR in Large Multimodal Models (OCRBench)

Yuliang-Liu
โญ 902 Python ๐Ÿ‘ค Yuliang-Liu User ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2023-05-12
19
mlfoundations/MINT-1T

๐Ÿƒ MINT-1T: A one trillion token multimodal interleaved dataset.

mlfoundations
โญ 828 ๐Ÿ‘ค mlfoundations Organization ๐Ÿ• Updated 2024-07-31 ๐Ÿ“… Created 2024-06-16
20
DoMusic/Hybrid-Net

Real-time audio to chords, lyrics, beat, and melody.

DoMusic
โญ 713 Python ๐Ÿ‘ค DoMusic Organization ๐Ÿ• Updated 2024-08-15 ๐Ÿ“… Created 2024-03-26
21
henghuiding/ReLA

[CVPR 2023 Highlight & IJCV 2026] GRES: Generalized Referring Expression Segmentation

henghuiding
โญ 686 Python ๐Ÿ‘ค henghuiding User ๐Ÿ• Updated 2025-11-26 ๐Ÿ“… Created 2023-03-11
22
m-bain/webvid

Large-scale text-video dataset. 10 million captioned short videos.

m-bain
โญ 670 Python ๐Ÿ‘ค m-bain User ๐Ÿ• Updated 2024-08-14 ๐Ÿ“… Created 2021-06-07
23
csuhan/OneLLM

[CVPR 2024] OneLLM: One Framework to Align All Modalities with Language

csuhan
โญ 666 Python ๐Ÿ‘ค csuhan User ๐Ÿ• Updated 2024-10-22 ๐Ÿ“… Created 2023-11-27
24
georgian-io/Multimodal-Toolkit

Multimodal model for text and tabular data with HuggingFace transformers as building block for text data

georgian-io
โญ 620 Python ๐Ÿ‘ค georgian-io Organization ๐Ÿ• Updated 2026-05-04 ๐Ÿ“… Created 2020-08-20
โญ 609 Python ๐Ÿ‘ค salesforce Organization ๐Ÿ• Updated 2026-06-02 ๐Ÿ“… Created 2022-12-06
26
pliang279/MultiBench

[NeurIPS 2021] Multiscale Benchmarks for Multimodal Representation Learning

pliang279
โญ 605 HTML ๐Ÿ‘ค pliang279 User ๐Ÿ• Updated 2024-01-27 ๐Ÿ“… Created 2021-03-05
27
MMMU-Benchmark/MMMU

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

MMMU-Benchmark
โญ 601 Python ๐Ÿ‘ค MMMU-Benchmark Organization ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2023-11-23
28
yangdongchao/UniAudio

The Open Source Code of UniAudio

yangdongchao
โญ 593 Python ๐Ÿ‘ค yangdongchao User ๐Ÿ• Updated 2024-07-22 ๐Ÿ“… Created 2023-10-01
29
sangminwoo/awesome-vision-and-language

A curated list of awesome vision and language resources (still under construction... stay tuned!)

sangminwoo
โญ 556 ๐Ÿ‘ค sangminwoo User ๐Ÿ• Updated 2024-11-04 ๐Ÿ“… Created 2019-10-25
30
llm-lab-org/Multimodal-RAG-Survey

A Survey on Multimodal Retrieval-Augmented Generation

llm-lab-org
โญ 546 ๐Ÿ‘ค llm-lab-org Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2025-02-11
31
DmitryRyumin/ICASSP-2023-24-Papers

ICASSP 2023-2024 Papers: A complete collection of influential and exciting research papers from the ICASSP 2023-24 conferences. Explore the latest advancements in acoustics, speech and signal processing. Code included. Star the repository to support the advancement of audio and signal processing!

DmitryRyumin
โญ 527 Python ๐Ÿ‘ค DmitryRyumin User ๐Ÿ• Updated 2025-05-05 ๐Ÿ“… Created 2023-08-01
32
henghuiding/MeViS

[ICCV 2023 & TPAMI 2025] MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions

henghuiding
โญ 523 Python ๐Ÿ‘ค henghuiding User ๐Ÿ• Updated 2026-01-08 ๐Ÿ“… Created 2023-08-01
33
microsoft/XPretrain

Multi-modality pre-training

microsoft
โญ 510 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2022-03-15
34
xiyuanzh/awesome-llm-time-series

tracking papers, datasets, and models of "large language model (LLM) for time series"

xiyuanzh
โญ 508 ๐Ÿ‘ค xiyuanzh User ๐Ÿ• Updated 2024-07-26 ๐Ÿ“… Created 2024-02-01
35
UCSC-VLAA/OpenVision

OpenVision (ICCV 2025), OpenVision 2 (CVPR 2026), and OpenVision 3

UCSC-VLAA
โญ 505 Python ๐Ÿ‘ค UCSC-VLAA Organization ๐Ÿ• Updated 2026-08-22 ๐Ÿ“… Created 2025-04-21

Frequently Asked Questions

1

What is the most popular #multimodal-learning repository on GitHub?

The top ranked open source #multimodal-learning project on GitHub is apple-aiml-research/ml-ferret with 8,646 stars.

2

Which programming languages are most used in #multimodal-learning repositories?

Among the top ranked #multimodal-learning projects on GitHub, the most common programming languages are Python, Jupyter Notebook, TypeScript, HTML.

3

How is the #multimodal-learning repository ranking calculated?

We track 35 active open source repositories tagged with #multimodal-learning (500+ stars), totaling 63,371 stars with an average of 1,810 stars per repository. Rankings update daily from the GitHub API.