Git Stars Git Stars

#multimodal Repositories

Discover 183 active open source GitHub repositories tagged with #multimodal, accumulating 700,629 total stars (averaging 3,828 stars per project). Leading this topic is Mintplex-Labs/anything-llm with 68,480 stars.

183
Total Repositories
700,629
Total Stars
3,828
Avg Stars/Repo
1
Mintplex-Labs/anything-llm

Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience

Mintplex-Labs
โญ 68,480 JavaScript ๐Ÿ‘ค Mintplex-Labs Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-06-04
2
bojieli/ai-agent-book

"Deep Understanding of AI Agents: Design Principles and Engineering Practice" (by Bojie Li) Open Source Main Repository: Full text, compiled PDF, and chapter-by-chapter code.

bojieli
โญ 58,976 Python ๐Ÿ‘ค bojieli User ๐Ÿ• Updated 2026-10-08 ๐Ÿ“… Created 2025-09-09
3
bytedance/UI-TARS-desktop

The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra

bytedance
โญ 39,583 TypeScript ๐Ÿ‘ค bytedance Organization ๐Ÿ• Updated 2026-10-09 ๐Ÿ“… Created 2025-01-19
4
haotian-liu/LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

haotian-liu
โญ 24,232 Python ๐Ÿ‘ค haotian-liu User ๐Ÿ• Updated 2024-08-12 ๐Ÿ“… Created 2023-04-17
5
screenpipe/screenpipe

YC (S26) | Screenpipe finds the work worth automating | Continuously record computer work locally, discover recurring workflows, and hand the result to any agent you already use.

screenpipe
โญ 22,492 Rust ๐Ÿ‘ค screenpipe Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-06-19
6
microsoft/unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

microsoft
โญ 22,247 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-09-21 ๐Ÿ“… Created 2019-07-23
7
jina-ai/serve

โ˜๏ธ Build multimodal AI applications with cloud-native stack

jina-ai
โญ 21,864 Python ๐Ÿ‘ค jina-ai Organization ๐Ÿ• Updated 2025-03-24 ๐Ÿ“… Created 2020-02-13
8
deepseek-ai/Janus

Janus-Series: Unified Multimodal Understanding and Generation Models

deepseek-ai
โญ 17,737 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-02-01 ๐Ÿ“… Created 2024-10-18
9
modelscope/ms-swift

Use PEFT or full-parameter training for CPT/SFT/DPO/GRPO on 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, LLaVA, Phi4, ...) (AAAI 2025).

modelscope
โญ 16,138 Python ๐Ÿ‘ค modelscope Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-08-01
10
datawhalechina/all-in-rag

๐Ÿ”Hands-on Large Model Application Development Part 1: Full Stack Guide to RAG Technology, Online reading address: https://datawhalechina.github.io/all-in-rag/

datawhalechina
โญ 13,275 Python ๐Ÿ‘ค datawhalechina Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2025-06-05
11
rerun-io/rerun

Visualize, query, and stream to train on multimodal robotics data.

rerun-io
โญ 11,848 Rust ๐Ÿ‘ค rerun-io Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2022-04-08
12
StarTrail-org/PixelRAG

https://arxiv.org/abs/2606.28344. The end of web parsing. The beginning of scalable pixel-native search. link: https://pixelrag.ai/

StarTrail-org
โญ 10,976 Python ๐Ÿ‘ค StarTrail-org Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2026-05-29
13
RunanywhereAI/runanywhere-sdks

Production ready toolkit to run AI locally

RunanywhereAI
โญ 10,354 C++ ๐Ÿ‘ค RunanywhereAI Organization ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2025-07-22
14
yzhao062/pyod

A Python library for anomaly detection across tabular, time series, graph, text, image, and audio data. 60+ detectors, benchmark-backed ADEngine orchestration, and an agentic workflow for AI agents.

yzhao062
โญ 10,098 Python ๐Ÿ‘ค yzhao062 User ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2017-10-03
15
apache/seatunnel

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

apache
โญ 9,821 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2017-08-05
16
X-PLUG/MobileAgent

Mobile-Agent: The Powerful GUI Agent Family

X-PLUG
โญ 9,396 Python ๐Ÿ‘ค X-PLUG Organization ๐Ÿ• Updated 2026-11-03 ๐Ÿ“… Created 2024-01-26
17
activeloopai/deeplake

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

activeloopai
โญ 9,273 C++ ๐Ÿ‘ค activeloopai Organization ๐Ÿ• Updated 2026-05-21 ๐Ÿ“… Created 2019-08-09
18
bentoml/BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

bentoml
โญ 8,944 Python ๐Ÿ‘ค bentoml Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2019-04-02
19
Neywa-Labs/mlx-audio

A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.

Neywa-Labs
โญ 8,315 Python ๐Ÿ‘ค Neywa-Labs Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-11-27
20
vllm-project/vllm-omni

A framework for efficient model inference with omni-modality models

vllm-project
โญ 7,884 Python ๐Ÿ‘ค vllm-project Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-09-11
21
genkit-ai/genkit

Open-source framework for building agentic apps in JavaScript, Go, Dart, and Python, built and used in production by Google

genkit-ai
โญ 6,609 TypeScript ๐Ÿ‘ค genkit-ai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-04-29
22
ARahim3/DigUp

Search your computer by what's in your files: photos, PDFs, documents, recordings, videos, and code. On-device on Mac, Windows, and Linux, built on Google DeepMind's EmbeddingGemma 2.

ARahim3
โญ 6,484 Swift ๐Ÿ‘ค ARahim3 User ๐Ÿ• Updated 2026-10-26 ๐Ÿ“… Created 2026-10-08
23
swyxio/ai-notes

notes for software engineers getting up to speed on new AI developments. Serves as datastore for https://latent.space writing, and product brainstorming, but has cleaned up canonical references under the /Resources folder.

swyxio
โญ 6,257 HTML ๐Ÿ‘ค swyxio User ๐Ÿ• Updated 2026-02-16 ๐Ÿ“… Created 2022-09-04
24
SkalskiP/courses

This repository is a curated collection of links to various courses and resources about Artificial Intelligence (AI)

SkalskiP
โญ 6,256 Python ๐Ÿ‘ค SkalskiP User ๐Ÿ• Updated 2024-04-22 ๐Ÿ“… Created 2023-04-02
25
om-ai-lab/VLM-R1

Solve Visual Understanding with Reinforced VLMs

om-ai-lab
โญ 6,048 Python ๐Ÿ‘ค om-ai-lab Organization ๐Ÿ• Updated 2026-07-07 ๐Ÿ“… Created 2025-02-06
26
Eventual-Inc/Daft

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

Eventual-Inc
โญ 5,858 Rust ๐Ÿ‘ค Eventual-Inc Organization ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2022-04-25
27
PySpur-Dev/pyspur

A visual playground for agentic workflows: Iterate over your agents 10x faster

PySpur-Dev
โญ 5,809 TypeScript ๐Ÿ‘ค PySpur-Dev Organization ๐Ÿ• Updated 2026-06-29 ๐Ÿ“… Created 2024-09-23
28
OpenBMB/UltraRAG

A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines

OpenBMB
โญ 5,748 Python ๐Ÿ‘ค OpenBMB Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-01-16
29
facebookresearch/mmf

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

facebookresearch
โญ 5,634 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-10-06 ๐Ÿ“… Created 2018-06-27
30
InternLM/xtuner

A Next-Generation Training Engine Built for Ultra-Large MoE Models

InternLM
โญ 5,223 Python ๐Ÿ‘ค InternLM Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-07-11
31
liustack/modlens

The first vision plugin for DeepSeek Harness, and the vision bridge for every text-only coding agent. Paste an image, get structured JSON evidence (OCR, layout, semantics).

liustack
โญ 5,050 TypeScript ๐Ÿ‘ค liustack User ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2026-02-22
32
PKU-Alignment/align-anything

Align Anything: Training All-modality Model with Feedback

PKU-Alignment
โญ 4,669 Python ๐Ÿ‘ค PKU-Alignment Organization ๐Ÿ• Updated 2025-11-27 ๐Ÿ“… Created 2024-07-14
33
kyegomez/tree-of-thoughts

The given text is already in English.

kyegomez
โญ 4,592 Python ๐Ÿ‘ค kyegomez User ๐Ÿ• Updated 2026-09-29 ๐Ÿ“… Created 2023-05-21
34
crmne/ruby_llm

The Ruby AI framework.

crmne
โญ 4,522 Ruby ๐Ÿ‘ค crmne User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2025-01-30
35
EvolvingLMMs-Lab/lmms-eval

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

EvolvingLMMs-Lab
โญ 4,501 Python ๐Ÿ‘ค EvolvingLMMs-Lab Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-03-07
36
rom1504/img2dataset

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

rom1504
โญ 4,458 Python ๐Ÿ‘ค rom1504 User ๐Ÿ• Updated 2025-10-19 ๐Ÿ“… Created 2021-08-11
37
luban-agi/Awesome-AIGC-Tutorials

Curated tutorials and resources for Large Language Models, AI Painting, and more.

luban-agi
โญ 4,419 ๐Ÿ‘ค luban-agi Organization ๐Ÿ• Updated 2024-03-31 ๐Ÿ“… Created 2023-08-22
38
OpenMOSS/MOSS-TTS

An open-source model family for long-form speech, dialogue synthesis, voice design, sound effects, and real-time streaming TTS

OpenMOSS
โญ 4,356 Python ๐Ÿ‘ค OpenMOSS Organization ๐Ÿ• Updated 2026-09-06 ๐Ÿ“… Created 2026-02-07
39
IDEA-CCNL/Fengshenbang-LM

Fengshenbang-LM (ๅฐ็ฅžๆฆœๅคงๆจกๅž‹) is an open-source large model system led by the Research Center for Cognitive Computing and Natural Language Processing of the IDEA Institute, serving as the infrastructure for Chinese AIGC and cognitive intelligence.

IDEA-CCNL
โญ 4,116 Python ๐Ÿ‘ค IDEA-CCNL Organization ๐Ÿ• Updated 2026-06-08 ๐Ÿ“… Created 2021-10-28
40
aiming-lab/SimpleMem

[ICML'26] SimpleMem: Efficient Lifelong Memory for LLM Agents โ€” Text & Multimodal

aiming-lab
โญ 3,883 Python ๐Ÿ‘ค aiming-lab Organization ๐Ÿ• Updated 2026-07-24 ๐Ÿ“… Created 2026-01-01
41
open-mmlab/mmpretrain

OpenMMLab Pre-training Toolbox and Benchmark

open-mmlab
โญ 3,807 Python ๐Ÿ‘ค open-mmlab Organization ๐Ÿ• Updated 2024-11-01 ๐Ÿ“… Created 2020-07-09
42
morphik-org/morphik-core

Open-source multimodal retrieval engine (Morphik Core)

morphik-org
โญ 3,722 Python ๐Ÿ‘ค morphik-org Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-11-11
43
atfortes/Awesome-LLM-Reasoning

From Chain-of-Thought prompting to OpenAI o1 and DeepSeek-R1 ๐Ÿ“

atfortes
โญ 3,713 ๐Ÿ‘ค atfortes User ๐Ÿ• Updated 2026-04-20 ๐Ÿ“… Created 2022-11-05
44
NExT-GPT/NExT-GPT

Code and models for ICML 2024 paper, NExT-GPT: Any-to-Any Multimodal Large Language Model

NExT-GPT
โญ 3,626 Python ๐Ÿ‘ค NExT-GPT User ๐Ÿ• Updated 2025-05-13 ๐Ÿ“… Created 2023-08-30
45
embeddings-benchmark/mteb

MTEB: State-of-the-art evaluation of embeddings across languages and modalities

embeddings-benchmark
โญ 3,519 Python ๐Ÿ‘ค embeddings-benchmark Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2022-04-05
46
TanStack/ai

๐Ÿค– Type-safe, provider-agnostic TypeScript AI SDK for streaming chat, tool calling, agents, and multimodal apps across OpenAI, Anthropic, Gemini, React, Vue, Svelte, and Solid.

TanStack
โญ 3,375 TypeScript ๐Ÿ‘ค TanStack Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-10-08
47
vortex-data/vortex

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

vortex-data
โญ 3,337 Rust ๐Ÿ‘ค vortex-data Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-02-27
48
xlang-ai/OSWorld

[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

xlang-ai
โญ 3,273 Python ๐Ÿ‘ค xlang-ai Organization ๐Ÿ• Updated 2026-11-14 ๐Ÿ“… Created 2023-10-16
49
OpenGVLab/InternGPT

InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (Online demo system supporting DragGAN, ChatGPT, ImageBind, SAM).

OpenGVLab
โญ 3,223 Python ๐Ÿ‘ค OpenGVLab Organization ๐Ÿ• Updated 2024-08-20 ๐Ÿ“… Created 2023-05-08
50
microsoft/torchscale

Foundation Architecture for (M)LLMs

microsoft
โญ 3,128 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2024-04-11 ๐Ÿ“… Created 2022-11-17

Frequently Asked Questions

1

What is the most popular #multimodal repository on GitHub?

The top ranked open source #multimodal project on GitHub is Mintplex-Labs/anything-llm with 68,480 stars. Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience

2

Which programming languages are most used in #multimodal repositories?

Among the top ranked #multimodal projects on GitHub, the most common programming languages are Python, TypeScript, Rust, C++.

3

How is the #multimodal repository ranking calculated?

We track 183 active open source repositories tagged with #multimodal (500+ stars), totaling 700,629 stars with an average of 3,828 stars per repository. Rankings update daily from the GitHub API.