#multimodal Repositories
Discover 183 active open source GitHub repositories tagged with #multimodal, accumulating 700,629 total stars (averaging 3,828 stars per project). Leading this topic is Mintplex-Labs/anything-llm with 68,480 stars.
183 repositories total
Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience
"Deep Understanding of AI Agents: Design Principles and Engineering Practice" (by Bojie Li) Open Source Main Repository: Full text, compiled PDF, and chapter-by-chapter code.
The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra
[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
YC (S26) | Screenpipe finds the work worth automating | Continuously record computer work locally, discover recurring workflows, and hand the result to any agent you already use.
Use PEFT or full-parameter training for CPT/SFT/DPO/GRPO on 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, LLaVA, Phi4, ...) (AAAI 2025).
๐Hands-on Large Model Application Development Part 1: Full Stack Guide to RAG Technology, Online reading address: https://datawhalechina.github.io/all-in-rag/
https://arxiv.org/abs/2606.28344. The end of web parsing. The beginning of scalable pixel-native search. link: https://pixelrag.ai/
A Python library for anomaly detection across tabular, time series, graph, text, image, and audio data. 60+ detectors, benchmark-backed ADEngine orchestration, and an agentic workflow for AI agents.
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.
Open-source framework for building agentic apps in JavaScript, Go, Dart, and Python, built and used in production by Google
Search your computer by what's in your files: photos, PDFs, documents, recordings, videos, and code. On-device on Mac, Windows, and Linux, built on Google DeepMind's EmbeddingGemma 2.
notes for software engineers getting up to speed on new AI developments. Serves as datastore for https://latent.space writing, and product brainstorming, but has cleaned up canonical references under the /Resources folder.
This repository is a curated collection of links to various courses and resources about Artificial Intelligence (AI)
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)
The first vision plugin for DeepSeek Harness, and the vision bridge for every text-only coding agent. Paste an image, get structured JSON evidence (OCR, layout, semantics).
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.
Curated tutorials and resources for Large Language Models, AI Painting, and more.
An open-source model family for long-form speech, dialogue synthesis, voice design, sound effects, and real-time streaming TTS
Fengshenbang-LM (ๅฐ็ฅๆฆๅคงๆจกๅ) is an open-source large model system led by the Research Center for Cognitive Computing and Natural Language Processing of the IDEA Institute, serving as the infrastructure for Chinese AIGC and cognitive intelligence.
[ICML'26] SimpleMem: Efficient Lifelong Memory for LLM Agents โ Text & Multimodal
Code and models for ICML 2024 paper, NExT-GPT: Any-to-Any Multimodal Large Language Model
MTEB: State-of-the-art evaluation of embeddings across languages and modalities
๐ค Type-safe, provider-agnostic TypeScript AI SDK for streaming chat, tool calling, agents, and multimodal apps across OpenAI, Anthropic, Gemini, React, Vue, Svelte, and Solid.
An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.
[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (Online demo system supporting DragGAN, ChatGPT, ImageBind, SAM).
Frequently Asked Questions
What is the most popular #multimodal repository on GitHub?
The top ranked open source #multimodal project on GitHub is Mintplex-Labs/anything-llm with 68,480 stars. Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience
Which programming languages are most used in #multimodal repositories?
Among the top ranked #multimodal projects on GitHub, the most common programming languages are Python, TypeScript, Rust, C++.
How is the #multimodal repository ranking calculated?
We track 183 active open source repositories tagged with #multimodal (500+ stars), totaling 700,629 stars with an average of 3,828 stars per repository. Rankings update daily from the GitHub API.