#multimodal-llm Repositories
Discover 27 active open source GitHub repositories tagged with #multimodal-llm, accumulating 63,138 total stars (averaging 2,338 stars per project). Leading this topic is anthropics/courses with 22,833 stars.
27 repositories total
Kimi-Audio, an open-source audio foundation model excelling in audio understanding, generation, and conversation
open-source multimodal large language model that can hear, talk while thinking. Featuring real-time end-to-end speech input and streaming audio output conversational capabilities.
OCRFlux is a lightweight yet powerful multimodal toolkit that significantly advances PDF-to-Markdown conversion, excelling in complex layout handling, complicated table parsing and cross-page content merging.
Open-source industrial-grade ASR models supporting Mandarin, Chinese dialects and English, achieving a new SOTA on public Mandarin ASR benchmarks, while also offering outstanding singing lyrics recognition capability.
WeMM-Embedding is a family of universal multimodal embedding models by the WeChat Vision Team at Tencent, supporting multimodal understanding and retrieval.
AgentCPM-GUI: An on-device GUI agent for operating Android apps, enhancing reasoning ability with reinforcement fine-tuning for efficient task execution.
Curated visual catalog of 200+ vision-language model (VLM/MLLM) architectures: papers, diagrams, training recipes, datasets, and a release timeline for multimodal AI agents.
VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
A Transparent Generalist Model towards Holistic Medical Vision-Language Understanding
Video-R1: Reinforcing Video Reasoning in MLLMs [๐ฅthe first paper to explore R1 for video]
Official implementation of paper "MiniGPT-5: Interleaved Vision-and-Language Generation via Generative Vokens"
A SOTA Industrial-Grade All-in-One ASR system with ASR, VAD, LID, and Punc modules. FireRedASR2 supports Chinese (Mandarin, 20+ dialects/accents), English, code-switching, and both speech and singing ASR. FireRedVAD supports speech/singing/music in 100+ langs. FireRedLID supports 100+ langs and 20+ zh dialects. FireRedPunc supports zh and en.
VideoLLM-online: Online Video Large Language Model for Streaming Video (CVPR 2024)
PyTorch Implementation of "V* : Guided Visual Search as a Core Mechanism in Multimodal LLMs"
Large language model fine-tuning, Qwen2VL, Qwen2, GLM4 instruction fine-tuning
Performed concatenated fine-tuning of the SmolVLM2 visual head and the Qwen3-0.6B model.
This project explores the integration of ChatGPT with WhatsApp, transforming the chatbot into a virtual assistant capable of performing tasks such as talking to friends, answering customer questions, and much more, with a touch of humanization in conversations.
[ICLR2026] VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling
Frequently Asked Questions
What is the most popular #multimodal-llm repository on GitHub?
The top ranked open source #multimodal-llm project on GitHub is anthropics/courses with 22,833 stars. Anthropic's educational courses
Which programming languages are most used in #multimodal-llm repositories?
Among the top ranked #multimodal-llm projects on GitHub, the most common programming languages are Python, Jupyter Notebook, Markdown, TypeScript.
How is the #multimodal-llm repository ranking calculated?
We track 27 active open source repositories tagged with #multimodal-llm (500+ stars), totaling 63,138 stars with an average of 2,338 stars per repository. Rankings update daily from the GitHub API.