#multimodal-large-language-models Repositories
Discover 43 active open source GitHub repositories tagged with #multimodal-large-language-models, accumulating 78,032 total stars (averaging 1,814 stars per project). Leading this topic is BradyFU/Awesome-Multimodal-Large-Language-Models with 18,090 stars.
43 repositories total
:sparkles::sparkles:Latest Advances on Multimodal Large Language Models
StarVector is a foundation model for SVG generation that transforms vectorization into a code generation task. Using a vision-language modeling architecture, StarVector processes both visual and textual inputs to produce high-quality SVG code with remarkable precision.
LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.
A cross-platform video structuring (video analysis) framework based on CV models & mLLM.
โจโจ[NeurIPS 2025] VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction
mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding
[ICML 2024] Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs (RPG)
Seed1.5-VL, a vision-language foundation model designed to advance general-purpose multimodal understanding and reasoning, achieving state-of-the-art performance on 38 out of 60 public benchmarks.
[ICLR2026] This is the first paper to explore how to effectively use R1-like RL for MLLMs and introduce VisionโR1, a reasoning MLLM that leverages coldโstart initialization and RL training to incentivize reasoning capability.
A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.
This repository provides valuable reference for researchers in the field of multimodality, please start your exploratory travel in RL-based Reasoning MLLMs!
Real-time voice interactive digital human, customizable appearance and voice, supporting voice cloning, with initial package delay as low as 3s.
PyTorch implementation of Audio Flamingo: Series of Advanced Audio Understanding Language Models
Reading notes about Multimodal Large Language Models, Large Language Models, and Diffusion Models
A Framework for Speech, Language, Audio, Music Processing with Large Language Model
A collection of resources on applications of multi-modal learning in medical imaging.
A paper list of some recent works about Token Compress for Vit and VLM
โจโจ[CVPR 2025] Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills
[CVPR 2024] MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
OmniVinci is an omni-modal LLM for joint understanding of vision, audio, and language.
Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train your own 8B/14B LLaVA-training-like MLLM on RTX3090/4090 24GB.
(Accepted by IJCV) Liquid: Language Models are Scalable and Unified Multi-modal Generators
โจโจWoodpecker: Hallucination Correction for Multimodal Large Language Models
A collection of multimodal reasoning papers, codes, datasets, benchmarks and resources.
NeurIPS 2024 Paper: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing
LLaVA-Mini is a unified large multimodal model (LMM) that can support the understanding of images, high-resolution images, and videos in an efficient manner.
๐ฅ๐ฅ๐ฅ A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).
๐ Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
Official code of "EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Model"
Frequently Asked Questions
What is the most popular #multimodal-large-language-models repository on GitHub?
The top ranked open source #multimodal-large-language-models project on GitHub is BradyFU/Awesome-Multimodal-Large-Language-Models with 18,090 stars. :sparkles::sparkles:Latest Advances on Multimodal Large Language Models
Which programming languages are most used in #multimodal-large-language-models repositories?
Among the top ranked #multimodal-large-language-models projects on GitHub, the most common programming languages are Python, Jupyter Notebook, HTML, C++.
How is the #multimodal-large-language-models repository ranking calculated?
We track 43 active open source repositories tagged with #multimodal-large-language-models (500+ stars), totaling 78,032 stars with an average of 1,814 stars per repository. Rankings update daily from the GitHub API.