#multi-modal Repositories
Discover 47 active open source GitHub repositories tagged with #multi-modal, accumulating 208,313 total stars (averaging 4,432 stars per project). Leading this topic is agentscope-ai/agentscope with 36,422 stars.
47 repositories total
[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. An open-source multimodal dialogue model with performance close to GPT-4o.
Data processing for and with foundation models! ๐ ๐ ๐ฝ โก๏ธ โก๏ธ๐ธ ๐น ๐ท
The AI workspace for experts. Switch across 800+ models mid-chat, tune 40+ parameters, Beam models side by side. BYO keys, self-hostable. Built on our native-protocol engine: reliable, hyper-configurable, and fast.
Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
Chinese and English multimodal conversational language model | Multimodal Chinese-English Bilingual Conversational Language Model
A C#/.NET library to run LLM (๐ฆLLaMA/LLaVA) on your local device efficiently.
Open-source AI assistant ecosystem with MCP integrations, multimodal workflows, IoT support, and cross-platform voice interaction.
ใEMNLP 2024๐ฅใVideo-LLaVA: Learning United Visual Representation by Alignment Before Projection
A collection of industry classics and cutting-edge papers in the field of recommendation/advertising/search. A collection of industry classics and cutting-edge papers in the field of recommendation/advertising/search.
[NeurIPS 2023] MotionGPT: Human Motion as a Foreign Language, a unified motion-language generation model using LLMs
A robust, all-in-one GPT interface for Discord. ChatGPT-style conversations, image generation, AI-moderation, custom indexes/knowledgebase, youtube summarizer, and more!
Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI
Large Model Data Engineering: Architecture, Algorithms, and Project Practice
[pip install medmnist] 18 Standardized Datasets for 2D and 3D Biomedical Image Classification
This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.
FarmVibes.AI: Multi-Modal GeoSpatial ML Models for Agriculture and Sustainability
ใICLR 2024๐ฅใ Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment
Use late-interaction multi-modal models such as ColPali in just a few lines of code.
[CVPR 2026๐ฅ] ๐งโ๐จ OmniLottie, an open-sourced multi-modal instructed vector animation generator that produces Lottie JSONs.
Multi-modal OCR pipeline optimized for ML training (text, figure, math, tables, diagrams)
[ICCV 2025 & ICCV 2025 RIWM Outstanding Paper] Aether: Geometric-Aware Unified World Modeling
Multi-model DAG-driven parallel AI film generation โ parallel speedup scales with scene independence; Generate film scenes simultaneously instead of one by one; "Transform the execution graph of audiovisual generation from topological order to critical path optimal scheduling"; The only film generation engine that models scene narrative dependencies as DAG and drives parallel scheduling with CPM algorithm
Democratization of RT-2 "RT-2: New model translates vision and language into action"
๐ Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
๐ฅ๐ฅ๐ฅ[AAAI 2026 Oral] Official Implementation of RobustโR1: DegradationโAware Reasoning for Robust Visual Understanding
Frequently Asked Questions
What is the most popular #multi-modal repository on GitHub?
The top ranked open source #multi-modal project on GitHub is agentscope-ai/agentscope with 36,422 stars. Build and run agents you can see, understand and trust.
Which programming languages are most used in #multi-modal repositories?
Among the top ranked #multi-modal projects on GitHub, the most common programming languages are Python, Jupyter Notebook, TypeScript, C++.
How is the #multi-modal repository ranking calculated?
We track 47 active open source repositories tagged with #multi-modal (500+ stars), totaling 208,313 stars with an average of 4,432 stars per repository. Rankings update daily from the GitHub API.