#multimodal Repositories
183 repositories total
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions
The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure
Images to inference with no labeling (use foundation models to train supervised models).
Easily compute clip embeddings and build a clip retrieval system with them
streamline the fine-tuning process for multimodal models: PaliGemma 2, Florence-2, and Qwen2.5-VL
Comprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.
[EMNLP-2024] Build multimodal language agents for fast prototype and production
Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
HuixiangDou: Overcoming Group Chat Scenarios with LLM-based Technical Assistance
A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.
(เท`๊ณยดเท) A Survey on Text-to-Image Generation/Synthesis.
SDK for interacting with stability.ai APIs (e.g. stable diffusion inference)
Roubao 2.0 for iPhone: an open-source iOS AI agent with owner-authorized Micro-X UI control, custom models, resumable tasks, and PDF reports. No Wi-Fi, jailbreak, computer, or extra hardware required at runtime (Mac + USB required for setup). Legacy Android version retained.
๐จ NeMo Data Designer: Generate high-quality synthetic data from scratch or from seed data.
mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding
Let Claude (or any LLM) actually watch a video โ scene-aware, deduplicated frames + transcript, from a URL or local file. Runs locally, MIT.
GenAI Processors is a lightweight Python library that enables efficient, parallel content processing.
[ICLR & NeurIPS 2025] Repository for Show-o series, One Single Transformer to Unify Multimodal Understanding and Generation.
An open-source implementaion for fine-tuning Qwen-VL series by Alibaba Cloud.
Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch
High-Performance Serving Runtime for Interactive Superintelligence Systems
A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.
Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ.
WeMM-Embedding is a family of universal multimodal embedding models by the WeChat Vision Team at Tencent, supporting multimodal understanding and retrieval.
The backend agents build with - Multimodal database, orchestration, and serving in one file
A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.
Give Claude the ability to watch and understand videos โ Claude Code plugin with frame extraction and multimodal audio analysis
Xtreme1 is an all-in-one data labeling and annotation platform for multimodal data training and supports 3D LiDAR point cloud, image, and LLM.
This repository is the official implementation of Disentangling Writer and Character Styles for Handwriting Generation (CVPR 2023)
Curated visual catalog of 200+ vision-language model (VLM/MLLM) architectures: papers, diagrams, training recipes, datasets, and a release timeline for multimodal AI agents.
Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows
Eyes for text-only DeepSeek Harness agents: built-in free vision chain (no key) + pixel-level vision tools (Q&A, grounding, crop, pixel diff, colors, OCR, SVG trace, cutout, screenshots). One-command install, no Python, image turns work like ordinary tool-calling turns.
Design a better vision toolkit and skills for text-only models to "see" images, supporting multi-image understanding, image Q&A, frontend UI restoration, GUI automation, etc., with optional seamless integration with multiple mainstream agents, and direct recognition of pasted images | A vision toolkit and skill designed for text-only LLMs โ image Q&A, long-screenshot OCR, frontend UI restoration, and GUI automation, with optional seamless integration for Codex, Claude Code, Pi, Oh My Pi, and OpenCode
Distributed training & inference for multimodal perception: shared embeddings across text, images, video, audio, code & graphs from 64 to 1024 dimensions ๐ผ๏ธ
Multi-AI MCP Server - Connect ChatGPT, Claude, Gemini & Perplexity to your coding tools without any API