#multimodal Repositories
183 repositories total
WIT (Wikipedia-based Image Text) Dataset is a large multimodal multilingual dataset comprising 37M+ image-text sets with 11M+ unique images across 100+ languages.
[ICLR'24 spotlight] Chinese and English Multimodal Large Model Series (Chat and Paint) | Chinese-English Bilingual Multimodal Large Model Series based on CPM Foundation Model
[ECCV 2024 Best Paper Candidate & TPAMI 2025] PointLLM: Empowering Large Language Models to Understand Point Clouds
A general representation model across vision, audio, language modalities. Paper: ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities
๐ฉบ The first Chinese Medical Multimodal Model that Chest Radiographs Summarization.
An official implementation for "CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval"
Resource, examples & tutorials for multimodal AI, RAG and agents using vector search and LLMs
Open multimodal decision models, 0.8B to 27B, for text, images and video. Decide with probabilities over your own candidates and locate with bounding boxes, in place of generated text. Free API, ONNX, GGUF, vLLM and SGLang; fine-tuning code included.
โก Self-hostable YesCaptcha-compatible captcha solver built with FastAPI, Playwright, and OpenAI-compatible multimodal models.
About This repository is a curated collection of the most exciting and influential CVPR 2025 papers. ๐ฅ [Paper + Code + Demo]
Local monitoring + AI vision โ LAN-based smartphone-powered AI monitoring framework with structured event output for data acquisition and analysis.
Halo is an open-source framework built by White Circle for training large language and multimodal models
An open-weight 11B model series for long-form and real-time video understanding
A simple, unified multimodal models training engine. Lean, flexible, and built for hacking at scale.
A local plugin for image-heavy chats in Codex, Claude Code, and other AI agents: earlier images are replaced with short placeholders instead of being sent with every message, the model retrieves the original by its ID (even after compaction), and the prompt cache keeps working.
[TMLR 2025๐ฅ] A survey for the autoregressive models in vision.
Paper list about multimodal and large language models, only used to record papers I read in the daily arxiv for personal needs.
Paddle Multimodal Integration and eXploration, supporting mainstream multi-modal tasks, including end-to-end large-scale multi-modal pretrain models and diffusion model toolbox. Equipped with high performance and flexibility.
An open-source model for understanding speech, environmental sounds, and music through captioning, question answering, and reasoning
This repo contains the code for "VLM2Vec / MMEB" [ICLR 2025], "VLM2Vec-V2 / MMEB-V2" [TMLR 2026], and "MMEB-V3" [COLM 2026]
[ECCV 2024] InstructIR: High-Quality Image Restoration Following Human Instructions https://huggingface.co/spaces/marcosv/InstructIR
Second Brain is an agentic framework that acts as an operating system, using local file intelligence, workflow automation, and LLMs to complete tasks and communicate over multiple modalities and messaging platforms.
A universal Agent Skill for holographic trading cardsโcompatible with any host and multimodal model. Generate a 3D card webpage from a single sentence, with cards you can drag and rotate, flowing light, and layered depth, plus an editable Blender project.
A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, an iPhone app, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows, macOS, iOS, and Linux with full GPU capability.
AI Manus is a general-purpose AI Agent system that supports running various tools and operations in a sandbox environment.
An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scale
AI Manus is a general-purpose AI agent system that supports running various tools and operations in a sandbox environment.
AI Manus is a general-purpose AI Agent system that supports running various tools and operations in a sandbox environment.
โจโจLatest Papers and Benchmarks in Reasoning with Foundation Models
AI Manus is a general-purpose AI Agent system that supports running various tools and operations in a sandbox environment.
AI Manus is a general-purpose AI Agent system that supports running various tools and operations in a sandbox environment.
AI Manus is a general-purpose AI Agent system that supports running various tools and operations in a sandbox environment.
AI Manus is a general-purpose AI Agent system that supports running various tools and operations in a sandbox environment.
AI Manus is a general-purpose AI Agent system that supports running various tools and operations in a sandbox environment.
AI Manus is a general-purpose AI Agent system that supports running various tools and operations in a sandbox environment.
AI Manus is a general-purpose AI Agent system that supports running various tools and operations in a sandbox environment.
AI Manus is a general-purpose AI Agent system that supports running various tools and operations in a sandbox environment.
AI Manus is a general-purpose AI Agent system that supports running various tools and operations in a sandbox environment.