#multimodal-ai Repositories
191 repositories total
VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo
Awesome-LLM-3D: a curated list of Multi-modal Large Language Model in 3D world Resources
Official repository of 'Visual-RFT: Visual Reinforcement Fine-Tuning' & 'Visual-ARFT: Visual Agentic Reinforcement Fine-Tuningโ
Large Language-and-Vision Assistant for Biomedicine, built towards multimodal GPT-4 level capabilities.
ใLLMs Nine-Layered Demon TowerใSharing LLMs practical experiences and knowledge in fields such as Natural Language Processing (ChatGLM, Chinese-LLaMA-Alpaca, Xiaoyangtuo Vicuna, LLaMA, GPT4ALL, etc.), Information Retrieval (langchain), Language Synthesis, Language Recognition, Multimodal (Stable Diffusion, MiniGPT-4, VisualGLM-6B, Ziya-Visual, etc.).
Extract structured data and insights from documents using LLMs and decision models
The official repo of Qwen2-Audio chat & pretrained large audio language model proposed by Alibaba Cloud.
Recipes for shrinking, optimizing, customizing cutting edge vision models. ๐
NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.
Semantic search. Search local photos and videos through natural language. AI semantic search local material. Search by image, find local material, match scenes by text description, video frame search, search videos by scene description.
EVA OS โ A real-time multimodal AIOS for next-generation hardware, enabling your devices being โaliveโ and as intelligent as a real brain.
OpenGUI is an Android GUI agent framework for phone-use AI that can see, plan, and operate real mobile apps through the GUI.
TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
Ragflow-Plus is a secondary development version of Ragflow, making it more concise and practical
Collect some papers about transformer for detection and segmentation. Awesome Detection Transformer for Computer Vision (CV)
Curated visual catalog of 200+ vision-language model (VLM/MLLM) architectures: papers, diagrams, training recipes, datasets, and a release timeline for multimodal AI agents.
๐ฐ Must-read papers and blogs on Speculative Decoding โก๏ธ
Your Open Autonomous Android Agent โ A production-ready, self-planning AI assistant powered by local/remote LLMs and accessibility-driven screen automation.
OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Data
The official GitHub repo for the survey paper "A Survey on Diffusion Language Models".