#vision-language-model Repositories
Discover 82 active open source GitHub repositories tagged with #vision-language-model, accumulating 263,383 total stars (averaging 3,211 stars per project). Leading this topic is haotian-liu/LLaVA with 24,232 stars.
82 repositories total
[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.
X-AnyLabeling: A lightweight, efficient, and unified cross-platform desktop application for annotating text, images, videos, point clouds, and multimodal data. It combines versatile built-in tools with state-of-the-art AI models and flexible export in multiple formats.
[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. An open-source multimodal dialogue model with performance close to GPT-4o.
Official code implementation of General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model
The official repo of Qwen-VL (้ไนๅ้ฎ-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.
MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.
MineContext is your proactive context-aware AI partner๏ผContext-Engineering+ChatGPT Pulse๏ผ
DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
Qwen3.8 is the large language model series developed by the Qwen team at Alibaba Group.
The official repo of MiniMax-Text-01 and MiniMax-VL-01, large-language-model & vision-language-model based on Linear Attention
The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac โ GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.
Official repo for "Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models"
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions
The code used to train and run inference with the ColVision models, e.g. ColPali, ColQwen2, and ColSmol.
The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.
NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.
An open-source implementaion for fine-tuning Qwen-VL series by Alibaba Cloud.
[CVPR 2025] Open-source, End-to-end, Vision-Language-Action model for GUI Agent & Computer Use.
A curated list of awesome LLM/VLM/VLA/World Model for Autonomous Driving(LLM4AD) resources (continually updated)
A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.
ใICML 2025 Spotlightใ Official Repo for Paper โโHealthGPT : A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptationโโ
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
Seed1.5-VL, a vision-language foundation model designed to advance general-purpose multimodal understanding and reasoning, achieving state-of-the-art performance on 38 out of 60 public benchmarks.
A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.
[ICCV 2025] Implementation for Describe Anything: Detailed Localized Image and Video Captioning
Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning โ natively on MLX. Unsloth-compatible API.
Design a better vision toolkit and skills for text-only models to "see" images, supporting multi-image understanding, image Q&A, frontend UI restoration, GUI automation, etc., with optional seamless integration with multiple mainstream agents, and direct recognition of pasted images | A vision toolkit and skill designed for text-only LLMs โ image Q&A, long-screenshot OCR, frontend UI restoration, and GUI automation, with optional seamless integration for Codex, Claude Code, Pi, Oh My Pi, and OpenCode
Kimi-VL: Mixture-of-Experts Vision-Language Model for Multimodal Reasoning, Long-Context Understanding, and Strong Agent Capabilities
This series will take you on a journey from the fundamentals of NLP and Computer Vision to the cutting edge of Vision-Language Models.
A Transparent Generalist Model towards Holistic Medical Vision-Language Understanding
[CVPR 2024 ๐ฅ] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.
Frequently Asked Questions
What is the most popular #vision-language-model repository on GitHub?
The top ranked open source #vision-language-model project on GitHub is haotian-liu/LLaVA with 24,232 stars. [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
Which programming languages are most used in #vision-language-model repositories?
Among the top ranked #vision-language-model projects on GitHub, the most common programming languages are Python, Jupyter Notebook, TypeScript, C++.
How is the #vision-language-model repository ranking calculated?
We track 82 active open source repositories tagged with #vision-language-model (500+ stars), totaling 263,383 stars with an average of 3,211 stars per repository. Rankings update daily from the GitHub API.