Git Stars Git Stars

#vision-language-model Repositories

Discover 82 active open source GitHub repositories tagged with #vision-language-model, accumulating 263,383 total stars (averaging 3,211 stars per project). Leading this topic is haotian-liu/LLaVA with 24,232 stars.

82
Total Repositories
263,383
Total Stars
3,211
Avg Stars/Repo
1
haotian-liu/LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

haotian-liu
โญ 24,232 Python ๐Ÿ‘ค haotian-liu User ๐Ÿ• Updated 2024-08-12 ๐Ÿ“… Created 2023-04-17
2
datalab-to/surya

OCR, layout analysis, reading order, table recognition in 90+ languages

datalab-to
โญ 21,608 Python ๐Ÿ‘ค datalab-to Organization ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2024-01-10
3
QwenLM/Qwen3-VL

Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.

QwenLM
โญ 20,240 Jupyter Notebook ๐Ÿ‘ค QwenLM Organization ๐Ÿ• Updated 2026-01-30 ๐Ÿ“… Created 2024-08-29
4
datalab-to/chandra

OCR model that handles complex tables, forms, handwriting with full layout.

datalab-to
โญ 12,741 Python ๐Ÿ‘ค datalab-to Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2025-10-08
5
CVHub520/X-AnyLabeling

X-AnyLabeling: A lightweight, efficient, and unified cross-platform desktop application for annotating text, images, videos, point clouds, and multimodal data. It combines versatile built-in tools with state-of-the-art AI models and flexible export in multiple formats.

CVHub520
โญ 11,004 Python ๐Ÿ‘ค CVHub520 User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2023-05-23
6
m87-labs/moondream

tiny vision language model

m87-labs
โญ 10,167 Python ๐Ÿ‘ค m87-labs Organization ๐Ÿ• Updated 2026-04-20 ๐Ÿ“… Created 2023-12-29
7
OpenGVLab/InternVL

[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. An open-source multimodal dialogue model with performance close to GPT-4o.

OpenGVLab
โญ 10,160 Python ๐Ÿ‘ค OpenGVLab Organization ๐Ÿ• Updated 2025-09-22 ๐Ÿ“… Created 2023-11-22
8
studio-dots-ai/dots.ocr

Multilingual Document Layout Parsing in a Single Vision-Language Model

studio-dots-ai
โญ 9,236 Python ๐Ÿ‘ค studio-dots-ai Organization ๐Ÿ• Updated 2026-03-24 ๐Ÿ“… Created 2025-07-30
9
jingyaogong/minimind-v

๐Ÿ‘€ Train a 65M-parameter VLM from scratch in just 2h!

jingyaogong
โญ 8,868 Python ๐Ÿ‘ค jingyaogong User ๐Ÿ• Updated 2026-10-27 ๐Ÿ“… Created 2024-09-11
10
Ucas-HaoranWei/GOT-OCR2.0

Official code implementation of General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model

Ucas-HaoranWei
โญ 8,133 Python ๐Ÿ‘ค Ucas-HaoranWei User ๐Ÿ• Updated 2025-02-10 ๐Ÿ“… Created 2024-09-02
11
QwenLM/Qwen-VL

The official repo of Qwen-VL (้€šไน‰ๅƒ้—ฎ-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.

QwenLM
โญ 6,453 Python ๐Ÿ‘ค QwenLM Organization ๐Ÿ• Updated 2024-08-07 ๐Ÿ“… Created 2023-08-21
12
ByteDance-Seed/Bagel

Open-source unified multimodal model

ByteDance-Seed
โญ 6,223 Python ๐Ÿ‘ค ByteDance-Seed Organization ๐Ÿ• Updated 2026-05-04 ๐Ÿ“… Created 2025-04-17
13
Neywa-Labs/mlx-vlm

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.

Neywa-Labs
โญ 5,886 Python ๐Ÿ‘ค Neywa-Labs Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-04-16
14
volcengine/MineContext

MineContext is your proactive context-aware AI partner๏ผˆContext-Engineering+ChatGPT Pulse๏ผ‰

volcengine
โญ 5,553 Python ๐Ÿ‘ค volcengine Organization ๐Ÿ• Updated 2026-05-07 ๐Ÿ“… Created 2025-06-24
15
deepseek-ai/DeepSeek-VL2

DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding

deepseek-ai
โญ 5,297 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2025-02-26 ๐Ÿ“… Created 2024-12-13
16
PKU-Alignment/align-anything

Align Anything: Training All-modality Model with Feedback

PKU-Alignment
โญ 4,669 Python ๐Ÿ‘ค PKU-Alignment Organization ๐Ÿ• Updated 2025-11-27 ๐Ÿ“… Created 2024-07-14
17
EvolvingLMMs-Lab/lmms-eval

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

EvolvingLMMs-Lab
โญ 4,501 Python ๐Ÿ‘ค EvolvingLMMs-Lab Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-03-07
18
QwenLM/Qwen3.8

Qwen3.8 is the large language model series developed by the Qwen team at Alibaba Group.

QwenLM
โญ 4,459 ๐Ÿ‘ค QwenLM Organization ๐Ÿ• Updated 2026-08-17 ๐Ÿ“… Created 2025-09-11
19
deepseek-ai/DeepSeek-VL

DeepSeek-VL: Towards Real-World Vision-Language Understanding

deepseek-ai
โญ 4,037 Python ๐Ÿ‘ค deepseek-ai Organization ๐Ÿ• Updated 2024-04-24 ๐Ÿ“… Created 2024-03-07
20
MiniMax-AI/MiniMax-01

The official repo of MiniMax-Text-01 and MiniMax-VL-01, large-language-model & vision-language-model based on Linear Attention

MiniMax-AI
โญ 3,445 Python ๐Ÿ‘ค MiniMax-AI Organization ๐Ÿ• Updated 2025-07-07 ๐Ÿ“… Created 2025-01-14
21
off-grid-ai/OGAM

The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac โ€” GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.

off-grid-ai
โญ 3,406 TypeScript ๐Ÿ‘ค off-grid-ai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-01-29
22
dvlab-research/MGM

Official repo for "Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models"

JIA-Lab-research
โญ 3,330 Python ๐Ÿ‘ค JIA-Lab-research Organization ๐Ÿ• Updated 2024-05-04 ๐Ÿ“… Created 2024-03-26
23
jingyi0000/VLM_survey

Collection of AWESOME vision-language models for vision tasks

jingyi0000
โญ 3,132 ๐Ÿ‘ค jingyi0000 User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2023-03-30
24
InternLM/InternLM-XComposer

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

InternLM
โญ 2,925 Python ๐Ÿ‘ค InternLM Organization ๐Ÿ• Updated 2025-05-26 ๐Ÿ“… Created 2023-09-26
25
illuin-tech/colpali

The code used to train and run inference with the ColVision models, e.g. ColPali, ColQwen2, and ColSmol.

illuin-tech
โญ 2,859 Python ๐Ÿ‘ค illuin-tech Organization ๐Ÿ• Updated 2026-09-21 ๐Ÿ“… Created 2024-06-20
26
BAAI-Agents/Cradle

The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.

BAAI-Agents
โญ 2,394 Python ๐Ÿ‘ค BAAI-Agents Organization ๐Ÿ• Updated 2024-11-07 ๐Ÿ“… Created 2024-03-03
27
NVIDIA-AI-Blueprints/video-search-and-summarization

NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.

NVIDIA-AI-Blueprints
โญ 2,003 Python ๐Ÿ‘ค NVIDIA-AI-Blueprints Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-10-22
28
2U1/Qwen-VL-Series-Finetune

An open-source implementaion for fine-tuning Qwen-VL series by Alibaba Cloud.

2U1
โญ 1,983 Python ๐Ÿ‘ค 2U1 User ๐Ÿ• Updated 2026-09-09 ๐Ÿ“… Created 2024-09-10
29
showlab/ShowUI

[CVPR 2025] Open-source, End-to-end, Vision-Language-Action model for GUI Agent & Computer Use.

showlab
โญ 1,932 Python ๐Ÿ‘ค showlab Organization ๐Ÿ• Updated 2026-04-24 ๐Ÿ“… Created 2024-10-31
30
Thinklab-SJTU/Awesome-LLM4AD

A curated list of awesome LLM/VLM/VLA/World Model for Autonomous Driving(LLM4AD) resources (continually updated)

Thinklab-SJTU
โญ 1,907 ๐Ÿ‘ค Thinklab-SJTU Organization ๐Ÿ• Updated 2026-10-15 ๐Ÿ“… Created 2023-10-06
31
AlibabaResearch/AdvancedLiterateMachinery

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.

AlibabaResearch
โญ 1,832 C++ ๐Ÿ‘ค AlibabaResearch Organization ๐Ÿ• Updated 2026-03-17 ๐Ÿ“… Created 2022-09-28
โญ 1,795 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-11-02 ๐Ÿ“… Created 2026-06-15
33
ZJU4HealthCare/HealthGPT

ใ€ICML 2025 Spotlightใ€‘ Official Repo for Paper โ€˜โ€™HealthGPT : A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptationโ€˜โ€™

ZJU4HealthCare
โญ 1,673 Python ๐Ÿ‘ค ZJU4HealthCare Organization ๐Ÿ• Updated 2026-07-31 ๐Ÿ“… Created 2025-02-17
34
waybarrios/vllm-mlx

High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

waybarrios
โญ 1,657 Python ๐Ÿ‘ค waybarrios User ๐Ÿ• Updated 2026-11-07 ๐Ÿ“… Created 2025-12-06
35
ByteDance-Seed/Seed1.5-VL

Seed1.5-VL, a vision-language foundation model designed to advance general-purpose multimodal understanding and reasoning, achieving state-of-the-art performance on 38 out of 60 public benchmarks.

ByteDance-Seed
โญ 1,581 Jupyter Notebook ๐Ÿ‘ค ByteDance-Seed Organization ๐Ÿ• Updated 2025-06-14 ๐Ÿ“… Created 2025-05-11
36
ATH-MaaS/Ovis

A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.

ATH-MaaS
โญ 1,536 Python ๐Ÿ‘ค ATH-MaaS Organization ๐Ÿ• Updated 2026-07-15 ๐Ÿ“… Created 2024-06-13
37
emcf/thepipe

Get clean data from tricky documents, powered by vision-language models โšก

emcf
โญ 1,529 Python ๐Ÿ‘ค emcf User ๐Ÿ• Updated 2026-09-28 ๐Ÿ“… Created 2024-03-22
38
NVlabs/describe-anything

[ICCV 2025] Implementation for Describe Anything: Detailed Localized Image and Video Captioning

NVlabs
โญ 1,497 Python ๐Ÿ‘ค NVlabs Organization ๐Ÿ• Updated 2025-06-26 ๐Ÿ“… Created 2025-04-04
39
ARahim3/mlx-tune

Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning โ€” natively on MLX. Unsloth-compatible API.

ARahim3
โญ 1,475 Python ๐Ÿ‘ค ARahim3 User ๐Ÿ• Updated 2026-06-23 ๐Ÿ“… Created 2026-01-03
40
llm-jp/awesome-japanese-llm

Japanese LLM Summary - Overview of Japanese LLMs

llm-jp
โญ 1,446 TypeScript ๐Ÿ‘ค llm-jp Organization ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2023-07-09
41
ATH-MaaS/Awesome-Unified-Multimodal-Models

Awesome Unified Multimodal Models

ATH-MaaS
โญ 1,325 ๐Ÿ‘ค ATH-MaaS Organization ๐Ÿ• Updated 2026-03-24 ๐Ÿ“… Created 2025-05-07
42
Meituan-AutoML/MobileVLM

Strong and Open Vision Language Assistant for Mobile Devices

Meituan-AutoML
โญ 1,317 Python ๐Ÿ‘ค Meituan-AutoML Organization ๐Ÿ• Updated 2024-04-15 ๐Ÿ“… Created 2023-12-29
43
NVlabs/prismer

The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".

NVlabs
โญ 1,308 Python ๐Ÿ‘ค NVlabs Organization ๐Ÿ• Updated 2024-01-17 ๐Ÿ“… Created 2023-03-02
44
Anionex/agent-vision-toolkit

Design a better vision toolkit and skills for text-only models to "see" images, supporting multi-image understanding, image Q&A, frontend UI restoration, GUI automation, etc., with optional seamless integration with multiple mainstream agents, and direct recognition of pasted images | A vision toolkit and skill designed for text-only LLMs โ€” image Q&A, long-screenshot OCR, frontend UI restoration, and GUI automation, with optional seamless integration for Codex, Claude Code, Pi, Oh My Pi, and OpenCode

Anionex
โญ 1,267 Python ๐Ÿ‘ค Anionex User ๐Ÿ• Updated 2026-10-13 ๐Ÿ“… Created 2026-08-01
45
EvolvingLMMs-Lab/LLaVA-OneVision-2.5

Fully Open Framework for Democratized Multimodal Training

EvolvingLMMs-Lab
โญ 1,245 Python ๐Ÿ‘ค EvolvingLMMs-Lab Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2025-09-16
46
MoonshotAI/Kimi-VL

Kimi-VL: Mixture-of-Experts Vision-Language Model for Multimodal Reasoning, Long-Context Understanding, and Strong Agent Capabilities

MoonshotAI
โญ 1,203 ๐Ÿ‘ค MoonshotAI Organization ๐Ÿ• Updated 2025-07-15 ๐Ÿ“… Created 2025-04-09
47
SkalskiP/vlms-zero-to-hero

This series will take you on a journey from the fundamentals of NLP and Computer Vision to the cutting edge of Vision-Language Models.

SkalskiP
โญ 1,172 Jupyter Notebook ๐Ÿ‘ค SkalskiP User ๐Ÿ• Updated 2025-01-23 ๐Ÿ“… Created 2024-12-20
48
ZJUI-AI4H/Hulu-Med

A Transparent Generalist Model towards Holistic Medical Vision-Language Understanding

ZJUI-AI4H
โญ 1,031 Python ๐Ÿ‘ค ZJUI-AI4H User ๐Ÿ• Updated 2026-08-30 ๐Ÿ“… Created 2025-10-08
49
OpenBMB/VisRAG

Parsing-free RAG supported by VLMs

OpenBMB
โญ 991 Python ๐Ÿ‘ค OpenBMB Organization ๐Ÿ• Updated 2026-11-02 ๐Ÿ“… Created 2024-10-14
50
mbzuai-oryx/groundingLMM

[CVPR 2024 ๐Ÿ”ฅ] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.

mbzuai-oryx
โญ 969 Python ๐Ÿ‘ค mbzuai-oryx Organization ๐Ÿ• Updated 2026-09-05 ๐Ÿ“… Created 2023-11-02

Frequently Asked Questions

1

What is the most popular #vision-language-model repository on GitHub?

The top ranked open source #vision-language-model project on GitHub is haotian-liu/LLaVA with 24,232 stars. [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

2

Which programming languages are most used in #vision-language-model repositories?

Among the top ranked #vision-language-model projects on GitHub, the most common programming languages are Python, Jupyter Notebook, TypeScript, C++.

3

How is the #vision-language-model repository ranking calculated?

We track 82 active open source repositories tagged with #vision-language-model (500+ stars), totaling 263,383 stars with an average of 3,211 stars per repository. Rankings update daily from the GitHub API.