#vlm Repositories
Discover 89 active open source GitHub repositories tagged with #vlm, accumulating 457,898 total stars (averaging 5,144 stars per project). Leading this topic is huggingface/transformers with 169,339 stars.
89 repositories total
๐ค Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra
SGLang is a high-performance serving framework for large language models and multimodal models.
Reverse-engineer any video, shot by shot: whether it was filmed, AI-generated, or codedโand how to recreate it. Breakdowns in the open VIDEO.md format. Reverse-engineer any video, shot by shot: see whether it was filmed, AI-generated, or coded, then learn how to recreate it; breakdowns are output in the open VIDEO.md format.
https://arxiv.org/abs/2606.28344. The end of web parsing. The beginning of scalable pixel-native search. link: https://pixelrag.ai/
A collection of tutorials on state-of-the-art computer vision models and techniques. Explore everything from foundational architectures like ResNet to cutting-edge models like RF-DETR, YOLO11, SAM 3, and Qwen3-VL.
Anomaly detection related books, papers, videos, and toolboxes. Last update late 2025 for LLM and VLM works!
Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code
The official repository for ERNIE 4.5 and ERNIEKit โ its industrial-grade development toolkit based on PaddlePaddle.
๐100+ Original LLM / RL Diagrams ๐, by the author of "Large Model Algorithms"! ๐ฅ (100+ LLM/RL Algorithm Maps)
StarVector is a foundation model for SVG generation that transforms vectorization into a code generation task. Using a vision-language modeling architecture, StarVector processes both visual and textual inputs to produce high-quality SVG code with remarkable precision.
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
[CVPR 2026] PromptEnhancer is a prompt-rewriting tool, refining prompts into clearer, structured versions for better image generation.
A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
An open-source document parsing model with state-of-the-art accuracy and speed, using an RSI loop during post-training.
The official repo of MiniMax-Text-01 and MiniMax-VL-01, large-language-model & vision-language-model based on Linear Attention
[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Skywork-R1V is an advanced multimodal AI model series developed by Skywork AI, specializing in vision-language reasoning.
Open-Source AI Camera Skills Platform, AI NVR & CCTV Surveillance. Local VLM video analysis with Qwen, DeepSeek, SmolVLM, LLaVA, YOLO26. LLM-powered agentic security camera agent โ watches, understands, remembers & guards your home via Telegram, Discord or Slack. Pluggable AI skills. OpenAI, Google, Anthropic or local AI. Runs on Mac Mini & AI PC.
An AI-powered file management tool that ensures privacy by organizing local texts, images. Using Llama3.2 3B and Llava v1.6 models with the Nexa SDK, it intuitively scans, restructures, and organizes files for quick, seamless access and easy retrieval.
[EMNLP-2024] Build multimodal language agents for fast prototype and production
Open source implementation and extension of Google Researchโs PaperBanana for automated academic figures, diagrams, and research visuals, expanded to new domains like slide generation.
GLM-4.6V/4.5V/4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.
LLM Agent Framework in ComfyUI includes MCP sever, Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llms with similar openai / aisuite interfaces, such as o1,ollama, gemini, grok, qwen, GLM, deepseek, kimi,doubao. Adapted to local llms, vlm, gguf such as llama-3.3 Janus-Pro, Linkage graphRAG
ChatGPT Plus/Team/Pro subscription protocol end-to-end replay toolkit ยท hCaptcha visual solver ยท Anti-fraud mechanism empirical research / End-to-end protocol replay toolkit for ChatGPT Plus/Team/Pro subscription with from-scratch hCaptcha solver and empirical anti-fraud research
Production-grade C++ edge AI engine for video analytics and on-device VLM across Sophon, Rockchip RKNN, and x86, with visual orchestration, real-time OSD, events, and reproducible benchmarks.
A reading list for large models safety, security, and privacy (including Awesome LLM Security, Safety, etc.).
Awesome-Jailbreak-on-LLMs is a collection of state-of-the-art, novel, exciting jailbreak methods on LLMs. It contains papers, codes, datasets, evaluations, and analyses.
NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.
An open-source implementaion for fine-tuning Qwen-VL series by Alibaba Cloud.
OpenGUI is an Android GUI agent framework for phone-use AI that can see, plan, and operate real mobile apps through the GUI.
Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.
High-performance, privacy-first on-device AI inference library for React Native, powered by PyTorch's ExecuTorch runtime
๐๐๐ A collection of some awesome public YOLO object detection series projects and the related object detection datasets.
Aircraft design optimization made fast through computational graph transformations (e.g., automatic differentiation). Composable analysis tools for aerodynamics, propulsion, structures, trajectory design, and much more.
Curated visual catalog of 200+ vision-language model (VLM/MLLM) architectures: papers, diagrams, training recipes, datasets, and a release timeline for multimodal AI agents.
JoyCaption is an image captioning Visual Language Model (VLM) being built from the ground up as a free, open, and uncensored model for the community to use in training Diffusion models.
InternRobotics' open platform for building generalized navigation foundation models.
Official Implementation of Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agents
The most comprehensive, continuously updated, and hottest collection of 100+ persona distillation skills on the web | Multi-agent systems | All categories: celebrities/directors/online legends/ancient texts/2D anime/workplace/emotions
Frequently Asked Questions
What is the most popular #vlm repository on GitHub?
The top ranked open source #vlm project on GitHub is huggingface/transformers with 169,339 stars. ๐ค Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Which programming languages are most used in #vlm repositories?
Among the top ranked #vlm projects on GitHub, the most common programming languages are Python, Jupyter Notebook, TypeScript, Go.
How is the #vlm repository ranking calculated?
We track 89 active open source repositories tagged with #vlm (500+ stars), totaling 457,898 stars with an average of 5,144 stars per repository. Rankings update daily from the GitHub API.