#vlm Repositories
89 repositories total
๐ Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support
Open multimodal decision models, 0.8B to 27B, for text, images and video. Decide with probabilities over your own candidates and locate with bounding boxes, in place of generated text. Free API, ONNX, GGUF, vLLM and SGLang; fine-tuning code included.
MindSpore + ๐คHuggingface: Run any Transformers/Diffusers model on MindSpore with seamless compatibility and acceleration.
ใNew Agent Mode AddedใAndroid full-scenario GPT assistant, can be activated via volume keys for voice interaction, supports networking, photo taking, templates, attachment parsing, agent mode, etc. | GPT assistant for Android, activated via volume keys for voice interaction, supporting features such as networking, taking photos, templates, parsing PDF and Office documents, and agent mode.
UniWorld: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation
A high-performance framework for training LLMs, VLMs, diffusion, and embodied models on NVIDIA GPUs and Kunlun XPUs. Supports Pi0.5, GR00T-N1.7, FastWAM, Cosmos3, DreamZero, DeepSeek-V4, GLM-5.3, Kimi-K3, and more.
[EMNLP 2026] MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research ยท Browser-hosted Android Simulator ยท Verifiable Evaluation ยท Scalable Online RL Training
A curated list of 3D Vision papers relating to Robotics domain in the era of large models i.e. LLMs/VLMs, inspired by awesome-computer-vision, including papers, codes, and related websites
[CVPR 2026๐ฅ] ๐งโ๐จ OmniLottie, an open-sourced multi-modal instructed vector animation generator that produces Lottie JSONs.
๐๐๐A collection of some awesome public projects about Large Language Model(LLM), Vision Language Model(VLM), Vision Language Action(VLA), AI Generated Content(AIGC), the related Datasets and Applications.
An all-in-one VLA engineering platform for embodied AI โ from data to real-robot deployment.
[ECCV 2026] SparkVSR: Interactive Video Super-Resolution via Sparse Keyframe Propagation
This repo contains the code for "VLM2Vec / MMEB" [ICLR 2025], "VLM2Vec-V2 / MMEB-V2" [TMLR 2026], and "MMEB-V3" [COLM 2026]
Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.
[CVPR 2024 ๐ฅ] GeoChat, the first grounded Large Vision Language Model for Remote Sensing
AI Video Narration Agent Skill Pack: automatically cuts scenes, watches the footage to write narration, adds voice-over and mixes audio to produce the final video, and exports drafts for Jianying/CapCutโno GPU required | Agent skills for narrated video recaps & CapCut drafts
RAI is a vendor agnostic agentic framework for Physical AI robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more.
Node Creative Coding / 3D / Image Processing tool inspired by Quartz Composer
Scaling in Games: Continued Pre-Training for Embodied Agents in Diverse Virtual Worlds. Weights on Hugging Face.
ComfyUI nodes for vision-language models: Qwen3-VL, Moondream 3, Florence-2, SmolVLM2, InternVL, Gemma 3, MiniCPM-V. Plus open-vocabulary detection, SAM2/SAM3 segmentation, video temporal reasoning, GGUF via llama.cpp, and hosted LLM/VLM APIs.
Use the reMarkable2 as an interface to vision-LLMs (ChatGPT, Claude, Gemini). Ghost in the machine!
Flame is an open-source multimodal AI system designed to translate UI design mockups into high-quality React code. It leverages vision-language modeling, automated data synthesis, and structured training workflows to bridge the gap between design and front-end development.
ScreenAgent: A Computer Control Agent Driven by Visual Language Large Model (IJCAI-24)
๐ Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
[Awesome-Spatial-VLMs] This repository is the official, community-maintained resource for the survey paper: Spatial Intelligence in Vision-Language Models: A Comprehensive Survey;
A curated list of awesome papers on Embodied AI and related research/industry-driven resources.
๐๐๐ This repository lists some awesome public CUDA, cuda-python, cuBLAS, cuDNN, CUTLASS, TensorRT, TensorRT-LLM, Triton, TVM, MLIR, PTX and High Performance Computing (HPC) projects.