#multimodal-ai Repositories
191 repositories total
[๐ ICLR 2026 Oral] NextStep-1: SOTA Autogressive Image Generation with Continuous Tokens. A research project developed by the StepFunโs Multimodal Intelligence team.
Ming - facilitating advanced multimodal understanding and generation capabilities built upon the Ling LLM.
Agentic ADK is an Agent application development framework launched by Alibaba International AI Business, based on Google-ADK and Ali-LangEngine.
Local Ollama and OpenAI-like GPT's assistance for maximum privacy and offline access
[CVPR 2024] Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers
Open-source framework for developing real-time multimodal conversational AI agents.
A curated collection of research papers exploring the utilization of LLMs for graph-related tasks.
ComfyUI Lumi Batcher is a batch processing extension plugin designed for ComfyUI, aiming to improve workflow debugging efficiency. Traditional debugging methods require adjusting parameters one by one, while this tool significantly enhances work efficiency through batch processing capabilities.
Official Implementation of "Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining"
ComfyUI nodes for Janus-Pro, a unified multimodal understanding and generation framework.
We collect papers about "large language models (LLM) for table-related tasks", e.g., using LLM for Table QA task. โTable + LLMโ related papers compilation
Chat with your documents using Vision Language Models. This repo implements an End to End RAG pipeline with both local and proprietary VLMs
(CVPR 2025 highlightโจ) Official repository of paper "LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models"
Official code implementation of Vary-toy (Small Language Model Meets with Reinforced Vision Vocabulary)
A novel human-interaction method for real-time speech extraction on headphones.
Official Repository for "Glyph: Scaling Context Windows via Visual-Text Compression"
Framework that enables fine-tuning of vision-language grounding models on custom datasets
Convert Claude's web service to an API service, supporting image recognition, file upload, thought output...
Coding a Multimodal (Vision) Language Model from scratch in PyTorch with full explanation: https://www.youtube.com/watch?v=vAmKB7iPkWw
[NeurIPS 2023] Official implementations of "Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models"