#flash-attention Repositories
Discover 48 active open source GitHub repositories tagged with #flash-attention, accumulating 156,291 total stars (averaging 3,256 stars per project). Leading this topic is deepseek-ai/DeepSeek-OCR with 24,180 stars.
48 repositories total
The official repo of Qwen (้ไนๅ้ฎ) chat & pretrained large language model proposed by Alibaba Cloud.
An Open-source book with Modern CUDA learning notes for beginners โ BF16/FP8/FP4, HGEMM, CuTe, Flash-Attention, etc.
Official release of InternLM series (InternLM, InternLM2, InternLM2.5, InternLM3).
Chinese LLaMA-2 & Alpaca-2 LLMs Phase II + 64K long context models (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)
This is a simple technical็งๆฎ tutorial project, mainly focusing on explaining some interesting, cutting-edge technical concepts and principles. Each article strives to be completed within 5 minutes of reading.
๐A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.๐
A fast inference library for running LLMs locally on modern consumer-class GPUs
Qwen3-ASR is an open-source series of ASR models developed by the Qwen team at Alibaba Cloud, supporting stable multilingual speech/music/song recognition, language detection and timestamp prediction.
Official codebase for "Self Forcing: Bridging Training and Inference in Autoregressive Video Diffusion" (NeurIPS 2025 Spotlight)
[ECCV 2024] Official code implementation of Vary: Scaling Up the Vision Vocabulary of Large Vision Language Models.
Provide with pre-build flash-attention 2 and 3 package wheels on Linux and Windows using GitHub Actions
SoulX-FlashTalk is the first 14B model to achieve sub-second start-up latency (0.87s) while maintaining a real-time throughput of 32 FPS on an 8xH800 node.
cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.
A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training
A flexible and efficient codebase for training visually-conditioned language models (VLMs)
A comprehensive library for implementing LLMs, including a unified training pipeline and comprehensive model evaluation.
rCM & Causal-rCM: Leading and Unified Algorithms/Infrastructures for Bidirectional/Autoregressive Video Diffusion Distillation at Scale
[ICCV 2023] StreamPETR: Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection
Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.
[CVPR 2026 Highlight] Official Implementations for Paper - HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narratives
PyTorch Implementation of "V* : Guided Visual Search as a Core Mechanism in Multimodal LLMs"
[ICML 2024 Best Paper] Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution (https://arxiv.org/abs/2310.16834)
WeDLM: The fastest diffusion language model with standard causal attention and native KV cache compatibility, delivering real speedups over vLLM-optimized baselines.
From teacher to tiles โ a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.
[ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
Frequently Asked Questions
What is the most popular #flash-attention repository on GitHub?
The top ranked open source #flash-attention project on GitHub is deepseek-ai/DeepSeek-OCR with 24,180 stars. Contexts Optical Compression
Which programming languages are most used in #flash-attention repositories?
Among the top ranked #flash-attention projects on GitHub, the most common programming languages are Python, Cuda, TeX, C++.
How is the #flash-attention repository ranking calculated?
We track 48 active open source repositories tagged with #flash-attention (500+ stars), totaling 156,291 stars with an average of 3,256 stars per repository. Rankings update daily from the GitHub API.