#linear-attention Repositories
Discover 7 active open source GitHub repositories tagged with #linear-attention, accumulating 31,942 total stars (averaging 4,563 stars per project). Leading this topic is BlinkDL/RWKV-LM with 14,786 stars.
7 repositories total
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable SparseโLinear Attention
Frequently Asked Questions
What is the most popular #linear-attention repository on GitHub?
The top ranked open source #linear-attention project on GitHub is BlinkDL/RWKV-LM with 14,786 stars. RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
Which programming languages are most used in #linear-attention repositories?
Among the top ranked #linear-attention projects on GitHub, the most common programming languages are Python, C.
How is the #linear-attention repository ranking calculated?
We track 7 active open source repositories tagged with #linear-attention (500+ stars), totaling 31,942 stars with an average of 4,563 stars per repository. Rankings update daily from the GitHub API.