#inference-optimization Repositories
Discover 41 active open source GitHub repositories tagged with #inference-optimization, accumulating 86,309 total stars (averaging 2,105 stars per project). Leading this topic is lllyasviel/stable-diffusion-webui-forge with 12,902 stars.
41 repositories total
Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation
[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
Qwen3.8 is the large language model series developed by the Qwen team at Alibaba Group.
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
High-efficiency floating-point neural network inference operators for mobile, server, and Web
MiMo: Unlocking the Reasoning Potential of Language Model โ From Pretraining to Posttraining
This repository contains LLM (Large language model) interview question asked in top companies like Google, Nvidia , Meta , Microsoft & fortune 500 companies.
https://wavespeed.ai/ [WIP] The all in one inference optimization solution for ComfyUI, universal, flexible, and fast.
A tutorial showing how to set up TensorFlow's Object Detection API on the Raspberry Pi
Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure
A paper list of some recent works about Token Compress for Vit and VLM
Industry-level video foundation model for unified Text-to-Video (T2V) and Image-to-Video (I2V) generation.
BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.
Omni SenseVoice: High-Speed Speech Recognition with words timestamps ๐ฃ๏ธ๐ฏ
An extension for stable-diffusion-webui to load lycoris models.
Hands-on LLM algorithms and systems with PyTorch, Triton & CUDA. โLLM Algorithms and Systems Tutorialโ: model architectures and training, post-training, inference acceleration, performance and operator optimization, with companion Notebook exercises, answer explanations, tests, and experiments.
Cline Chinese localized version, Cline is an autonomous programming assistant that runs in your IDE and can create/edit files, run commands, use the browser, and other functions with your permission.
Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware
Learn LLM Inference Engineering step by stepโfrom KV cache, PagedAttention, and continuous batching to vLLM, SGLang, and GPUs.
[ICLR 2024 Spotlight] Official implementation of ScaleCrafter for higher-resolution visual generation at inference time.
Frequently Asked Questions
What is the most popular #inference-optimization repository on GitHub?
The top ranked open source #inference-optimization project on GitHub is lllyasviel/stable-diffusion-webui-forge with 12,902 stars.
Which programming languages are most used in #inference-optimization repositories?
Among the top ranked #inference-optimization projects on GitHub, the most common programming languages are Python, Jupyter Notebook, C++, C.
How is the #inference-optimization repository ranking calculated?
We track 41 active open source repositories tagged with #inference-optimization (500+ stars), totaling 86,309 stars with an average of 2,105 stars per repository. Rankings update daily from the GitHub API.