#quantization Repositories
Discover 109 active open source GitHub repositories tagged with #quantization, accumulating 487,465 total stars (averaging 4,472 stars per project). Leading this topic is hiyouga/LlamaFactory with 76,171 stars.
109 repositories total
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
Chinese LLaMA & Alpaca Large Language Models + Local CPU/GPU Training Deployment (Chinese LLaMA & Alpaca LLMs)
[๐ฅupdating ...] AI Automatic Quantitative Trading Bot (Fully Local Deployment) AI-powered Quantitative Investment Research Platform. ๐ online docs: https://ufund-me.github.io/Qbot โจ :news: qbot-mini: https://github.com/Charmve/iQuant
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
Accessible large language models via k-bit quantization for PyTorch.
Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.
Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
fastllm is a high-performance large language model inference library with no backend dependencies, helping you run inference efficiently on most devices!
A fast inference library for running LLMs locally on modern consumer-class GPUs
Sharing AI Infra knowledge & code practice: PyTorch, vLLM/SGLang, slime/vime framework introduction โก๏ธ, performance acceleration ๐, large model basics ๐ง , AI software and hardware ๐ง, etc.
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.
State-of-the-art LLM compression, built for production inference with vLLM
[ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
VILA is a family of state-of-the-art vision language models (VLMs) for diverse multimodal AI tasks across the edge, data center, and cloud.
Your Cheat Sheet for AI Engineering Interview โ Questions and Answers.
[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
๐ Accelerate inference and training of ๐ค Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B), and Hy4-Flash (220B) on your own GPUs โ starting from 8 GB of VRAM, with NVIDIA and AMD support, one dashboard, and an OpenAI- and Anthropic-compatible API.
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6
micronet, a model compression and deploy lib. compression: 1ใquantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)ใLow-Bit(โค2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2ใ pruning: normalใregular and group convolutional channel pruning; 3ใ group convolution structure; 4ใbatch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/in
Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
A Python package for extending the official PyTorch that can easily obtain performance on Intel platform
An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.
Examples for using ONNX Runtime for machine learning inferencing.
Frequently Asked Questions
What is the most popular #quantization repository on GitHub?
The top ranked open source #quantization project on GitHub is hiyouga/LlamaFactory with 76,171 stars. Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
Which programming languages are most used in #quantization repositories?
Among the top ranked #quantization projects on GitHub, the most common programming languages are Python, C++, Jupyter Notebook, C.
How is the #quantization repository ranking calculated?
We track 109 active open source repositories tagged with #quantization (500+ stars), totaling 487,465 stars with an average of 4,472 stars per repository. Rankings update daily from the GitHub API.