Git Stars Git Stars

#quantization Repositories

109
Total Repositories
487,465
Total Stars
4,472
Avg Stars/Repo
51
open-mmlab/mmrazor

OpenMMLab Model Compression Toolbox and Benchmark.

open-mmlab
โญ 1,653 Python ๐Ÿ‘ค open-mmlab Organization ๐Ÿ• Updated 2024-06-11 ๐Ÿ“… Created 2021-12-22
52
PaddlePaddle/PaddleSlim

PaddleSlim is an open-source library for deep model compression and architecture search.

PaddlePaddle
โญ 1,613 Python ๐Ÿ‘ค PaddlePaddle Organization ๐Ÿ• Updated 2026-01-04 ๐Ÿ“… Created 2019-12-16
53
pytorch/FBGEMM

FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/

pytorch
โญ 1,604 C++ ๐Ÿ‘ค pytorch Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-09-24
54
Xilinx/brevitas

Brevitas: neural network quantization in PyTorch

Xilinx
โญ 1,601 Python ๐Ÿ‘ค Xilinx Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-07-10
55
tensorflow/model-optimization

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

tensorflow
โญ 1,584 Python ๐Ÿ‘ค tensorflow Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2018-10-31
56
RWKV/rwkv.cpp

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

RWKV
โญ 1,570 C++ ๐Ÿ‘ค RWKV Organization ๐Ÿ• Updated 2025-03-23 ๐Ÿ“… Created 2023-03-30
57
AnswerDotAI/fsdp_qlora

Training LLMs with QLoRA + FSDP

AnswerDotAI
โญ 1,534 Jupyter Notebook ๐Ÿ‘ค AnswerDotAI Organization ๐Ÿ• Updated 2024-11-09 ๐Ÿ“… Created 2024-01-14
58
synchopate/llama-modchip

Zero-GPU CPU inference for 30Bโ€“284B MoE models (Qwen3.8 176.5B cold-cache certified in 28.26/48.35 GiB Cgroup v2, DeepSeek-V4 284B, Nemotron-30B) + RouteCrack 2.0.

synchopate
โญ 1,398 Python ๐Ÿ‘ค synchopate User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-29
59
RahulSChand/gpu_poor

Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

RahulSChand
โญ 1,383 JavaScript ๐Ÿ‘ค RahulSChand User ๐Ÿ• Updated 2024-12-03 ๐Ÿ“… Created 2023-09-12
60
open-edge-platform/geti

Build, train, optimize, and run computer vision models locally, from raw images to live inference. Open source, optimized for Intelยฎ XPU (CUDA also supported).

open-edge-platform
โญ 1,379 Python ๐Ÿ‘ค open-edge-platform Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-10-26
62
huawei-noah/Efficient-Computing

Efficient computing methods developed by Huawei Noah's Ark Lab

huawei-noah
โญ 1,306 Jupyter Notebook ๐Ÿ‘ค huawei-noah Organization ๐Ÿ• Updated 2024-11-05 ๐Ÿ“… Created 2019-09-04
63
ModelCloud/GPTQModel

LLM model quantization (compression) toolkit with hardware acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU, Tenstorrent, Huawei Ascend NPU, and Intel/AMD/Apple CPUs via HF, vLLM, and SGLang.

ModelCloud
โญ 1,294 Python ๐Ÿ‘ค ModelCloud Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-06-17
64
thu-ml/SpargeAttn

[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.

thu-ml
โญ 1,269 Cuda ๐Ÿ‘ค thu-ml Organization ๐Ÿ• Updated 2026-02-25 ๐Ÿ“… Created 2025-02-25
65
openvinotoolkit/nncf

Neural Network Compression Framework for enhanced OpenVINOโ„ข inference

openvinotoolkit
โญ 1,221 Python ๐Ÿ‘ค openvinotoolkit Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2020-05-13
66
gpu-mode/awesomeMLSys

An ML Systems Onboarding list

gpu-mode
โญ 1,154 ๐Ÿ‘ค gpu-mode Organization ๐Ÿ• Updated 2026-02-19 ๐Ÿ“… Created 2024-05-06
67
HarryR/z80ai

Z80-ฮผLM is a 2-bit quantized language model small enough to run on an 8-bit Z80 processor. Train conversational models in Python, export them as CP/M .COM binaries, and chat with your vintage computer.

HarryR
โญ 1,141 Python ๐Ÿ‘ค HarryR User ๐Ÿ• Updated 2026-04-29 ๐Ÿ“… Created 2025-12-21
68
Xilinx/finn

Dataflow compiler for QNN inference on FPGAs

Xilinx
โญ 1,093 Python ๐Ÿ‘ค Xilinx Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-06-29
69
hyperionaidev/smolmaxxing

Maximizing the ability to run different LLMs on the smallest devices possible

hyperionaidev
โญ 1,079 C ๐Ÿ‘ค hyperionaidev Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2026-11-04
70
huggingface/optimum-quanto

A pytorch quantization backend for optimum

huggingface
โญ 1,055 Python ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2023-09-19
71
darrencxl0301/StageRAG

A blueprint for building production-ready RAG systems that minimize hallucination, featuring switchable 3-step (Speed) and 4-step (Precision) pipelines.

darrencxl0301
โญ 1,036 Python ๐Ÿ‘ค darrencxl0301 User ๐Ÿ• Updated 2025-10-13 ๐Ÿ“… Created 2025-10-01
72
MAC-AutoML/MindPipe

A powerful model compression framework for LLMs and LVLMs, adapted for NVIDIA GPUs and Huawei Ascend NPUs.

MAC-AutoML
โญ 1,013 Python ๐Ÿ‘ค MAC-AutoML Organization ๐Ÿ• Updated 2026-08-11 ๐Ÿ“… Created 2026-01-22
73
PINTO0309/onnx2tf

A tool for converting ONNX files to LiteRT/TFLite/TensorFlow, PyTorch native code (nn.Module), TorchScript (.pt), state_dict (.pt), Exported Program (.pt2), and Dynamo ONNX. It also supports direct conversion from LiteRT to PyTorch.

PINTO0309
โญ 1,004 Python ๐Ÿ‘ค PINTO0309 User ๐Ÿ• Updated 2026-09-14 ๐Ÿ“… Created 2022-09-27
74
IST-DASLab/marlin

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

IST-DASLab
โญ 973 Python ๐Ÿ‘ค IST-DASLab Organization ๐Ÿ• Updated 2024-09-04 ๐Ÿ“… Created 2024-01-17
75
OpenGVLab/OmniQuant

[ICLR2024 Spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

OpenGVLab
โญ 970 Python ๐Ÿ‘ค OpenGVLab Organization ๐Ÿ• Updated 2025-11-26 ๐Ÿ“… Created 2023-08-22
76
dropbox/hqq

Official implementation of Half-Quadratic Quantization (HQQ)

dropbox
โญ 963 Python ๐Ÿ‘ค dropbox Organization ๐Ÿ• Updated 2026-02-26 ๐Ÿ“… Created 2023-11-07
77
ImageOptim/libimagequant

Palette quantization library that powers pngquant and other PNG optimizers

ImageOptim
โญ 940 Rust ๐Ÿ‘ค ImageOptim Organization ๐Ÿ• Updated 2026-06-21 ๐Ÿ“… Created 2016-11-03
78
mit-han-lab/TinyChatEngine

TinyChatEngine: On-Device LLM Inference Library

mit-han-lab
โญ 934 C++ ๐Ÿ‘ค mit-han-lab Organization ๐Ÿ• Updated 2024-07-04 ๐Ÿ“… Created 2023-05-24
79
facebookresearch/mobile-vision

Mobile vision models and code

facebookresearch
โญ 923 Python ๐Ÿ‘ค facebookresearch Organization ๐Ÿ• Updated 2026-10-21 ๐Ÿ“… Created 2019-01-09
80
mit-han-lab/tinyengine

[NeurIPS 2020] MCUNet: Tiny Deep Learning on IoT Devices; [NeurIPS 2021] MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning; [NeurIPS 2022] MCUNetV3: On-Device Training Under 256KB Memory

mit-han-lab
โญ 916 C ๐Ÿ‘ค mit-han-lab Organization ๐Ÿ• Updated 2024-11-27 ๐Ÿ“… Created 2022-08-24
81
amd/RyzenAI-SW

AMD Ryzenโ„ข AI Software includes the tools and runtime libraries for optimizing and deploying AI inference on AMD Ryzenโ„ข AI powered PCs.

amd
โญ 902 Python ๐Ÿ‘ค amd Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2023-05-12
82
whitecircle/halo

Halo is an open-source framework built by White Circle for training large language and multimodal models

whitecircle
โญ 868 Python ๐Ÿ‘ค whitecircle Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-08-26
83
IST-DASLab/sparsegpt

Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

IST-DASLab
โญ 859 Python ๐Ÿ‘ค IST-DASLab Organization ๐Ÿ• Updated 2024-08-20 ๐Ÿ“… Created 2023-03-20
84
mit-han-lab/omniserve

[MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention

mit-han-lab
โญ 840 C++ ๐Ÿ‘ค mit-han-lab Organization ๐Ÿ• Updated 2025-03-06 ๐Ÿ“… Created 2024-05-06
85
warpfront/hipfire

RDNA-native LLM inference engine in Rust.

warpfront
โญ 826 Rust ๐Ÿ‘ค warpfront Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-03-21
86
Zhen-Dong/Awesome-Quantization-Papers

List of papers related to neural network quantization in recent AI conferences and journals.

Zhen-Dong
โญ 825 ๐Ÿ‘ค Zhen-Dong User ๐Ÿ• Updated 2025-03-27 ๐Ÿ“… Created 2022-01-01
87
datawhalechina/llm-algo-leetcode

Hands-on LLM algorithms and systems with PyTorch, Triton & CUDA. โ€œLLM Algorithms and Systems Tutorialโ€: model architectures and training, post-training, inference acceleration, performance and operator optimization, with companion Notebook exercises, answer explanations, tests, and experiments.

datawhalechina
โญ 811 Jupyter Notebook ๐Ÿ‘ค datawhalechina Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2026-04-01
88
huggingface/ratchet

A cross-platform browser ML framework.

huggingface
โญ 771 Rust ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2026-09-24 ๐Ÿ“… Created 2024-01-11
89
ModelTC/LightCompress

[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.

ModelTC
โญ 755 Python ๐Ÿ‘ค ModelTC Organization ๐Ÿ• Updated 2026-05-14 ๐Ÿ“… Created 2024-03-06
90
prabanta-dev/janas

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

prabanta-dev
โญ 738 C ๐Ÿ‘ค prabanta-dev Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-09-20
91
hailo-ai/hailo_model_zoo

The Hailo Model Zoo includes pre-trained models and a full building and evaluation environment

hailo-ai
โญ 725 Python ๐Ÿ‘ค hailo-ai Organization ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2021-07-13
92
SqueezeAILab/SqueezeLLM

[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

SqueezeAILab
โญ 710 Python ๐Ÿ‘ค SqueezeAILab Organization ๐Ÿ• Updated 2024-08-13 ๐Ÿ“… Created 2023-06-12
โญ 663 Python ๐Ÿ‘ค nyunAI Organization ๐Ÿ• Updated 2025-01-22 ๐Ÿ“… Created 2024-07-08
94
wanshuiyin/ARIS-in-AI-Offer

Bilingual (Chinese + English) ML / LLM / diffusion / agent interview cheat sheets for AI autumn recruiting โ€” generated by ARIS /interview-cheatsheet, rendered by /render-html into single-file HTML that can be read anywhere โ€” plus a CVโ†’DBLP fact-checked academic homepage generator and hand-authored long-form blogs ๐ŸŒฑ

wanshuiyin
โญ 661 Python ๐Ÿ‘ค wanshuiyin User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2026-05-19
95
huawei-csl/SINQ

Welcome to the official repository of SINQ! A novel, fast and high-quality quantization method designed to make any Large Language Model smaller while preserving accuracy [ICML 2026]

huawei-csl
โญ 638 Python ๐Ÿ‘ค huawei-csl Organization ๐Ÿ• Updated 2026-05-08 ๐Ÿ“… Created 2025-09-26
96
ServiceStack/llms

LLM Client, Server API and UI

ServiceStack
โญ 631 JavaScript ๐Ÿ‘ค ServiceStack Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-09-23
97
hahnyuan/LLM-Viewer

Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

hahnyuan
โญ 600 Python ๐Ÿ‘ค hahnyuan User ๐Ÿ• Updated 2024-09-11 ๐Ÿ“… Created 2024-01-07
98
google/qkeras

QKeras: a quantization deep learning library for Tensorflow Keras

google
โญ 583 Python ๐Ÿ‘ค google Organization ๐Ÿ• Updated 2026-02-23 ๐Ÿ“… Created 2019-08-06
99
MPolaris/onnx2tflite

Tool for onnx->keras or onnx->tflite. Hope this tool can help you.

MPolaris
โญ 577 Python ๐Ÿ‘ค MPolaris User ๐Ÿ• Updated 2026-06-21 ๐Ÿ“… Created 2022-04-11
100
csyhhu/Awesome-Deep-Neural-Network-Compression

Summary, Code for Deep Neural Network Quantization

csyhhu
โญ 562 Python ๐Ÿ‘ค csyhhu User ๐Ÿ• Updated 2026-09-01 ๐Ÿ“… Created 2019-02-28