Git Stars Git Stars

#quantization Repositories

Discover 109 active open source GitHub repositories tagged with #quantization, accumulating 487,465 total stars (averaging 4,472 stars per project). Leading this topic is hiyouga/LlamaFactory with 76,171 stars.

109
Total Repositories
487,465
Total Stars
4,472
Avg Stars/Repo
1
hiyouga/LlamaFactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

hiyouga
โญ 76,171 Python ๐Ÿ‘ค hiyouga User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-05-28
2
xai-org/grok-1

Grok open release

xai-org
โญ 50,567 Python ๐Ÿ‘ค xai-org Organization ๐Ÿ• Updated 2024-08-30 ๐Ÿ“… Created 2024-03-17
3
microsoft/BitNet

Official inference framework for 1-bit LLMs

microsoft
โญ 42,358 C++ ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-07-27 ๐Ÿ“… Created 2024-08-05
4
SYSTRAN/faster-whisper

Faster Whisper transcription with CTranslate2

SYSTRAN
โญ 26,924 Python ๐Ÿ‘ค SYSTRAN Organization ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2023-02-11
5
kvcache-ai/ktransformers

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

kvcache-ai
โญ 19,687 Python ๐Ÿ‘ค kvcache-ai Organization ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2024-07-26
6
ymcui/Chinese-LLaMA-Alpaca

Chinese LLaMA & Alpaca Large Language Models + Local CPU/GPU Training Deployment (Chinese LLaMA & Alpaca LLMs)

ymcui
โญ 18,915 Python ๐Ÿ‘ค ymcui User ๐Ÿ• Updated 2026-04-19 ๐Ÿ“… Created 2023-03-15
7
UFund-Me/Qbot

[๐Ÿ”ฅupdating ...] AI Automatic Quantitative Trading Bot (Fully Local Deployment) AI-powered Quantitative Investment Research Platform. ๐Ÿ“ƒ online docs: https://ufund-me.github.io/Qbot โœจ :news: qbot-mini: https://github.com/Charmve/iQuant

UFund-Me
โญ 18,849 Jupyter Notebook ๐Ÿ‘ค UFund-Me Organization ๐Ÿ• Updated 2026-03-11 ๐Ÿ“… Created 2022-11-23
8
RyanCodrai/turbovec

A vector index built on TurboQuant, written in Rust with Python bindings

RyanCodrai
โญ 17,505 Rust ๐Ÿ‘ค RyanCodrai User ๐Ÿ• Updated 2026-10-07 ๐Ÿ“… Created 2026-03-26
9
FareedKhan-dev/kimi-k3-in-c

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

FareedKhan-dev
โญ 12,523 C ๐Ÿ‘ค FareedKhan-dev User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2026-08-01
10
artidoro/qlora

QLoRA: Efficient Finetuning of Quantized LLMs

artidoro
โญ 10,804 Jupyter Notebook ๐Ÿ‘ค artidoro User ๐Ÿ• Updated 2024-06-10 ๐Ÿ“… Created 2023-05-11
11
bitsandbytes-foundation/bitsandbytes

Accessible large language models via k-bit quantization for PyTorch.

bitsandbytes-foundation
โญ 8,548 Python ๐Ÿ‘ค bitsandbytes-foundation Organization ๐Ÿ• Updated 2026-11-02 ๐Ÿ“… Created 2021-06-04
12
gpu-mode/lectures

Material for gpu-mode lectures

gpu-mode
โญ 7,109 Jupyter Notebook ๐Ÿ‘ค gpu-mode Organization ๐Ÿ• Updated 2026-09-09 ๐Ÿ“… Created 2024-01-20
13
meta-pytorch/gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

meta-pytorch
โญ 6,244 Python ๐Ÿ‘ค meta-pytorch Organization ๐Ÿ• Updated 2025-08-22 ๐Ÿ“… Created 2023-10-17
14
Lightning-AI/lit-llama

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

Lightning-AI
โญ 6,084 Python ๐Ÿ‘ค Lightning-AI Organization ๐Ÿ• Updated 2025-07-01 ๐Ÿ“… Created 2023-03-22
15
kornelski/pngquant

Lossy PNG compressor โ€” pngquant command based on libimagequant library

kornelski
โญ 5,773 C ๐Ÿ‘ค kornelski User ๐Ÿ• Updated 2026-10-27 ๐Ÿ“… Created 2009-09-17
16
ztxz16/fastllm

fastllm is a high-performance large language model inference library with no backend dependencies, helping you run inference efficiently on most devices!

ztxz16
โญ 5,133 C++ ๐Ÿ‘ค ztxz16 User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-05-13
17
OpenNMT/CTranslate2

Fast inference engine for Transformer models

OpenNMT
โญ 4,758 C++ ๐Ÿ‘ค OpenNMT Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2019-09-23
18
turboderp-org/exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

turboderp-org
โญ 4,653 Python ๐Ÿ‘ค turboderp-org Organization ๐Ÿ• Updated 2026-03-04 ๐Ÿ“… Created 2023-08-30
19
CalvinXKY/InfraTech

Sharing AI Infra knowledge & code practice: PyTorch, vLLM/SGLang, slime/vime framework introduction โšก๏ธ, performance acceleration ๐Ÿš€, large model basics ๐Ÿง , AI software and hardware ๐Ÿ”ง, etc.

CalvinXKY
โญ 4,411 Jupyter Notebook ๐Ÿ‘ค CalvinXKY User ๐Ÿ• Updated 2026-09-20 ๐Ÿ“… Created 2025-11-14
20
thu-ml/SageAttention

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

thu-ml
โญ 4,017 Cuda ๐Ÿ‘ค thu-ml Organization ๐Ÿ• Updated 2026-01-17 ๐Ÿ“… Created 2024-10-03
21
vllm-project/llm-compressor

State-of-the-art LLM compression, built for production inference with vLLM

vllm-project
โญ 3,981 Python ๐Ÿ‘ค vllm-project Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-06-20
22
nunchux-ai/nunchaku

[ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

nunchux-ai
โญ 3,969 Python ๐Ÿ‘ค nunchux-ai Organization ๐Ÿ• Updated 2026-09-06 ๐Ÿ“… Created 2024-11-06
23
NVlabs/VILA

VILA is a family of state-of-the-art vision language models (VLMs) for diverse multimodal AI tasks across the edge, data center, and cloud.

NVlabs
โญ 3,873 Python ๐Ÿ‘ค NVlabs Organization ๐Ÿ• Updated 2026-03-12 ๐Ÿ“… Created 2024-02-23
24
amitshekhariitbhu/ai-engineering-interview-questions

Your Cheat Sheet for AI Engineering Interview โ€“ Questions and Answers.

amitshekhariitbhu
โญ 3,707 Markdown ๐Ÿ‘ค amitshekhariitbhu User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-03-21
25
mit-han-lab/llm-awq

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

mit-han-lab
โญ 3,590 Python ๐Ÿ‘ค mit-han-lab Organization ๐Ÿ• Updated 2025-07-17 ๐Ÿ“… Created 2023-06-01
26
huggingface/optimum

๐Ÿš€ Accelerate inference and training of ๐Ÿค— Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

huggingface
โญ 3,525 Python ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2021-07-20
27
ikawrakow/ik_llama.cpp

llama.cpp fork with additional SOTA quants and improved performance

ikawrakow
โญ 3,411 C++ ๐Ÿ‘ค ikawrakow User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-06-27
28
huawei-noah/Pretrained-Language-Model

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

huawei-noah
โญ 3,152 Python ๐Ÿ‘ค huawei-noah Organization ๐Ÿ• Updated 2024-01-22 ๐Ÿ“… Created 2019-12-02
29
mw00/project-maya

Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B), and Hy4-Flash (220B) on your own GPUs โ€” starting from 8 GB of VRAM, with NVIDIA and AMD support, one dashboard, and an OpenAI- and Anthropic-compatible API.

mw00
โญ 3,140 C++ ๐Ÿ‘ค mw00 User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-06
30
pytorch/ao

PyTorch native quantization for training and inference

pytorch
โญ 3,032 Python ๐Ÿ‘ค pytorch Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-11-03
31
nunchux-ai/ComfyUI-nunchaku

ComfyUI Plugin of Nunchaku

nunchux-ai
โญ 2,934 Python ๐Ÿ‘ค nunchux-ai Organization ๐Ÿ• Updated 2026-09-06 ๐Ÿ“… Created 2025-03-12
32
black-forest-labs/flux2

Official inference repo for FLUX.2 models

black-forest-labs
โญ 2,886 Python ๐Ÿ‘ค black-forest-labs Organization ๐Ÿ• Updated 2026-03-12 ๐Ÿ“… Created 2025-11-24
33
syv-ai/HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

syv-ai
โญ 2,778 Python ๐Ÿ‘ค syv-ai Organization ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2026-08-15
34
qualcomm/aimet

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

qualcomm
โญ 2,748 Python ๐Ÿ‘ค qualcomm Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-04-21
35
intel/neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

intel
โญ 2,733 Python ๐Ÿ‘ค intel Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2020-07-21
36
stochasticai/xTuring

Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6

stochasticai
โญ 2,680 Python ๐Ÿ‘ค stochasticai Organization ๐Ÿ• Updated 2026-11-05 ๐Ÿ“… Created 2023-03-19
37
adithya-s-k/AI-Engineering.academy

Mastering Applied AI, One Concept at a Time

adithya-s-k
โญ 2,388 Jupyter Notebook ๐Ÿ‘ค adithya-s-k User ๐Ÿ• Updated 2026-02-27 ๐Ÿ“… Created 2023-10-05
38
dvmazur/mixtral-offloading

Run Mixtral-8x7B models in Colab or consumer desktops

dvmazur
โญ 2,327 Python ๐Ÿ‘ค dvmazur User ๐Ÿ• Updated 2024-04-08 ๐Ÿ“… Created 2023-12-15
39
aixcoder-plugin/aiXcoder-7B

official repository of aiXcoder-7B Code Large Language Model

aixcoder-plugin
โญ 2,273 Python ๐Ÿ‘ค aixcoder-plugin Organization ๐Ÿ• Updated 2025-07-09 ๐Ÿ“… Created 2024-03-30
40
666DZY666/micronet

micronet, a model compression and deploy lib. compression: 1ใ€quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)ใ€Low-Bit(โ‰ค2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2ใ€ pruning: normalใ€regular and group convolutional channel pruning; 3ใ€ group convolution structure; 4ใ€batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/in

666DZY666
โญ 2,270 Python ๐Ÿ‘ค 666DZY666 User ๐Ÿ• Updated 2025-05-06 ๐Ÿ“… Created 2019-12-04
41
IST-DASLab/gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

IST-DASLab
โญ 2,241 Python ๐Ÿ‘ค IST-DASLab Organization ๐Ÿ• Updated 2024-03-27 ๐Ÿ“… Created 2022-10-19
42
RightNow-AI/picolm

Run a 1-billion parameter LLM on a $10 board with 256MB RAM.

RightNow-AI
โญ 2,218 C ๐Ÿ‘ค RightNow-AI Organization ๐Ÿ• Updated 2026-02-22 ๐Ÿ“… Created 2026-02-19
43
bigcode-project/starcoder2

Home of StarCoder2!

bigcode-project
โญ 2,018 Python ๐Ÿ‘ค bigcode-project Organization ๐Ÿ• Updated 2024-03-21 ๐Ÿ“… Created 2023-12-08
44
intel/intel-extension-for-pytorch

A Python package for extending the official PyTorch that can easily obtain performance on Intel platform

intel
โญ 2,012 Python ๐Ÿ‘ค intel Organization ๐Ÿ• Updated 2026-03-13 ๐Ÿ“… Created 2020-04-15
45
turboderp-org/exllamav3

An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

turboderp-org
โญ 1,853 Python ๐Ÿ‘ค turboderp-org Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2025-04-06
46
Tencent/AngelSlim

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.

Tencent
โญ 1,811 Python ๐Ÿ‘ค Tencent Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2025-07-04
47
OpenPPL/ppq

PPL Quantization Tool (PPQ) is a powerful offline neural network quantization tool.

OpenPPL
โญ 1,774 Python ๐Ÿ‘ค OpenPPL Organization ๐Ÿ• Updated 2024-03-28 ๐Ÿ“… Created 2021-12-30
48
Anemll/Anemll

Artificial Neural Engine Machine Learning Library

Anemll
โญ 1,710 Python ๐Ÿ‘ค Anemll User ๐Ÿ• Updated 2026-10-31 ๐Ÿ“… Created 2025-01-06
49
intel/auto-round

PTQ-Level Efficiency, QAT-Level Accuracy

intel
โญ 1,706 Python ๐Ÿ‘ค intel Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-01-04
50
microsoft/onnxruntime-inference-examples

Examples for using ONNX Runtime for machine learning inferencing.

microsoft
โญ 1,703 C++ ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2021-04-09

Frequently Asked Questions

1

What is the most popular #quantization repository on GitHub?

The top ranked open source #quantization project on GitHub is hiyouga/LlamaFactory with 76,171 stars. Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

2

Which programming languages are most used in #quantization repositories?

Among the top ranked #quantization projects on GitHub, the most common programming languages are Python, C++, Jupyter Notebook, C.

3

How is the #quantization repository ranking calculated?

We track 109 active open source repositories tagged with #quantization (500+ stars), totaling 487,465 stars with an average of 4,472 stars per repository. Rankings update daily from the GitHub API.