Git Stars Git Stars

#quantization Repositories

109
Total Repositories
487,465
Total Stars
4,472
Avg Stars/Repo
101
StartLuxLabs/StartLux-Prism

Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files run on int8 and 4-bit tensor cores behind the /v1/systemone API, delivering up to 5.3ร— lower latency than vLLM in FP8 with bit-identical answers.

StartLuxLabs
โญ 560 Python ๐Ÿ‘ค StartLuxLabs Organization ๐Ÿ• Updated 2026-10-27 ๐Ÿ“… Created 2026-10-13
102
zengxiao-he/tessera

From teacher to tiles โ€” a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.

zengxiao-he
โญ 556 Python ๐Ÿ‘ค zengxiao-he User ๐Ÿ• Updated 2026-10-09 ๐Ÿ“… Created 2026-06-05
103
Indras-Mirror/Xolotl

Xolotl: run DeepSeek-V4.1-Flash, all 552B parameters, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s agent, ~41 fast, 512K context. Plus Inkling-2-Mini (Thinking Machines, own engine + MTP drafter), GLM-5.3-Flash, Qwen3.8, LFM3-Omni. MoE experts tiered across VRAM, RAM, and NVMe (Xolotl Direct). OpenAI + Anthropic API.

Indras-Mirror
โญ 549 C++ ๐Ÿ‘ค Indras-Mirror User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-06
104
cedrickchee/awesome-ml-model-compression

Awesome machine learning model compression research papers, quantization, tools, and learning material.

cedrickchee
โญ 540 ๐Ÿ‘ค cedrickchee User ๐Ÿ• Updated 2024-09-21 ๐Ÿ“… Created 2018-12-06
105
NVlabs/qerl

[ICLR 2026] QeRL enables RL for 32B LLMs on a single H100 GPU.

NVlabs
โญ 527 Python ๐Ÿ‘ค NVlabs Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2025-09-22
106
slavabarkov/tidy

Offline semantic Text-to-Image and Image-to-Image search on Android powered by quantized state-of-the-art vision-language pretrained CLIP model and ONNX Runtime inference engine

slavabarkov
โญ 525 Kotlin ๐Ÿ‘ค slavabarkov User ๐Ÿ• Updated 2024-03-28 ๐Ÿ“… Created 2023-02-24
107
huawei-csl/KVarN

KVarN is a native vLLM KV-cache quantization backend for your agents: 3โ€“5ร— more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.

huawei-csl
โญ 521 Python ๐Ÿ‘ค huawei-csl Organization ๐Ÿ• Updated 2026-06-22 ๐Ÿ“… Created 2026-05-29
108
turboderp-org/exui

Web UI for ExLlamaV2

turboderp-org
โญ 513 JavaScript ๐Ÿ‘ค turboderp-org Organization ๐Ÿ• Updated 2025-02-05 ๐Ÿ“… Created 2023-10-29
109
AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-DFlash

Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/aeon-7/aeon-vllm-ultimate:latest container, tuned for long-context draft acceptance on DGX Spark. 6 HF variants (BF16/NVFP4/MTP/MTP-XS), docker-compose, and QuickStart.

AEON-7
โญ 501 Python ๐Ÿ‘ค AEON-7 User ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2026-04-25