Git Stars Git Stars

#llm-inference-optimization Repositories

Discover 18 active open source GitHub repositories tagged with #llm-inference-optimization, accumulating 44,885 total stars (averaging 2,493 stars per project). Leading this topic is guidance-ai/guidance with 21,828 stars.

18
Total Repositories
44,885
Total Stars
2,493
Avg Stars/Repo
1
guidance-ai/guidance

A guidance language for controlling large language models.

guidance-ai
โญ 21,828 Jupyter Notebook ๐Ÿ‘ค guidance-ai Organization ๐Ÿ• Updated 2026-05-21 ๐Ÿ“… Created 2022-11-10
2
Sumanth077/ai-engineering-toolkit

A curated list of 100+ libraries and frameworks for AI engineers building with LLMs

Sumanth077
โญ 3,402 ๐Ÿ‘ค Sumanth077 User ๐Ÿ• Updated 2026-05-11 ๐Ÿ“… Created 2025-08-06
3
km1994/LLMs_interview_notes

This repository mainly records interview questions related to Large Language Model (LLM) Algorithm Engineers.

km1994
โญ 2,670 ๐Ÿ‘ค km1994 User ๐Ÿ• Updated 2026-10-11 ๐Ÿ“… Created 2023-09-16
4
maitrix-org/llm-reasoners

A library for advanced large language model reasoning

maitrix-org
โญ 2,344 Python ๐Ÿ‘ค maitrix-org Organization ๐Ÿ• Updated 2025-06-10 ๐Ÿ“… Created 2023-06-01
5
IPADS-SAI/MobiAgent

The Intelligent GUI Agent for Mobile Phones

IPADS-SAI
โญ 1,936 Python ๐Ÿ‘ค IPADS-SAI Organization ๐Ÿ• Updated 2026-07-17 ๐Ÿ“… Created 2025-08-29
โญ 1,588 Python ๐Ÿ‘ค Tencent-Hunyuan Organization ๐Ÿ• Updated 2024-12-06 ๐Ÿ“… Created 2024-10-23
7
HazyResearch/minions

Big & Small LLMs working together

HazyResearch
โญ 1,394 Python ๐Ÿ‘ค HazyResearch Organization ๐Ÿ• Updated 2026-03-12 ๐Ÿ“… Created 2025-02-03
9
VILA-Lab/Awesome-DLMs

The official GitHub repo for the survey paper "A Survey on Diffusion Language Models".

VILA-Lab
โญ 1,252 ๐Ÿ‘ค VILA-Lab Organization ๐Ÿ• Updated 2026-10-02 ๐Ÿ“… Created 2025-07-14
10
kebijuelun/Awesome-LLM-Learning

Learning Large Language Model (LLM๏ผ‰(ๅคง่ฏญ่จ€ๆจกๅž‹ๅญฆไน )

kebijuelun
โญ 984 TypeScript ๐Ÿ‘ค kebijuelun User ๐Ÿ• Updated 2026-01-05 ๐Ÿ“… Created 2023-08-06
11
sgl-project/sgl-learning-materials

Materials for learning SGLang

sgl-project
โญ 906 ๐Ÿ‘ค sgl-project Organization ๐Ÿ• Updated 2026-01-05 ๐Ÿ“… Created 2024-09-10
12
feifeibear/LLMSpeculativeSampling

Fast inference from large language models via speculative decoding

feifeibear
โญ 875 Python ๐Ÿ‘ค feifeibear User ๐Ÿ• Updated 2024-08-22 ๐Ÿ“… Created 2023-08-16
13
Nebius-Academy/LLM-Engineering-Essentials

Materials for the LLM Engineering Essentials course

Nebius-Academy
โญ 844 Jupyter Notebook ๐Ÿ‘ค Nebius-Academy Organization ๐Ÿ• Updated 2026-03-30 ๐Ÿ“… Created 2025-02-17
14
LLMServe/DistServe

Disaggregated serving system for Large Language Models (LLMs).

LLMServe
โญ 817 Jupyter Notebook ๐Ÿ‘ค LLMServe Organization ๐Ÿ• Updated 2025-04-06 ๐Ÿ“… Created 2024-01-16
15
Tencent-Hunyuan/Hunyuan-A13B

Tencent Hunyuan A13B (short as Hunyuan-A13B), an innovative and open-source LLM built on a fine-grained MoE architecture.

Tencent-Hunyuan
โญ 815 Python ๐Ÿ‘ค Tencent-Hunyuan Organization ๐Ÿ• Updated 2025-07-08 ๐Ÿ“… Created 2025-06-27
16
castorini/rank_llm

RankLLM is a Python toolkit for reproducible information retrieval research using rerankers, with a focus on listwise reranking.

castorini
โญ 715 Python ๐Ÿ‘ค castorini Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-08-07
17
thu-coai/Glyph

Official Repository for "Glyph: Scaling Context Windows via Visual-Text Compression"

thu-coai
โญ 612 Python ๐Ÿ‘ค thu-coai Organization ๐Ÿ• Updated 2025-11-04 ๐Ÿ“… Created 2025-10-20
18
hahnyuan/LLM-Viewer

Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

hahnyuan
โญ 600 Python ๐Ÿ‘ค hahnyuan User ๐Ÿ• Updated 2024-09-11 ๐Ÿ“… Created 2024-01-07

Frequently Asked Questions

1

What is the most popular #llm-inference-optimization repository on GitHub?

The top ranked open source #llm-inference-optimization project on GitHub is guidance-ai/guidance with 21,828 stars. A guidance language for controlling large language models.

2

Which programming languages are most used in #llm-inference-optimization repositories?

Among the top ranked #llm-inference-optimization projects on GitHub, the most common programming languages are Python, Jupyter Notebook, TypeScript.

3

How is the #llm-inference-optimization repository ranking calculated?

We track 18 active open source repositories tagged with #llm-inference-optimization (500+ stars), totaling 44,885 stars with an average of 2,493 stars per repository. Rankings update daily from the GitHub API.