Git Stars Git Stars

#inference-server Repositories

Discover 13 active open source GitHub repositories tagged with #inference-server, accumulating 53,014 total stars (averaging 4,078 stars per project). Leading this topic is jundot/omlx with 23,831 stars.

13
Total Repositories
53,014
Total Stars
4,078
Avg Stars/Repo
Programming Languages: Python Rust C++
1
jundot/omlx

LLM inference server with continuous batching & SSD caching for Apple Silicon โ€” managed from the macOS menu bar

jundot
โญ 23,831 Python ๐Ÿ‘ค jundot User ๐Ÿ• Updated 2026-11-22 ๐Ÿ“… Created 2026-02-13
2
Michael-A-Kuykendall/shimmy

โšก Pure-Rust WebGPU inference engine โ€” OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

Michael-A-Kuykendall
โญ 6,022 Rust ๐Ÿ‘ค Michael-A-Kuykendall User ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2025-08-28
3
raullenchai/Rapid-MLX

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4ร— faster than Apple's MLX (mlx-lm) on the same weights.

raullenchai
โญ 4,172 Python ๐Ÿ‘ค raullenchai User ๐Ÿ• Updated 2026-11-22 ๐Ÿ“… Created 2026-02-25
4
superlinked/sie

Open-source inference server and production cluster for all the models your agent needs.

superlinked
โญ 3,534 Python ๐Ÿ‘ค superlinked Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-11-07
5
containers/ramalama

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

containers
โญ 3,118 Python ๐Ÿ‘ค containers Organization ๐Ÿ• Updated 2026-11-14 ๐Ÿ“… Created 2024-07-24
6
roboflow/inference

Turn any computer or edge device into a command center for your computer vision projects.

roboflow
โญ 2,499 Python ๐Ÿ‘ค roboflow Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-07-31
7
gpt-omni/mini-omni2

Towards Open-source GPT-4o with Vision, Speech and Duplex Capabilities.

gpt-omni
โญ 1,896 Python ๐Ÿ‘ค gpt-omni Organization ๐Ÿ• Updated 2025-01-16 ๐Ÿ“… Created 2024-10-14
8
LlamaEdge/LlamaEdge

The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge

LlamaEdge
โญ 1,665 Rust ๐Ÿ‘ค LlamaEdge Organization ๐Ÿ• Updated 2026-02-08 ๐Ÿ“… Created 2023-10-09
9
waybarrios/vllm-mlx

High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

waybarrios
โญ 1,657 Python ๐Ÿ‘ค waybarrios User ๐Ÿ• Updated 2026-11-07 ๐Ÿ“… Created 2025-12-06
10
ollaya-dev/ollaya

Jev-compatible API server for open decision models. Ollama for decision models: pull and serve 29 model families (Laya, pplx-decider, Clef, Winnow, coba...) locally or self-hosted behind /v1/systemone. The TypeSafe SDK works unchanged: set its base URL and pick an open model. CLI, desktop app, Docker, MCP server, public bench.

ollaya-dev
โญ 1,474 Rust ๐Ÿ‘ค ollaya-dev Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-09-23
11
basetenlabs/truss

The simplest way to serve AI/ML models in production

basetenlabs
โญ 1,236 Python ๐Ÿ‘ค basetenlabs Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2022-07-06
12
aiptimizer/TurboOCR

TurboOCR, >200 img/s OmnidocBench. TensorRT FP16, PP-OCRv6, HTTP + gRPC

aiptimizer
โญ 1,162 C++ ๐Ÿ‘ค aiptimizer Organization ๐Ÿ• Updated 2026-09-27 ๐Ÿ“… Created 2026-03-20
13
pipeless-ai/pipeless

An open-source computer vision framework to build and deploy apps in minutes

pipeless-ai
โญ 794 Rust ๐Ÿ‘ค pipeless-ai Organization ๐Ÿ• Updated 2024-05-08 ๐Ÿ“… Created 2023-07-21

Frequently Asked Questions

1

What is the most popular #inference-server repository on GitHub?

The top ranked open source #inference-server project on GitHub is jundot/omlx with 23,831 stars. LLM inference server with continuous batching & SSD caching for Apple Silicon โ€” managed from the macOS menu bar

2

Which programming languages are most used in #inference-server repositories?

Among the top ranked #inference-server projects on GitHub, the most common programming languages are Python, Rust, C++.

3

How is the #inference-server repository ranking calculated?

We track 13 active open source repositories tagged with #inference-server (500+ stars), totaling 53,014 stars with an average of 4,078 stars per repository. Rankings update daily from the GitHub API.