#inference-server Repositories
Discover 13 active open source GitHub repositories tagged with #inference-server, accumulating 53,014 total stars (averaging 4,078 stars per project). Leading this topic is jundot/omlx with 23,831 stars.
13 repositories total
LLM inference server with continuous batching & SSD caching for Apple Silicon โ managed from the macOS menu bar
โก Pure-Rust WebGPU inference engine โ OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.
Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4ร faster than Apple's MLX (mlx-lm) on the same weights.
Open-source inference server and production cluster for all the models your agent needs.
RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.
Turn any computer or edge device into a command center for your computer vision projects.
The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
Jev-compatible API server for open decision models. Ollama for decision models: pull and serve 29 model families (Laya, pplx-decider, Clef, Winnow, coba...) locally or self-hosted behind /v1/systemone. The TypeSafe SDK works unchanged: set its base URL and pick an open model. CLI, desktop app, Docker, MCP server, public bench.
An open-source computer vision framework to build and deploy apps in minutes
Frequently Asked Questions
What is the most popular #inference-server repository on GitHub?
The top ranked open source #inference-server project on GitHub is jundot/omlx with 23,831 stars. LLM inference server with continuous batching & SSD caching for Apple Silicon โ managed from the macOS menu bar
Which programming languages are most used in #inference-server repositories?
Among the top ranked #inference-server projects on GitHub, the most common programming languages are Python, Rust, C++.
How is the #inference-server repository ranking calculated?
We track 13 active open source repositories tagged with #inference-server (500+ stars), totaling 53,014 stars with an average of 4,078 stars per repository. Rankings update daily from the GitHub API.