Git Stars Git Stars

#serving Repositories

Discover 17 active open source GitHub repositories tagged with #serving, accumulating 90,023 total stars (averaging 5,295 stars per project). Leading this topic is ray-project/ray with 44,352 stars.

17
Total Repositories
90,023
Total Stars
5,295
Avg Stars/Repo
1
ray-project/ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

ray-project
โญ 44,352 Python ๐Ÿ‘ค ray-project Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2016-10-25
2
tensorflow/serving

A flexible, high-performance serving system for machine learning models

tensorflow
โญ 6,364 C++ ๐Ÿ‘ค tensorflow Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2016-01-26
3
volcano-sh/volcano

A Cloud Native Batch System (Project under CNCF)

volcano-sh
โญ 6,106 Go ๐Ÿ‘ค volcano-sh Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2019-03-14
4
skyzh/tiny-llm

Learn LLM inference systems on Apple Silicon for systems engineers: build a tiny vLLM + Qwen

skyzh
โญ 5,801 Python ๐Ÿ‘ค skyzh User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2025-04-19
5
SeldonIO/seldon-core

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models

SeldonIO
โญ 4,781 Go ๐Ÿ‘ค SeldonIO Organization ๐Ÿ• Updated 2026-03-23 ๐Ÿ“… Created 2017-12-20
6
ahkarami/Deep-Learning-in-Production

In this repository, I will share some useful notes and references about deploying deep learning-based models in production.

ahkarami
โญ 4,380 ๐Ÿ‘ค ahkarami User ๐Ÿ• Updated 2024-11-09 ๐Ÿ“… Created 2018-05-03
7
Lightning-AI/LitServe

A minimal Python framework for building custom AI inference servers with full control over logic, batching, and scaling.

Lightning-AI
โญ 4,015 Python ๐Ÿ‘ค Lightning-AI Organization ๐Ÿ• Updated 2026-11-01 ๐Ÿ“… Created 2023-12-12
8
PaddlePaddle/FastDeploy

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

PaddlePaddle
โญ 3,722 Python ๐Ÿ‘ค PaddlePaddle Organization ๐Ÿ• Updated 2026-09-24 ๐Ÿ“… Created 2022-06-27
9
ray-project/llm-applications

A comprehensive guide to building RAG-based LLM applications for production.

ray-project
โญ 1,865 Jupyter Notebook ๐Ÿ‘ค ray-project Organization ๐Ÿ• Updated 2026-08-15 ๐Ÿ“… Created 2023-08-16
10
dingodb/dingo

A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ultra-low latency.

dingodb
โญ 1,706 Java ๐Ÿ‘ค dingodb Organization ๐Ÿ• Updated 2026-10-30 ๐Ÿ“… Created 2021-10-14
11
pixeltable/pixeltable

The backend agents build with - Multimodal database, orchestration, and serving in one file

pixeltable
โญ 1,664 Python ๐Ÿ‘ค pixeltable Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-05-10
12
Delta-ML/delta

DELTA is a deep learning based natural language and speech processing platform. LF AI & DATA Projects: https://lfaidata.foundation/projects/delta/

Delta-ML
โญ 1,602 Python ๐Ÿ‘ค Delta-ML Organization ๐Ÿ• Updated 2025-04-16 ๐Ÿ“… Created 2019-05-29
13
openvinotoolkit/model_server

A scalable inference server for models optimized with OpenVINOโ„ข

openvinotoolkit
โญ 961 C++ ๐Ÿ‘ค openvinotoolkit Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-09-26
14
PaddlePaddle/Serving

A flexible, high-performance carrier for machine learning models (ใ€Ž้ฃžๆกจใ€ Servitization Deployment Framework)

PaddlePaddle
โญ 926 C++ ๐Ÿ‘ค PaddlePaddle Organization ๐Ÿ• Updated 2026-02-20 ๐Ÿ“… Created 2019-03-31
15
tobegit3hub/simple_tensorflow_serving

Generic and easy-to-use serving service for machine learning models

tobegit3hub
โญ 758 JavaScript ๐Ÿ‘ค tobegit3hub User ๐Ÿ• Updated 2025-03-20 ๐Ÿ“… Created 2018-01-23
16
meta-soul/MetaSpore

A unified end-to-end machine intelligence platform

meta-soul
โญ 540 Python ๐Ÿ‘ค meta-soul Organization ๐Ÿ• Updated 2024-09-02 ๐Ÿ“… Created 2022-03-24
17
vectorch-ai/ScaleLLM

A high-performance inference system for large language models, designed for production environments.

vectorch-ai
โญ 502 C++ ๐Ÿ‘ค vectorch-ai Organization ๐Ÿ• Updated 2025-12-19 ๐Ÿ“… Created 2023-07-24

Frequently Asked Questions

1

What is the most popular #serving repository on GitHub?

The top ranked open source #serving project on GitHub is ray-project/ray with 44,352 stars. Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

2

Which programming languages are most used in #serving repositories?

Among the top ranked #serving projects on GitHub, the most common programming languages are Python, C++, Go, Jupyter Notebook.

3

How is the #serving repository ranking calculated?

We track 17 active open source repositories tagged with #serving (500+ stars), totaling 90,023 stars with an average of 5,295 stars per repository. Rankings update daily from the GitHub API.