Git Stars Git Stars

#llmops Repositories

Discover 111 active open source GitHub repositories tagged with #llmops, accumulating 716,901 total stars (averaging 6,458 stars per project). Leading this topic is BerriAI/litellm with 63,649 stars.

111
Total Repositories
716,901
Total Stars
6,458
Avg Stars/Repo
1
BerriAI/litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

BerriAI
โญ 63,649 Python ๐Ÿ‘ค BerriAI Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-07-27
2
GokuMohandas/Made-With-ML

Learn how to develop, deploy and iterate on production-grade ML applications.

GokuMohandas
โญ 50,112 Jupyter Notebook ๐Ÿ‘ค GokuMohandas User ๐Ÿ• Updated 2026-03-04 ๐Ÿ“… Created 2018-11-05
3
langfuse/langfuse

๐Ÿชข Open-source AI engineering platform: AI Gateway, agent observability, evaluations, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.

langfuse
โญ 37,696 TypeScript ๐Ÿ‘ค langfuse Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-05-18
4
ComposioHQ/composio

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

ComposioHQ
โญ 31,218 TypeScript ๐Ÿ‘ค ComposioHQ Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-02-23
5
mlflow/mlflow

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

mlflow
โญ 28,920 Python ๐Ÿ‘ค mlflow Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-06-05
6
promptfoo/promptfoo

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

promptfoo
โญ 27,516 TypeScript ๐Ÿ‘ค promptfoo Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-04-28
7
liguodongiot/llm-action

This project aims to share the technical principles and practical experience related to large models (large model engineering, practical application of large models).

liguodongiot
โญ 25,297 HTML ๐Ÿ‘ค liguodongiot User ๐Ÿ• Updated 2026-10-14 ๐Ÿ“… Created 2023-05-23
8
comet-ml/opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

comet-ml
โญ 23,479 Java ๐Ÿ‘ค comet-ml Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-05-10
9
dipakkr/awesome-ai-engineering

๐Ÿš€ A curated collection of AI engineering resources, roadmaps, tools, and projects covering LLMs, RAG, AI Agents, MCP, fine-tuning, evaluations, and production AI systems.

dipakkr
โญ 22,611 ๐Ÿ‘ค dipakkr User ๐Ÿ• Updated 2026-10-21 ๐Ÿ“… Created 2018-09-17
10
jina-ai/serve

โ˜๏ธ Build multimodal AI applications with cloud-native stack

jina-ai
โญ 21,864 Python ๐Ÿ‘ค jina-ai Organization ๐Ÿ• Updated 2025-03-24 ๐Ÿ“… Created 2020-02-13
11
rocketride-org/rocketride-server

RocketRide is an open-source AI meta-harness with the only enterprise-grade runtime built on C++. Build, test, and publish durable agents that scale in production.

rocketride-org
โญ 20,003 Python ๐Ÿ‘ค rocketride-org Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-02-11
12
TransformerOptimus/SuperAGI

<โšก๏ธ> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.

TransformerOptimus
โญ 17,556 Python ๐Ÿ‘ค TransformerOptimus Organization ๐Ÿ• Updated 2025-01-22 ๐Ÿ“… Created 2023-05-13
13
vibrantlabsai/ragas

Supercharge Your LLM Application Evaluations ๐Ÿš€

vibrantlabsai
โญ 16,395 Python ๐Ÿ‘ค vibrantlabsai Organization ๐Ÿ• Updated 2026-11-14 ๐Ÿ“… Created 2023-05-08
14
raga-ai-hub/RagaAI-Catalyst

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

raga-ai-hub
โญ 16,193 Python ๐Ÿ‘ค raga-ai-hub Organization ๐Ÿ• Updated 2026-02-11 ๐Ÿ“… Created 2024-08-26
15
Portkey-AI/gateway

A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

Portkey-AI
โญ 13,424 TypeScript ๐Ÿ‘ค Portkey-AI Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-08-23
16
bentoml/OpenLLM

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

bentoml
โญ 12,610 Python ๐Ÿ‘ค bentoml Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2023-04-19
17
dataelement/bisheng

BISHENG is an open-source enterprise Agent platform for building, deploying, and managing AI applications. It combines AI agents, workflows, RAG and knowledge management, desktop agent integration, and vibecoding app hosting with unified model management, evaluation, observability, access control, and enterprise administration.

dataelement
โญ 12,391 Python ๐Ÿ‘ค dataelement Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-08-28
18
Arize-ai/phoenix

AI Observability & Evaluation

Arize-ai
โญ 12,368 Python ๐Ÿ‘ค Arize-ai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2022-11-09
19
tensorzero/tensorzero

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

tensorzero
โญ 11,456 Rust ๐Ÿ‘ค tensorzero Organization ๐Ÿ• Updated 2026-06-11 ๐Ÿ“… Created 2024-07-16
20
MakazhanAlpamys/Soup

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

MakazhanAlpamys
โญ 10,703 Python ๐Ÿ‘ค MakazhanAlpamys User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-02-20
21
Netflix/metaflow

Build, Manage and Deploy AI/ML Systems

Netflix
โญ 10,355 Python ๐Ÿ‘ค Netflix Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2019-09-17
22
0xPlaygrounds/rig

โš™๏ธ๐Ÿฆ€ Build modular and scalable LLM Applications in Rust

0xPlaygrounds
โญ 9,209 Rust ๐Ÿ‘ค 0xPlaygrounds Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-06-05
23
bentoml/BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

bentoml
โญ 8,944 Python ๐Ÿ‘ค bentoml Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2019-04-02
24
evidentlyai/evidently

Evidently is โ€‹โ€‹an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

evidentlyai
โญ 8,073 Jupyter Notebook ๐Ÿ‘ค evidentlyai Organization ๐Ÿ• Updated 2026-09-29 ๐Ÿ“… Created 2020-11-25
25
traceloop/openllmetry

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

traceloop
โญ 7,546 Python ๐Ÿ‘ค traceloop Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2023-09-02
26
katanemo/plano

Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.

katanemo
โญ 7,118 Rust ๐Ÿ‘ค katanemo Organization ๐Ÿ• Updated 2026-10-24 ๐Ÿ“… Created 2024-07-09
27
clearml/clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution

clearml
โญ 6,943 Python ๐Ÿ‘ค clearml Organization ๐Ÿ• Updated 2026-11-03 ๐Ÿ“… Created 2019-06-10
28
GoogleCloudPlatform/agent-starter-pack

Ship AI Agents to Google Cloud in minutes, not months. Production-ready templates with built-in CI/CD, evaluation, and observability.

GoogleCloudPlatform
โญ 6,575 Python ๐Ÿ‘ค GoogleCloudPlatform Organization ๐Ÿ• Updated 2026-07-21 ๐Ÿ“… Created 2025-01-31
29
Helicone/helicone

๐ŸงŠ Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 ๐Ÿ“

Helicone
โญ 6,292 TypeScript ๐Ÿ‘ค Helicone Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2023-01-31
30
tensorchord/Awesome-LLMOps

An awesome & curated list of best LLMOps tools for developers

tensorchord
โญ 5,979 Shell ๐Ÿ‘ค tensorchord Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2022-04-15
31
Giskard-AI/giskard-oss

๐Ÿข Open-Source Evaluation & Testing library for LLM Agents

Giskard-AI
โญ 5,965 Python ๐Ÿ‘ค Giskard-AI Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2022-03-06
32
coze-dev/coze-loop

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

coze-dev
โญ 5,799 Go ๐Ÿ‘ค coze-dev Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2025-06-24
33
zenml-io/zenml

ZenML ๐Ÿ™: One AI Platform from Pipelines to Agents. https://zenml.io.

zenml-io
โญ 5,636 Python ๐Ÿ‘ค zenml-io Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2020-11-19
34
PacktPublishing/LLM-Engineers-Handbook

The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps best practices

PacktPublishing
โญ 5,438 Python ๐Ÿ‘ค PacktPublishing Organization ๐Ÿ• Updated 2026-04-22 ๐Ÿ“… Created 2024-04-09
35
superduper-io/superduper

Superduper: End-to-end framework for building custom AI applications and agents.

superduper-io
โญ 5,317 Python ๐Ÿ‘ค superduper-io Organization ๐Ÿ• Updated 2025-09-01 ๐Ÿ“… Created 2022-08-30
36
landing-ai/vision-agent

This tool has been deprecated. Use Agentic Document Extraction instead.

landing-ai
โญ 5,297 Python ๐Ÿ‘ค landing-ai Organization ๐Ÿ• Updated 2026-01-29 ๐Ÿ“… Created 2024-02-13
37
tencentmusic/cube-studio

cube studio Open-Source Cloud-Native All-in-One Machine Learning/Deep Learning/Large Model AI Platform, MLOps algorithm link full process, computing power rental platform, notebook online development, drag-and-drop task flow pipeline orchestration, multi-machine multi-GPU distributed training, hyperparameter search, inference service VGPU virtualization, edge computing, annotation platform automated annotation, deepseek and other large models sft fine-tuning/reward model/reinforcement learning training, vllm/ollama/mindie large model multi-machine inference, private knowledge base, AI model market, supports domestic CPU/GPU/NPU Ascend ecosystem, supports RDMA, supports distributed training such as pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano

tencentmusic
โญ 5,085 ๐Ÿ‘ค tencentmusic Organization ๐Ÿ• Updated 2026-10-09 ๐Ÿ“… Created 2021-08-17
38
langwatch/langwatch

Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.

langwatch
โญ 5,050 TypeScript ๐Ÿ‘ค langwatch Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-09-09
39
itsdecodingai/llm-twin-course

๐Ÿค– ๐—Ÿ๐—ฒ๐—ฎ๐—ฟ๐—ป for ๐—ณ๐—ฟ๐—ฒ๐—ฒ how to ๐—ฏ๐˜‚๐—ถ๐—น๐—ฑ an end-to-end ๐—ฝ๐—ฟ๐—ผ๐—ฑ๐˜‚๐—ฐ๐˜๐—ถ๐—ผ๐—ป-๐—ฟ๐—ฒ๐—ฎ๐—ฑ๐˜† ๐—Ÿ๐—Ÿ๐—  & ๐—ฅ๐—”๐—š ๐˜€๐˜†๐˜€๐˜๐—ฒ๐—บ using ๐—Ÿ๐—Ÿ๐— ๐—ข๐—ฝ๐˜€ best practices: ~ ๐˜ด๐˜ฐ๐˜ถ๐˜ณ๐˜ค๐˜ฆ ๐˜ค๐˜ฐ๐˜ฅ๐˜ฆ + 12 ๐˜ฉ๐˜ข๐˜ฏ๐˜ฅ๐˜ด-๐˜ฐ๐˜ฏ ๐˜ญ๐˜ฆ๐˜ด๐˜ด๐˜ฐ๐˜ฏ๐˜ด

itsdecodingai
โญ 4,413 Python ๐Ÿ‘ค itsdecodingai Organization ๐Ÿ• Updated 2026-04-20 ๐Ÿ“… Created 2024-03-08
40
truefoundry/cognita

RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry

truefoundry
โญ 4,329 Python ๐Ÿ‘ค truefoundry Organization ๐Ÿ• Updated 2026-02-23 ๐Ÿ“… Created 2023-07-26
41
predibase/lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

predibase
โญ 3,836 Python ๐Ÿ‘ค predibase Organization ๐Ÿ• Updated 2026-05-28 ๐Ÿ“… Created 2023-10-20
42
memodb-io/Acontext

Agent Skills as a Memory Layer

memodb-io
โญ 3,697 JavaScript ๐Ÿ‘ค memodb-io Organization ๐Ÿ• Updated 2026-07-14 ๐Ÿ“… Created 2025-07-16
43
truera/trulens

Evaluation and Tracking for LLM Experiments and AI Agents

truera
โญ 3,665 Python ๐Ÿ‘ค truera Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2020-11-02
44
lmnr-ai/lmnr

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

lmnr-ai
โญ 3,516 TypeScript ๐Ÿ‘ค lmnr-ai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-08-29
45
pezzolabs/pezzo

๐Ÿ•น๏ธ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

pezzolabs
โญ 3,287 TypeScript ๐Ÿ‘ค pezzolabs Organization ๐Ÿ• Updated 2026-11-08 ๐Ÿ“… Created 2023-04-22
46
Josh-XT/AGiXT

AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memory, smart features, and a versatile plugin system, AGiXT delivers efficient and comprehensive AI solutions.

Josh-XT
โญ 3,214 Python ๐Ÿ‘ค Josh-XT User ๐Ÿ• Updated 2026-07-28 ๐Ÿ“… Created 2023-04-17
47
protectai/llm-guard

The Security Toolkit for LLM Interactions

protectai
โญ 3,156 Python ๐Ÿ‘ค protectai Organization ๐Ÿ• Updated 2025-12-15 ๐Ÿ“… Created 2023-07-27
48
itsdecodingai/second-brain-ai-assistant-course

Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems techniques.

itsdecodingai
โญ 3,138 Jupyter Notebook ๐Ÿ‘ค itsdecodingai Organization ๐Ÿ• Updated 2026-04-06 ๐Ÿ“… Created 2024-12-30
49
vllm-project/vllm-ascend

Community maintained hardware plugin for vLLM on Huawei Ascend

vllm-project
โญ 3,128 Python ๐Ÿ‘ค vllm-project Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-01-29
50
openlit/openlit

Same agent, smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.

openlit
โญ 3,094 TypeScript ๐Ÿ‘ค openlit Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2024-01-23

Frequently Asked Questions

1

What is the most popular #llmops repository on GitHub?

The top ranked open source #llmops project on GitHub is BerriAI/litellm with 63,649 stars. The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

2

Which programming languages are most used in #llmops repositories?

Among the top ranked #llmops projects on GitHub, the most common programming languages are Python, TypeScript, Jupyter Notebook, Rust.

3

How is the #llmops repository ranking calculated?

We track 111 active open source repositories tagged with #llmops (500+ stars), totaling 716,901 stars with an average of 6,458 stars per repository. Rankings update daily from the GitHub API.