Git Stars Git Stars

#llm-as-judge Repositories

Discover 5 active open source GitHub repositories tagged with #llm-as-judge, accumulating 5,114 total stars (averaging 1,022 stars per project). Leading this topic is chunxiaoxx/nautilus-compass with 1,260 stars.

5
Total Repositories
5,114
Total Stars
1,022
Avg Stars/Repo
Programming Languages: Python Jupyter Notebook
1
chunxiaoxx/nautilus-compass

Open memory and reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) โ€” by Nautilus Platform

chunxiaoxx
โญ 1,260 Python ๐Ÿ‘ค chunxiaoxx User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2026-04-27
2
langchain-ai/openevals

Readymade evaluators for your LLM apps

langchain-ai
โญ 1,259 Python ๐Ÿ‘ค langchain-ai Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2025-02-08
3
Jwuthri/Tracely-ai

Trace-native CI/CD for AI agents โ€” production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.

Jwuthri
โญ 1,122 Python ๐Ÿ‘ค Jwuthri User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-06-03
4
calmrocks/ai-engineer-notebooks

Hands-on, framework-free Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set โ€” model APIs, structured output, tool calling, RAG, evals-as-the-spine, agents (loop from scratch, tool design, guardrails, MCP, Skills), fine-tuning vs LoRA, prompt-injection/security, LLMOps, and customer craft. Runs on the free Groq API.

calmrocks
โญ 756 Jupyter Notebook ๐Ÿ‘ค calmrocks User ๐Ÿ• Updated 2026-11-07 ๐Ÿ“… Created 2026-08-11
5
ksimback/looper

Design visual, review-gated agent loops for Claude Code before you run them.

ksimback
โญ 720 Python ๐Ÿ‘ค ksimback User ๐Ÿ• Updated 2026-08-09 ๐Ÿ“… Created 2026-06-18

Frequently Asked Questions

1

What is the most popular #llm-as-judge repository on GitHub?

The top ranked open source #llm-as-judge project on GitHub is chunxiaoxx/nautilus-compass with 1,260 stars. Open memory and reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) โ€” by Nautilus Platform

2

Which programming languages are most used in #llm-as-judge repositories?

Among the top ranked #llm-as-judge projects on GitHub, the most common programming languages are Python, Jupyter Notebook.

3

How is the #llm-as-judge repository ranking calculated?

We track 5 active open source repositories tagged with #llm-as-judge (500+ stars), totaling 5,114 stars with an average of 1,022 stars per repository. Rankings update daily from the GitHub API.