#evals Repositories
Discover 25 active open source GitHub repositories tagged with #evals, accumulating 108,810 total stars (averaging 4,352 stars per project). Leading this topic is mastra-ai/mastra with 29,719 stars.
25 repositories total
Mastra is the modern TypeScript framework for AI-powered applications and agents.
Observability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement.
Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.
AI system design guide for engineers building production AI systems and evals.
Build tested agent skills and govern their lifecycle through a user-defined marketplace: evidence, discovery, updates, rollback, quarantine, and 17-platform distribution.
Testing and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps.
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing ยท Evals ยท Simulations ยท Datasets ยท Gateway ยท Guardrails. Self-hostable. Apache 2.0.
Waku Agent is an open-source AI agent harness you own: loop, memory, evals, and an always-on layer (event inbox, timers, approvals, outbox), in code that stays readable.
Build agentic systems. Run them with confidence. Orchestrate agents, automate business processes, inspect every execution, and keep humans in control. Deploy Heym on your own infrastructure.
OpenSource Production ready Customer service with built in Evals and monitoring
The skillbook for recursive self-improvement (RSI). NVIDIA's latest proof, unpacked with timeless engineering. Distilled from the OpenAI harness engineering hackathon winner's zero-touch method.
๐ฅค RAGLite is a Python toolkit for Retrieval-Augmented Generation (RAG) with DuckDB or PostgreSQL
Trace-native CI/CD for AI agents โ production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.
A curated, non-BS library of the best resources for building and evaluating AI agents โ papers, blogs, talks, tools, benchmarks. Maintained by BenchFlow.
A skill creator that proves its skills work. Evidence-driven skill creation for Claude Code and Codex: baseline-tested generation, per-skill regression evals, ecosystem doctor, cross-runtime compile, and an opt-in proactive advisor.
Hands-on, framework-free Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set โ model APIs, structured output, tool calling, RAG, evals-as-the-spine, agents (loop from scratch, tool design, guardrails, MCP, Skills), fine-tuning vs LoRA, prompt-injection/security, LLMOps, and customer craft. Runs on the free Groq API.
The self-improvement layer for AI agents. Ratel reads your agent's traces, finds what fails, recommends fixes, and runs evaluations on real sessions before you ship. Self-hosted: one app, one Postgres. Includes the TypeScript and Python SDKs.
Frequently Asked Questions
What is the most popular #evals repository on GitHub?
The top ranked open source #evals project on GitHub is mastra-ai/mastra with 29,719 stars. Mastra is the modern TypeScript framework for AI-powered applications and agents.
Which programming languages are most used in #evals repositories?
Among the top ranked #evals projects on GitHub, the most common programming languages are Python, TypeScript, JavaScript, Jupyter Notebook.
How is the #evals repository ranking calculated?
We track 25 active open source repositories tagged with #evals (500+ stars), totaling 108,810 stars with an average of 4,352 stars per repository. Rankings update daily from the GitHub API.