#evaluation-framework Repositories
Discover 13 active open source GitHub repositories tagged with #evaluation-framework, accumulating 100,331 total stars (averaging 7,717 stars per project). Leading this topic is promptfoo/promptfoo with 27,516 stars.
13 repositories total
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing ยท Evals ยท Simulations ยท Datasets ยท Gateway ยท Guardrails. Self-hostable. Apache 2.0.
:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI
This project aims to compare and evaluate the telemetry of various EDR products.
AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.
Inspiring the Next Generation of Segment Anything Models: Comprehensively Evaluate SAM and SAM 2 with Diverse Prompts Towards Context-Dependent Concepts under Different Scenes
Frequently Asked Questions
What is the most popular #evaluation-framework repository on GitHub?
The top ranked open source #evaluation-framework project on GitHub is promptfoo/promptfoo with 27,516 stars. Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
Which programming languages are most used in #evaluation-framework repositories?
Among the top ranked #evaluation-framework projects on GitHub, the most common programming languages are Python, TypeScript, Jupyter Notebook.
How is the #evaluation-framework repository ranking calculated?
We track 13 active open source repositories tagged with #evaluation-framework (500+ stars), totaling 100,331 stars with an average of 7,717 stars per repository. Rankings update daily from the GitHub API.