Git Stars Git Stars

#llm-eval Repositories

Discover 7 active open source GitHub repositories tagged with #llm-eval, accumulating 60,582 total stars (averaging 8,654 stars per project). Leading this topic is promptfoo/promptfoo with 27,516 stars.

7
Total Repositories
60,582
Total Stars
8,654
Avg Stars/Repo
Programming Languages: Python TypeScript
1
promptfoo/promptfoo

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

promptfoo
โญ 27,516 TypeScript ๐Ÿ‘ค promptfoo Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-04-28
2
Arize-ai/phoenix

AI Observability & Evaluation

Arize-ai
โญ 12,368 Python ๐Ÿ‘ค Arize-ai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2022-11-09
3
open-compass/opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc., across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.

open-compass
โญ 7,610 Python ๐Ÿ‘ค open-compass Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2023-06-15
4
Giskard-AI/giskard-oss

๐Ÿข Open-Source Evaluation & Testing library for LLM Agents

Giskard-AI
โญ 5,965 Python ๐Ÿ‘ค Giskard-AI Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2022-03-06
5
truera/trulens

Evaluation and Tracking for LLM Experiments and AI Agents

truera
โญ 3,665 Python ๐Ÿ‘ค truera Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2020-11-02
6
uptrain-ai/uptrain

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

uptrain-ai
โญ 2,337 Python ๐Ÿ‘ค uptrain-ai Organization ๐Ÿ• Updated 2024-08-18 ๐Ÿ“… Created 2022-11-07
7
AI-QL/tuui

A desktop MCP client designed as a tool unitary utility integration, accelerating AI adoption through the Model Context Protocol (MCP) and enabling cross-vendor LLM API orchestration.

AI-QL
โญ 1,161 TypeScript ๐Ÿ‘ค AI-QL Organization ๐Ÿ• Updated 2026-05-14 ๐Ÿ“… Created 2025-04-01

Frequently Asked Questions

1

What is the most popular #llm-eval repository on GitHub?

The top ranked open source #llm-eval project on GitHub is promptfoo/promptfoo with 27,516 stars. Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

2

Which programming languages are most used in #llm-eval repositories?

Among the top ranked #llm-eval projects on GitHub, the most common programming languages are Python, TypeScript.

3

How is the #llm-eval repository ranking calculated?

We track 7 active open source repositories tagged with #llm-eval (500+ stars), totaling 60,582 stars with an average of 8,654 stars per repository. Rankings update daily from the GitHub API.