Git Stars Git Stars

#evaluation-metrics Repositories

Discover 18 active open source GitHub repositories tagged with #evaluation-metrics, accumulating 46,395 total stars (averaging 2,577 stars per project). Leading this topic is confident-ai/deepeval with 19,585 stars.

18
Total Repositories
46,395
Total Stars
2,577
Avg Stars/Repo
Programming Languages: Python Jupyter Notebook
1
confident-ai/deepeval

The LLM Evaluation Framework

confident-ai
โญ 19,585 Python ๐Ÿ‘ค confident-ai Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2023-08-10
2
AgentOps-AI/agentops

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

AgentOps-AI
โญ 5,971 Python ๐Ÿ‘ค AgentOps-AI Organization ๐Ÿ• Updated 2026-06-25 ๐Ÿ“… Created 2023-08-15
3
datawhalechina/tiny-universe

"Big Model White-Box Construction Guide": A completely hand-built Tiny-Universe

datawhalechina
โญ 5,148 Jupyter Notebook ๐Ÿ‘ค datawhalechina Organization ๐Ÿ• Updated 2026-02-12 ๐Ÿ“… Created 2024-04-06
4
huggingface/lighteval

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

huggingface
โญ 2,587 Python ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-01-26
5
huggingface/evaluation-guidebook

Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!

huggingface
โญ 2,166 Jupyter Notebook ๐Ÿ‘ค huggingface Organization ๐Ÿ• Updated 2025-12-03 ๐Ÿ“… Created 2024-10-09
6
xinshuoweng/AB3DMOT

(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"

xinshuoweng
โญ 1,805 Python ๐Ÿ‘ค xinshuoweng User ๐Ÿ• Updated 2024-04-03 ๐Ÿ“… Created 2019-06-19
8
activitynet/ActivityNet

This repository is intended to host tools and demos for ActivityNet

activitynet
โญ 965 Jupyter Notebook ๐Ÿ‘ค activitynet Organization ๐Ÿ• Updated 2024-03-21 ๐Ÿ“… Created 2015-10-23
9
jitsi/jiwer

Evaluate your speech-to-text system with similarity measures such as word error rate (WER)

jitsi
โญ 946 Python ๐Ÿ‘ค jitsi Organization ๐Ÿ• Updated 2026-09-03 ๐Ÿ“… Created 2018-06-19
10
Yuliang-Liu/MultimodalOCR

On the Hidden Mystery of OCR in Large Multimodal Models (OCRBench)

Yuliang-Liu
โญ 902 Python ๐Ÿ‘ค Yuliang-Liu User ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2023-05-12
11
MIND-Lab/OCTIS

OCTIS: Comparing Topic Models is Simple! A python package to optimize and evaluate topic models (accepted at EACL2021 demo track)

MIND-Lab
โญ 807 Python ๐Ÿ‘ค MIND-Lab Organization ๐Ÿ• Updated 2026-06-21 ๐Ÿ“… Created 2020-03-13
12
Unbabel/COMET

A Neural Framework for MT Evaluation

Unbabel
โญ 791 Python ๐Ÿ‘ค Unbabel Organization ๐Ÿ• Updated 2026-04-21 ๐Ÿ“… Created 2020-05-28
13
mir-evaluation/mir_eval

Evaluation functions for music/audio information retrieval/signal processing algorithms.

mir-evaluation
โญ 719 Python ๐Ÿ‘ค mir-evaluation Organization ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2013-08-06
14
AmenRa/ranx

โšก๏ธA Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion ๐Ÿ

AmenRa
โญ 691 Python ๐Ÿ‘ค AmenRa User ๐Ÿ• Updated 2025-08-07 ๐Ÿ“… Created 2020-06-02
15
nekhtiari/image-similarity-measures

:chart_with_upwards_trend: Implementation of eight evaluation metrics to access the similarity between two images. The eight metrics are as follows: RMSE, PSNR, SSIM, ISSM, FSIM, SRE, SAM, and UIQ.

nekhtiari
โญ 649 Python ๐Ÿ‘ค nekhtiari User ๐Ÿ• Updated 2026-09-05 ๐Ÿ“… Created 2020-04-06
16
statisticianinstilettos/recmetrics

A library of metrics for evaluating recommender systems

statisticianinstilettos
โญ 581 Jupyter Notebook ๐Ÿ‘ค statisticianinstilettos User ๐Ÿ• Updated 2024-01-11 ๐Ÿ“… Created 2018-10-15
โญ 567 Python ๐Ÿ‘ค yuvalkirstain User ๐Ÿ• Updated 2024-12-21 ๐Ÿ“… Created 2023-04-30
18
relari-ai/continuous-eval

Data-Driven Evaluation for LLM-Powered Applications

relari-ai
โญ 519 Python ๐Ÿ‘ค relari-ai Organization ๐Ÿ• Updated 2026-08-10 ๐Ÿ“… Created 2023-12-08

Frequently Asked Questions

1

What is the most popular #evaluation-metrics repository on GitHub?

The top ranked open source #evaluation-metrics project on GitHub is confident-ai/deepeval with 19,585 stars. The LLM Evaluation Framework

2

Which programming languages are most used in #evaluation-metrics repositories?

Among the top ranked #evaluation-metrics projects on GitHub, the most common programming languages are Python, Jupyter Notebook.

3

How is the #evaluation-metrics repository ranking calculated?

We track 18 active open source repositories tagged with #evaluation-metrics (500+ stars), totaling 46,395 stars with an average of 2,577 stars per repository. Rankings update daily from the GitHub API.