Git Stars Git Stars

#llm-benchmarking Repositories

Discover 17 active open source GitHub repositories tagged with #llm-benchmarking, accumulating 47,048 total stars (averaging 2,767 stars per project). Leading this topic is HKUDS/AI-Trader with 23,345 stars.

17
Total Repositories
47,048
Total Stars
2,767
Avg Stars/Repo
1
HKUDS/AI-Trader

"AI-Trader: 100% Fully-Automated Agent-Native Trading"

HKUDS
โญ 23,345 Python ๐Ÿ‘ค HKUDS Organization ๐Ÿ• Updated 2026-06-11 ๐Ÿ“… Created 2025-10-23
2
microsoft/promptbase

All things prompt engineering

microsoft
โญ 5,723 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2024-06-04 ๐Ÿ“… Created 2023-12-12
4
NVIDIA/RULER

This repo contains the source code for RULER: Whatโ€™s the Real Context Size of Your Long-Context Language Models?

NVIDIA
โญ 1,641 Python ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2026-11-03 ๐Ÿ“… Created 2024-04-09
6
SalesforceAIResearch/enterprise-deep-research

Salesforce Enterprise Deep Research

SalesforceAIResearch
โญ 1,226 Python ๐Ÿ‘ค SalesforceAIResearch Organization ๐Ÿ• Updated 2026-10-26 ๐Ÿ“… Created 2025-09-16
7
bird-bench/BIRD-CRITIC-1

[NeurIPS 2025 Main] SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications

bird-bench
โญ 1,106 Python ๐Ÿ‘ค bird-bench User ๐Ÿ• Updated 2026-03-23 ๐Ÿ“… Created 2025-01-28
8
ray-project/llmperf

LLMPerf is a library for validating and benchmarking LLMs

ray-project
โญ 1,066 Python ๐Ÿ‘ค ray-project Organization ๐Ÿ• Updated 2024-12-09 ๐Ÿ“… Created 2023-10-10
9
carlini/yet-another-applied-llm-benchmark

A benchmark to evaluate language models on questions I've previously asked them to solve.

carlini
โญ 1,058 Python ๐Ÿ‘ค carlini User ๐Ÿ• Updated 2025-04-27 ๐Ÿ“… Created 2023-12-18
10
bigcode-project/bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.

bigcode-project
โญ 1,054 Python ๐Ÿ‘ค bigcode-project Organization ๐Ÿ• Updated 2025-07-22 ๐Ÿ“… Created 2022-08-09
11
VILA-Lab/ATLAS

A principled instruction benchmark on formulating effective queries and prompts for large language models (LLMs). Our paper: https://arxiv.org/abs/2312.16171

VILA-Lab
โญ 978 Python ๐Ÿ‘ค VILA-Lab Organization ๐Ÿ• Updated 2024-05-28 ๐Ÿ“… Created 2023-12-08
12
defog-ai/sql-eval

Evaluate the accuracy of LLM generated outputs

defog-ai
โญ 744 Jupyter Notebook ๐Ÿ‘ค defog-ai Organization ๐Ÿ• Updated 2025-08-14 ๐Ÿ“… Created 2023-08-10
13
lupantech/ScienceQA

Data and code for NeurIPS 2022 Paper "Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering".

lupantech
โญ 715 Python ๐Ÿ‘ค lupantech User ๐Ÿ• Updated 2024-09-19 ๐Ÿ“… Created 2022-10-17
14
getomni-ai
โญ 639 TypeScript ๐Ÿ‘ค getomni-ai Organization ๐Ÿ• Updated 2025-10-21 ๐Ÿ“… Created 2024-12-16
16
TIGER-AI-Lab/WebsiteBench

Can AI agents rebuild websites through browser exploration?

TIGER-AI-Lab
โญ 595 Python ๐Ÿ‘ค TIGER-AI-Lab Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2026-11-04

Frequently Asked Questions

1

What is the most popular #llm-benchmarking repository on GitHub?

The top ranked open source #llm-benchmarking project on GitHub is HKUDS/AI-Trader with 23,345 stars. "AI-Trader: 100% Fully-Automated Agent-Native Trading"

2

Which programming languages are most used in #llm-benchmarking repositories?

Among the top ranked #llm-benchmarking projects on GitHub, the most common programming languages are Python, Jupyter Notebook, TypeScript.

3

How is the #llm-benchmarking repository ranking calculated?

We track 17 active open source repositories tagged with #llm-benchmarking (500+ stars), totaling 47,048 stars with an average of 2,767 stars per repository. Rankings update daily from the GitHub API.