#benchmark Repositories
193 repositories total
BenchMARL is a library for benchmarking Multi-Agent Reinforcement Learning (MARL). BenchMARL allows to quickly compare different MARL algorithms, tasks, and models while being systematically grounded in its two core tenets: reproducibility and standardization.
AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.
A.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.
Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. A curated list of tools, benchmarks/datasets, demos, leaderboards, and large models, primarily focused on the evaluation of foundation large models, aiming to explore the technological boundaries of generative AI.
A repository of pretty cool datasets that I collected for network science and machine learning research.
CAIRI Supervised, Semi- and Self-Supervised Visual Representation Learning Toolbox and Benchmark
[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.
The first-ever vast natural language processing benchmark for Indonesian Language. We provide multiple downstream tasks, pre-trained IndoBERT models, and a starter code! (AACL-IJCNLP 2020)
A global network of probes to run network tests like ping, traceroute and DNS resolve
Federated Learning Benchmark - Federated Learning on Non-IID Data Silos: An Experimental Study (ICDE 2022)
JailbreakBench: An Open Robustness Benchmark for Jailbreaking Language Models [NeurIPS 2024 Datasets and Benchmarks Track]
A modern iperf3 alternative with a live TUI, multi-client server, and QUIC support. Built in Rust.
Aixio Layer v2 recovers editable layers from flat designs where possible. 150-design benchmark vs Canva Magic Layers, Qwen-Image-Layer and GPT-6 Astra Ultra.
Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.
A more modern http framework benchmarker supporting HTTP/1 and HTTP/2 benchmarks.
Serialization library written in C++17 - Pack C++ structs into a compact byte-array without any macros or boilerplate code
Naive performance comparison of a few programming languages (JavaScript, Kotlin, Rust, Swift, Nim, Python, Go, Haskell, D, C++, Java, C#, Object Pascal, Ada, Lua, Ruby)
SpeechIO Leaderboard: a large, robust, comprehensive, benchmarking platform for Automatic Speech Recognition.
Performance benchmarking and testing framework for .NET applications :chart_with_upwards_trend:
Peak compute, memory bandwidth and latency of CPUs, GPUs and NPUs: the same tests through every API that reaches a device. Command line and app for Windows, macOS, Linux and Android
The official repo of GraphRAG-Bench for evaluating GraphRAG models. "When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation". (ICLR'26)
Current state of supervised and unsupervised depth completion methods
Token overruns? Better call Smith! Better Call Smith. Adaptive model routing, a glassy Codex usage monitor, and reproducible benchmarks with receipts.
The official GitHub repository of the paper "Recent advances in large language model benchmarks against data contamination: From static to dynamic evaluation"