#benchmark Repositories
193 repositories total
An objective comparison of multiple frameworks that allow us to "transform" our web apps to desktop applications.
SkillsBench evaluates how well skills work and how effective agents are at using them.
Playing around "Less Slow" coding practices in C++ 20, C, CUDA, PTX, & Assembly, from numerics & SIMD to coroutines, ranges, exception handling, networking and user-space IO
Rust profiler for CPU, memory, SQL, HTTP, and async performance, with Prometheus and Grafana support.
BEHAVIOR-1K: a platform for accelerating Embodied AI research. Join our Discord for support: https://discord.gg/bccR5vGFEx
Simple, fast, accurate single-header microbenchmarking functionality for C++11/14/17/20
A deep learning toolkit for healthcare AI: EHR (MIMIC, eICU, OMOP), clinical text, signals, and imaging. 30+ models, reproducible pipelines.
The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".
Benchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM
Fast Compiler for C# Expression Trees and the lightweight LightExpression alternative. Diagnostic and code generation tools for the expressions.
[pip install medmnist] 18 Standardized Datasets for 2D and 3D Biomedical Image Classification
Modular Deep Reinforcement Learning framework in PyTorch. Companion library of the book "Foundations of Deep Reinforcement Learning".
KernelBench: Can LLMs Write GPU Kernels? - Benchmark + Toolkit with Torch -> CUDA (+ more DSLs)
GitHub Action for continuous benchmarking to keep performance
Open memory and reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) โ by Nautilus Platform
Reading notes about Multimodal Large Language Models, Large Language Models, and Diffusion Models
A clinical decision-evidence benchmark: pull the one fact an answer rests on, and see whether the assistant's action moves with the evidence. Built on HealthBench.
JMLR: OmniSafe is an infrastructural framework for accelerating SafeRL research.