Benchmarks

GAIA

Benchmark for evaluating general AI agents on reasoning and tool use

WebArena

Self-hosted, open-source benchmark for autonomous web agents

WebVoyager

Open-source multimodal web agent with a 643-task benchmark

AssistantBench

Academic benchmark testing AI agents on realistic web tasks

Bananalyzer

Open-source Playwright benchmark for evaluating AI web agents

BrowserGym

Gym-style API unifying web-agent benchmarks like WebArena

Mind2Web

Benchmark of 2,350 tasks across 137 real websites for web agents

MiniWoB++

Benchmark suite of 100+ simulated web tasks for web agents

OSWorld

Benchmark for evaluating computer-use AI agents on real tasks

VisualWebArena

Benchmark of 910 visually grounded tasks for multimodal web agents

WebCanvas

Open-source benchmark for evaluating web agents online

WebShop

Simulated e-commerce benchmark for training web-shopping agents