What is WebArena?
WebArena is an open-source, self-hostable web environment for building and evaluating autonomous web agents. It was created by researchers including Shuyan Zhou and Frank F. Xu, with the original paper affiliated with Carnegie Mellon University. The benchmark provides 812 human-annotated test tasks across five fully functional, self-hosted websites: an e-commerce shop, a Reddit-like forum, GitLab, Wikipedia, and a map app. Agents are evaluated via accessibility-tree or HTML observations of a real browser. The webarena.dev site also hosts related sibling benchmarks, including VisualWebArena, WebArena-Infinity, and TheAgentCompany.
Our verdict
WebArena is a widely-cited reference point in web-agent research. It was published as a NeurIPS 2024 oral paper, has a public leaderboard, and its Apache-2.0 licensed, self-hostable codebase has roughly 1.6k GitHub stars. It suits teams benchmarking agent architectures against a fixed, reproducible task set, but it is less useful for gauging performance on the live, constantly changing web, since its five environments are static clones rather than production sites.
Categories:
These are research benchmarks, not products you buy. They define fixed web tasks and score how reliably an agent or model completes them, which is the closest thing the field has to an objective measure of web-agent capability. They are listed here for evaluation, not procurement.
