What is Mind2Web?
Mind2Web is an academic benchmark from the Ohio State University NLP Group. It tests generalist web agents that follow natural-language instructions on real, live websites instead of simulated environments. The dataset has 2,350 tasks across 137 websites and 31 domains, averaging 7.3 actions per task, and checks three kinds of generalization: cross-task, cross-website, and cross-domain. The paper appeared at NeurIPS 2023; the code and eval harness are on GitHub, and the dataset (DOM snapshots, screenshots, action traces) is hosted on Hugging Face and Globus.
Our verdict
Mind2Web is a useful reference for testing whether an agent generalizes to unseen websites and domains, since its tasks run against real production sites instead of sandboxes. It's built around raw DOM elements rather than visual grounding, so teams evaluating screenshot- or vision-driven agents need a benchmark built for that modality.
Categories:
These are research benchmarks, not products you buy. They define fixed web tasks and score how reliably an agent or model completes them, which is the closest thing the field has to an objective measure of web-agent capability. They are listed here for evaluation, not procurement.
