What is WebCanvas?
WebCanvas is an open-source, MIT-licensed framework for building, training, and evaluating web agents in live, real-time online environments instead of static offline snapshots. It scores task completion with a "key nodes" methodology, breaking each task into phases rather than a single pass/fail check. The project ships the Mind2Web-Live dataset (542 tasks, 2,439 intermediate evaluation states, hosted on Hugging Face) with raw HTML, screenshots, DOM trees, and action videos for each task.
Our verdict
It suits builders who need to test agents against live, changing websites rather than cached pages, thanks to phase-based key-nodes scoring that gives partial credit on multi-step tasks. The project is comparatively small, about 280 GitHub stars and 20 forks, with its most recent tagged release in December 2024, so it looks less actively maintained than the major academic web-agent benchmarks it sits alongside.
Categories:
These are research benchmarks, not products you buy. They define fixed web tasks and score how reliably an agent or model completes them, which is the closest thing the field has to an objective measure of web-agent capability. They are listed here for evaluation, not procurement.
