What is WebVoyager?
WebVoyager is an open-source web agent and benchmark, built by researchers identified as affiliated with Tencent and described in a 2024 arXiv paper. It pairs a large multimodal model (GPT-4V in the reference implementation) with a Selenium-based browser environment: screenshots and the accessibility tree feed in as input, and the agent clicks, types, and scrolls on real, live websites. The repo ships 643 curated task queries across 15 websites, plus 90 tasks drawn from GAIA, reference answers, and a GPT-4V-based automated evaluation script.
Our verdict
WebVoyager is a reasonable pick for teams who want to test multimodal agents against real, live websites instead of sandboxed replicas, and its automated GPT-4V grading keeps large-scale evaluation practical. The live-site design means tasks can break or drift as pages change, and tying the reference evaluator to GPT-4V limits reproducibility for anyone without access to that model.
Categories:
These are research benchmarks, not products you buy. They define fixed web tasks and score how reliably an agent or model completes them, which is the closest thing the field has to an objective measure of web-agent capability. They are listed here for evaluation, not procurement.
