What is BrowserGym?
BrowserGym is an open-source Python framework from ServiceNow Research that provides a Gymnasium-style (OpenAI Gym) API for building and evaluating web-browsing AI agents, using Playwright for real browser automation. It wraps third-party benchmarks, including MiniWoB++, WebArena, VisualWebArena, WorkArena, AssistantBench, WebLINX, OpenApps, and TimeWarp, in a single harness, so the same agent code runs against any of them. It installs via pip and ships with a demo LLM-backed agent implementation.
Our verdict
It suits researchers and teams who want one API for evaluating web-browsing agents across multiple benchmarks instead of writing separate adapters for each. As research infrastructure from a corporate lab with roughly 1.3k GitHub stars, it lacks the productization, support, and stability guarantees of a commercial tool.
Categories:
These are research benchmarks, not products you buy. They define fixed web tasks and score how reliably an agent or model completes them, which is the closest thing the field has to an objective measure of web-agent capability. They are listed here for evaluation, not procurement.
