BrowserGym
BrowserGym is an open-source framework for web agent research that provides a standardized environment for training and evaluating AI agents on web tasks. It solves the problem of benchmarking web agents by offering a unified interface to multiple benchmarks and an easy way to create new ones. It is intended for researchers and developers working on web agents.
✨ Key features
- Unified Gymnasium interface for multiple web benchmarks
- Includes MiniWoB, WebArena, VisualWebArena, WorkArena, and more
- Easy to create custom benchmarks by inheriting AbstractBrowserTask
- Supports open-ended interactive tasks with chat
- Integrates with AgentLab for running and analyzing agent experiments
- Provides demo agent with OpenAI backend
🎯 Use cases
- Benchmarking web agents on standard tasks like MiniWoB and WebArena
- Developing and testing custom web agent tasks
- Running interactive open-ended web tasks with a chat interface
- Evaluating agents on realistic web tasks from WorkArena and AssistantBench
📦 Installation
🧰 Requirements: Python, pip, and Playwright with Chromium installed. Some benchmarks may require additional setup (e.g., API keys, self-hosted environments).
pip install browsergym
Then install Chromium:
playwright install chromium
For specific benchmarks, install additional packages and follow their setup instructions.
🚀 Usage
import gymnasium as gym
import browsergym.core # register the openended task as a gym environment
# start an openended environment
env = gym.make(
"browsergym/openended",
task_kwargs={"start_url": "https://www.google.com/"},
wait_for_user_message=True,
)
obs, info = env.reset()
while True:
action = ... # implement your agent here
obs, reward, terminated, truncated, info = env.step(action)
if terminated or truncated:
break
env.close()
⚠️ Good to know
BrowserGym is not a consumer product and is intended for research; use with caution. Some benchmarks require additional setup steps.
❓ FAQ
How do I install BrowserGym?
Run pip install browsergym and then playwright install chromium to set up the browser.
Which benchmarks are included by default?
MiniWoB, WebArena, WebArenaVerified, VisualWebArena, WorkArena, AssistantBench, WebLINX, OpenApps, and TimeWarp.
Can I create my own benchmark?
Yes, by inheriting the AbstractBrowserTask class.
How do I run a demo agent?
Follow the demo setup in the README, set your OPENAI_API_KEY, and run python demo_agent/run_demo.py --task_name <task>.
📊 Repository
🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.