WorkArena
WorkArena is a benchmark suite of browser-based tasks for evaluating web agents on routine knowledge work tasks, built on the ServiceNow platform. It provides a standardized environment to test agents' ability to interact with common enterprise web interfaces. It is intended for researchers and developers working on web automation and AI agents.
✨ Key features
- Includes 19,912 unique instances across 33 atomic tasks (WorkArena-L1).
- WorkArena++ adds 682 compositional tasks for planning and reasoning.
- Covers ServiceNow UI components: lists, forms, menus, dashboards, catalogs.
- Integrates with BrowserGym and AgentLab for evaluation.
- Provides oracle (cheat) functions for task validation and demos.
- Tasks are designed to be unsolved, challenging current agents.
🎯 Use cases
- Benchmarking web agents on knowledge work tasks.
- Evaluating agent performance on ServiceNow-like enterprise interfaces.
- Testing planning and reasoning in compositional tasks (WorkArena++).
- Developing and comparing web automation agents.
📦 Installation
🧰 Requirements: Requires Python, access to ServiceNow instances via Hugging Face gated repository, and Playwright installed.
pip install browsergym-workarena
playwright install
Before installing, gain access to ServiceNow instances by filling the form at https://huggingface.co/datasets/ServiceNow/WorkArena-Instances and authenticating with Hugging Face (e.g., huggingface-cli login).
🚀 Usage
import random
from browsergym.core.env import BrowserEnv
from browsergym.workarena import ATOMIC_TASKS
from time import sleep
random.shuffle(ATOMIC_TASKS)
for task in ATOMIC_TASKS:
print("Task:", task)
env = BrowserEnv(task_entrypoint=task, headless=False)
env.reset()
env.chat.add_message(role="assistant", msg="On it. Please wait...")
cheat_messages = []
env.task.cheat(env.page, cheat_messages)
for cheat_msg in cheat_messages:
env.chat.add_message(role=cheat_msg["role"], msg=cheat_msg["message"])
env.chat.add_message(role="assistant", msg="I'm done!")
reward, stop, message, info = env.task.validate(env.page, cheat_messages)
if reward == 1:
env.chat.add_message(role="user", msg="Yes, that works. Thanks!")
else:
env.chat.add_message(role="user", msg=f"No, that doesn't work. {info.get('message', '')}")
sleep(3)
env.close()
⚠️ Good to know
The benchmark is not solved; current agents often fail on tasks, as shown in the videos.
❓ FAQ
How do I get access to ServiceNow instances?
You need to fill the form on the Hugging Face dataset page (ServiceNow/WorkArena-Instances), accept the terms, and wait for approval. Then authenticate your machine with Hugging Face.
What is the difference between WorkArena-L1 and WorkArena++?
WorkArena-L1 includes 19,912 atomic tasks covering basic UI components, while WorkArena++ (L2/L3) contains 682 compositional tasks that require planning and reasoning.
Can I use WorkArena with AgentLab?
Yes, the preferred way to evaluate is with AgentLab, which runs experiments via BrowserGym and reports on a unified leaderboard.
What does the cheat function do?
The cheat function uses Playwright to automatically solve the task, which is useful for validation and demonstration purposes, but not for evaluating agents.
📊 Repository
🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.