agent-qa
agent-qa is a self-improving agentic QA harness that lets you write tests in natural language for web and mobile. It solves the problem of flaky, brittle UI tests by self-healing failed actions, building execution memory to improve future runs, and providing a dashboard, CLI, and MCP for developers and coding agents. It is for developers and QA engineers who want to create and maintain robust end-to-end tests with less effort.
✨ Key features
- Write tests in natural language for web and mobile.
- Self-healing test execution recovers from UI drift.
- Builds execution memory to improve future runs.
- Smart cache reuses validated plans to reduce runtime.
- Run sandboxed hooks in Docker for setup and teardown.
- Bring your own LLM via OpenAI, Anthropic, Gemini, etc.
🎯 Use cases
- Create end-to-end tests for web apps using plain English.
- Automate mobile app testing with natural language scenarios.
- Set up test environments and call APIs using sandboxed hooks.
- Run tests in CI with self-healing to reduce flaky failures.
- Integrate QA with coding agents via MCP and skills.
📦 Installation
🧰 Requirements: Requires Node.js and npm; Docker is required for hooks. Optional subscription auth for Codex or Claude Code.
npm install -D agent-qa
For Codex or Claude Code subscription auth, also install:
npm install -D @vostride/agent-qa-subscription-auth
Install Docker before using hooks. agent-qa runs hooks in a sandboxed runtime, and Docker is required for the Node, Bun, Python, and Bash hook containers.
Initialize agent-qa and install the runtime support you need:
npx agent-qa init
npx agent-qa install-browsers --chromiumnpx agent-qa install-mobile-drivers --all🚀 Usage
Start the dashboard, complete auth, and run tests from the UI:
npx agent-qa dashboard --open
Run tests from the CLI:
npx agent-qa run tests/hacker-news-top-story.yaml
❓ FAQ
What LLMs can I use with agent-qa?
You can use any model via OpenAI- and Anthropic-compatible endpoints, Gemini, local or open-source models, and subscriptions like Codex and Claude Code.
Do I need Docker?
Docker is required only if you use hooks, as they run in sandboxed containers for Node, Bun, Python, and Bash.
How does self-healing work?
When a sub-action like click or fill fails, agent-qa re-observes the UI and tries a different path in the same run, recovering from UI drift instead of failing immediately.
Can I run tests from the command line?
Yes, you can run tests using the CLI command npx agent-qa run <test-file>.
📊 Repository
🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.