Autoresearch Agent — Development

Q: Can I use my own evaluation script?

Yes, you can write a custom evaluate.py that prints 'metric_name: value' to stdout. It must not be modified after the experiment starts.

Autoresearch Agent is an autonomous experiment loop that optimizes any file against a measurable metric. It iteratively edits a target file, runs a fixed evaluation command, keeps improvements via git commits, and discards failures via git resets, looping until stopped or a goal is met.

engineering claude-code

📂 View skill on GitHub ↗ 🚀 How to use

✨ What it does

Autonomous loop with configurable intervals (10m, 1h, daily, weekly, monthly).
One-change-per-experiment rule to isolate what works.
Built-in evaluators for speed, size, test pass rate, and LLM-judged content quality.
Automatic git commit on improvement and reset on failure.
Strategy escalation from low-hanging fruit to radical experiments.
Self-improvement: updates program.md with learned patterns every 10 runs.

🎯 When to use it

When you want to optimize code speed, memory usage, bundle size, or test pass rate.
When you need to improve content quality such as headlines, copy, or prompts based on a measurable score.
When you want to run overnight experiments to automatically find improvements.
When you have a target file and a way to measure success, and you want a systematic, iterative optimization loop.

🚀 How to use

Trigger: /ar:setup

Trigger with slash commands like /ar:setup to create an experiment, /ar:run for a single iteration, or /ar:loop to start the autonomous loop. The skill activates when you ask to optimize a file for a metric (e.g., 'make this faster', 'improve CTR'). Setup requires a target file, an evaluation command that outputs a metric, and a git repo. Example prompts:

/ar:setup --domain engineering --name api-speed --target src/api/search.py --eval "pytest bench.py --tb=no -q" --metric p50_ms --direction lower

Optimize src/api/search.py for p50_ms (lower is better) using the eval command 'pytest bench.py --tb=no -q'.

📄 Output: The skill produces an experiment directory with results.tsv logging each run's metric and status, plus optional dashboards and exported reports.

📦 Add this skill to Claude Code

# 1. Get the skills repo
git clone --depth 1 https://github.com/alirezarezvani/claude-skills /tmp/claude-skills

# 2. Copy this skill into your project (or ~/.claude/skills for all projects)
mkdir -p .claude/skills
cp -r /tmp/claude-skills/engineering/autoresearch-agent/skills/autoresearch-agent .claude/skills/autoresearch-agent

Skill source: engineering/autoresearch-agent/skills/autoresearch-agent/SKILL.md

⚠️ Good to know

Requires a git repository, a target file, and a working evaluation command that outputs a numeric metric; the agent must not modify the evaluator, and no new dependencies are allowed.

❓ FAQ

What happens if an experiment crashes?

If it's a typo or missing import, the agent fixes and re-runs. If the idea is fundamentally broken, it reverts, logs 'crash', and moves on. After 5 consecutive crashes, the loop pauses and alerts the user.

Can I use my own evaluation script?

Yes, you can write a custom evaluate.py that prints metric_name: value to stdout. It must not be modified after the experiment starts.

How does the agent decide what to change?

It reviews results.tsv for history, then makes one change per iteration, escalating from simple optimizations to structural changes over runs.

🤖 Overview, features, install steps and FAQ were generated from the project's SKILL.md on Sep 4, 2026. Always check the original source before running commands.