agent-browser

---
name: agent-browser
description: "Drive a real browser with the agent-browser CLI (headless Chromium over CDP). Use for any browser task: verifying a UI change, taking a screenshot, walking a web page, filling a form, extracting page text, reproducing a frontend bug, or attaching UI evidence to a PR, issue, Linear comment, or Slack thread. Starts with a `command -v agent-browser` check and falls back to manual QA on the slim worker image. Screenshots are kept on agent-fs and embedded in GitHub as user attachments. This is the only browser automation tool; do not use qa-use."
---

# agent-browser

`agent-browser` (vercel-labs) is the swarm's browser-automation CLI. It drives a headless Chromium over CDP and returns accessibility-tree snapshots with compact `@eN` refs. Use it for any browser task: verify a UI change, take a screenshot, walk a page, fill a form, extract page text, reproduce a frontend bug, or attach UI evidence to a PR, issue, Linear comment, or Slack thread.

Do not use `qa-use`. It is not installed in the worker image any more.

## Step 0: check that the CLI is present

The full worker image (`agent-swarm-worker:latest`) ships `agent-browser` and a headless Chromium. The slim image (`:slim`) has neither. The first command of any browser work is:

```bash
command -v agent-browser
```

If it prints nothing, you are on the slim image. Say so in your progress report ("agent-browser is not available on this worker image") and fall back to manual QA: write the exact steps a human should run and mark the browser test cases as **Blocked**, not failed. Do not install a browser or the CLI yourself.

## Step 1: load the version-matched guide

The CLI bundles its own usage guide. Load it once per task before the first browser command. Do not guess flags from memory.

```bash
agent-browser skills get core          # overview + common patterns
agent-browser skills get core --full   # adds the full command reference
agent-browser skills list              # specialized guides (dogfood, electron, slack, ...)
```

## Step 2: the core loop

```bash
agent-browser open <url>
agent-browser snapshot -i              # accessibility tree, interactive elements only, @eN refs
agent-browser click @e12               # act on refs from the snapshot
agent-browser fill @e7 "text"
agent-browser screenshot /tmp/<name>.png
agent-browser close
```

- Re-run `snapshot` after every navigation or state change. Refs are only valid for the snapshot that produced them.
- Run one browser session at a time. Call `store-progress` between page batches so the heartbeat watchdog sees you.
- The browser is headless. Local servers inside the worker container are reachable at `http://localhost:<port>`.
- The image launches Chromium with `--no-sandbox,--disable-dev-shm-usage` through `AGENT_BROWSER_ARGS`. If you pass your own `--args`, include `--no-sandbox` again or Chrome will not start.

## Recording

Pass `record start` an absolute path (`/tmp/<topic>/raw.webm`). A relative path resolves against the agent-browser daemon's cwd, not yours, and fails with `ffmpeg write failed: Broken pipe` while `doctor` stays green.

Default: record with `--cursor`, drive every supported pointer action (`click`, `mouse move`, `drag`) with `--human`, and ship the **1.5x video**. Keep the 1x original only if a reviewer asks for it. Human pointer movement follows an eased curve; use `--seed <n>` on `mouse move` to make the take reproducible instead of using a bare move.

Open the page first, then record the interaction:

```bash
agent-browser record start ./in.mp4 --cursor
agent-browser snapshot -i
agent-browser mouse move 600 400 --human --seed 42 --duration 1400
agent-browser click @e3 --human          # use a ref from the current snapshot
# Perform the remaining QA steps; use --human for drag actions too.
agent-browser record stop
ffmpeg -y -i in.mp4 -filter:v "setpts=PTS/1.5,fps=30" -an \
  -c:v libx264 -preset veryfast -crf 23 -pix_fmt yuv420p out-1.5x.mp4
```

Speed-up is a post-process step: `record start` has no speed or human flag. Keep `fps=30` to preserve a steady 30 fps output rather than uneven frame drops at an inferred rate. `-an` disables audio because agent-browser recordings carry no audio track. The worker ships only `ffmpeg`, not `ffprobe`; inspect duration with `ffmpeg -i out-1.5x.mp4 2>&1 | grep Duration`.

Add `--contact-sheet` to the start command for a timestamped PNG summary of visual changes. Recording needs `ffmpeg` on PATH; check `agent-browser doctor` before starting. Stop the recording before closing the browser so the video is saved. Keep the 1.5x file on the same agent-fs QA path as screenshots below. In a PR, upload it as a GitHub user attachment and put the URL on its own line, so GitHub plays it inline.

## Step 3: share the screenshot through agent-fs

Screenshots on the worker disk disappear with the task. Upload them to agent-fs under the qa path convention: that copy is the durable original. A person opens it through the live app link `${AGENT_FS_LIVE_URL}/file/~/<org>/<drive>/<path>`. `--file` is binary-safe. `--content` is text-only and mangles PNGs.

```bash
agent-fs write thoughts/<agent-id>/qa/<topic>-screenshots/<name>.png \
  --file /tmp/<name>.png -m "<what it shows>"
agent-fs stat thoughts/<agent-id>/qa/<topic>-screenshots/<name>.png --json        # confirm size > 0
```

- **GitHub PR body or comment**: upload the PNG, GIF or mp4 as a GitHub user attachment with the `github-attach` skill and embed `![caption](https://github.com/user-attachments/assets/<uuid>)`. Attachments never expire and mp4s play inline. A UI PR puts a before (main) and after pair in the `## Before / after` table. Fallback for images only, when the GitHub token is an App token (`ghu_`/`ghs_`): `agent-fs signed-url <path> --inline --json` (expires in 7 days). Never a presigned URL for an mp4 or a doc: it downloads.
- **Linear comment or Slack message**: paste the live app link.
- In `store-progress`, list the upload in the `attachments` field with `kind: "agent-fs"` and the path, and paste the live app link for people who need to open the file.
- Make an `agent-fs share-create <path> --json` link only when the requester asks for one or the reader has no agent-fs login (customer, public post, email, named outsider). Never make one in case the file gets forwarded.
- If `agent-fs auth whoami` fails, report the local path, say the upload was skipped, and continue.

The `agent-fs` skill holds the full agent-fs recipe.

## Gotchas

- `wait <selector>` waits for a VISIBLE match. When the first match is hidden it times out (about 25 s) even though the element exists. Wait on rendered text instead: `wait --text "<rendered text>"`.
- Never run `pkill -f <pattern>` inside a Bash call: the pattern also matches the calling shell's command line and kills it. Record the PID when you start a background server (`echo $!`) and `kill <pid>`.

• SKILL.md 7 KB
• config.json 1 KB
• content.md 6 KB