browser-automation
/browser-automation
Steruje stronami przez narzędzie przeglądarki Omnipolymath: przepływy wieloetapowe, logowanie, karty i odzyskiwanie po przestarzałych referencjach.
- Autor
- krishnashahane ↗
- Repozytorium
- Omnipolymath ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 13.11.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 2
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 2
Brak sygnałów. Repozytorium zmienione 13.11.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
Co robi i kiedy
Steruje stronami przez narzędzie przeglądarki Omnipolymath: przepływy wieloetapowe, logowanie, karty i odzyskiwanie po przestarzałych referencjach.
Oryginalny opis (EN)
Use when controlling web pages with the Omnipolymath browser tool, especially multi-step flows, login checks, tab management, or recovery from stale refs/timeouts.
Jak działa
- Operating Loop
- Browser batch CLI
- Code Mode Loop
- Tab Hygiene
- Stale Ref Recovery
- Existing User Browser
- Google Meet Notes
Podgląd SKILL.md
---
name: browser-automation
description: Use when controlling web pages with the Omnipolymath browser tool, especially multi-step flows, login checks, tab management, or recovery from stale refs/timeouts.
user-invocable: false
---
# Browser Automation
Use this skill when you need the `browser` tool for anything beyond a single page check.
## Operating Loop
1. Check browser state before acting:
- `omnipolymath browser doctor` or `action="status"` when the browser/plugin setup itself may be broken.
- `action="status"` for availability.
- `action="profiles"` if login state or profile choice matters.
- `action="tabs"` before opening a new tab if retries/timeouts may have left windows behind.
2. Prefer stable tab handles:
- Open important tabs with `label`, for example `label="meet"`.
- After `action="tabs"` or `action="open"`, store `suggestedTargetId` and pass it as `targetId` in later calls.
- `suggestedTargetId` is the label when one exists, otherwise the stable `tabId` handle like `t1`.
- Avoid relying on raw DevTools `targetId` except for immediate diagnostics; it can change under Chromium target replacement.
3. Read before you click:
- For “read the page and answer X,” use `action="text"` with optional `selector` and `maxChars` for bounded visible prose (first selector match, otherwise article/main/body). On existing-session profiles, use `snapshot` instead. Efficient snapshots omit most prose.
- For virtualized lists, scroll through each segment, capture only the relevant rows, then merge the results.
- Use `action="snapshot"` on the intended `targetId`.
- Add snapshot `query` to find lines containing all query tokens, ignoring case; matching lines keep their refs.
- Use the same `targetId` for follow-up actions so refs stay on the same tab.
- For durable Playwright refs, request `refs="aria"` when supported. If you receive `axN` refs from `snapshotFormat="aria"`, use them only after that same snapshot call; stale or unbound `axN` refs fail fast and need a fresh snapshot.
- Use `urls=true` when link text is ambiguous or a direct navigation target would avoid brittle clicks.
- Use `labels=true` on snapshot or screenshot when visual position matters. On Playwright-backed profiles, the response includes an `annotations` array (`{ref, number, role, name?, box}`) with each ref's bounding box in the captured image's coordinate space, so you can reason about position without re-snapshotting; screenshot labels can also combine with `fullPage=true` (CLI: `--full-page`) to label the whole document, or `ref` / `element` to clip to one element. `profile="user"` and other existing-session (chrome-mcp) profiles render an overlay into page screenshots but do not attach `annotations` or use the Playwright full-page/ref/element projection helper, so read positions from the labeled image itself on those profiles. The raw-CDP fallback (no Playwright) does not support labeled screenshots at all and returns a 501, so only request `labels` when Playwright is available.
4. Act narrowly:
- Prefer `action="act"` with a ref from the latest snapshot.
- `navigate` returns the loaded page's compact snapshot inline, and batch `act` results that report a cross-document navigation include fresh page state; use those refs directly instead of a follow-up snapshot call.
- After a single act that triggers navigation, and after modal changes or form submissions, snapshot again before the next action.
- Avoid blind waits. Wait for visible UI state when possible.
- Use `action="emulate"` with `device`, `colorScheme`, `timezoneId`, or `locale` when testing those settings; snapshot again afterward. Existing-session profiles do not support emulation.
5. Report real blockers:
- Debug network failures with `action="requests"`, optional URL/type `filter`, and `limit` (default 50 recent entries). `clear=true` clears the collected log after reading. Use a managed profile; existing-session profiles do not support this log.
- Debug page errors with `action="errors"` and `limit` (default 50 recent entries). `clear=true` clears the collected log after reading. Existing-session profiles do not support this log.
- If the page needs login, permission, captcha, 2FA, camera/microphone approval, or another manual step, stop and tell the user exactly what is needed.
- Do not claim the browser is not logged in just because the current page shows a permission or onboarding dialog. Inspect the visible UI first.
## Browser batch CLI
`omnipolymath browser batch` runs an array of nested `/act` actions in one `/act` call (the same `kind="batch"` runtime reached through the agent tool), so CLI users and scripts can combine actions like `wait`, `click`, `type`, and `evaluate` into a single replayable plan without per-action round trips. Each entry in `actions[]` is a `BrowserActRequest` — the closed union the `/act` route accepts — not arbitrary `omnipolymath browser` subcommands. `batch` is not supported on `profile="user"` and other existing-session (chrome-mcp) profiles; send actions individually there.
- CLI: `omnipolymath browser batch --actions '<json>'`, `--actions-file plan.json`, or `--actions-file -` for stdin. `--actions-file` and stdin input are capped at 1,000,000 bytes; split larger plans into multiple batch commands. `--continue` sets `stopOnError=false`; default stops on first error.
- Ref lifecycle: refs come from a `snapshot` run before the batch (snapshot is not a nested action). A nested action that changes page state — such as a `click` that triggers navigation, or an `evaluate` that mutates the DOM — can invalidate earlier refs for the rest of the batch; put state-changing actions first, or split into a follow-up batch after re-snapshotting. Navigation and re-snapshotting happen outside the batch, since `open`, `navigate`, and `snapshot` are not `/act` kinds.
- Target id: nested actions share the request's tab; an explicit nested `targetId` that resolves to a different tab is rejected with `ACT_TARGET_ID_MISMATCH`.
- Response: `{ "results": [{ "ok": true } | { "ok": false, "error": "..." }, ...] }` in order; with default `stopOnError` the array ends at the first failure. Any failed entry exits nonzero; use `--json` to preserve the full response in scripts.
## Code Mode Loop
For explicit page-blocker triage with a configured Decision model, read
[Page triage](references/page-triage.md). Browser `action="judge"` classifies a
bounded observation through the selected Decision provider and returns answers,
not page text, to the primary model. It works on text-capable Chromium and
Lightpanda profiles; it is not Lightpanda native `judge()`.
When `tools.codeMode` is enabled, the Browser tool has no normal turn — it is cataloged behind `exec`/`wait`. Call it from exec cells as an async global, using the callable name the exec quick index advertises for the Browser tool (normally `browser`; colliding names get suffixed, and a client tool can win an identical name). An exact `catalog.search("browser")` returns a handle already bound to the effective callable name, so resolve the handle in each cell and call it instead of hard-coding the literal global; an empty result means the Browser tool is not cataloged in this run.
Keep the same labeled tab through the loop, and alternate reads with actions. Each `exec` cell starts a fresh VM — bindings from a completed cell are gone in the next, and only runs left `waiting` keep their state until `wait` resumes them — so carry comparison state across cells by returning it and re-embedding the returned values in the next cell:
```javascript
// previous = the url/newElements returned by the last completed cell (a fresh
// VM runs this cell, so prior bindings do not exist here).
const previous = { url: "https://example.com/inbox", newElements: 0 };
const [browser] = await catalog.search("browser", { limit: 1 });
const details = await browser({
action: "snapshot",
snapshotFormat: "ai",
targetId: "task",
refs: "aria",
interactive: true,
});
const changed =
details?.url !== previous.url ||
(details?.newElements ?? 0) > 0 ||
details?.blockedByDialog === true;
return {
targetId: details?.targetId,
url: details?.url,
newElements: details?.newElements,
stats: details?.stats,
text: details?.text,
changed,
};
```
- Code-mode calls return the tool's structured `details` directly. Snapshot and text observations include bounded `text` with the same untrusted-content wrapper as direct tool output. Keep that wrapper when returning page data to the model. Existing identity, `stats`, and dialog fields remain available.
- To read visible prose inside code mode, request `text` with a narrow selector. It does not require the evaluate capability:
```javascript
const [browser] = await catalog.search("browser", { limit: 1 });
const read = await browser({
action: "text",
selector: "main",
maxChars: 2000,
targetId: "task",
});
return { url: read?.url, text: read?.text };
```
Use snapshot `query` and text `selector`/`maxChars` to narrow reads at the Browser owner before wrapping. A blocked dialog still needs its reported recovery step; page text does not grant permission to evaluate scripts.
- Return only the fields the next step needs; never return the whole details object.
- Completed cells share no state: re-embed the previous cell's returned `url`/`newElements` in the next cell, or keep the comparison inside one cell. Only `waiting` runs persist, resumed by `wait`.
- Interleave each act with a URL or tabs check before the next dependent act.
- If a batch returns `aborted`, take a fresh snapshot before continuing.
- If `newElements` is positive, inspect those elements first, then update the re-embedded state.
- Use separate act calls when navigation is expected between steps.
## Tab Hygiene
Before creating a tab for a named task, list tabs and reuse an existing matching label or URL when it is still usable.
Example:
```json
{ "action": "tabs" }
```
If no suitable tab exists:
```json
{ "action": "open", "url": "https://example.com", "label": "task" }
```
Then target it by label:
```json
{ "action": "snapshot", "targetId": "task", "refs": "aria" }
```
If a retry creates duplicates, close the extras by `tabId`:
```json
{ "action": "close", "targetId": "t3" }
```
Do not pass bare numbers like `"2"` as `targetId`. Numeric tab positions are only for the CLI `omnipolymath browser tab select 2` helper; browser tool calls need a `suggestedTargetId`, label, `tabId`, or raw target id.
## Stale Ref Recovery
If an action fails with a missing or stale ref:
1. Snapshot the same `targetId` again.
2. Find the current visible control.
3. Retry once with the new ref.
4. If the UI moved to a blocker state, report the blocker instead of looping.
## Existing User Browser
Use `profile="user"` only when existing cookies/login matter. This attaches to the user's running Chromium-based browser.
On macOS, `action="importprofile"` is the alternative when the agent should use an isolated managed browser with cookies copied from a real Chrome-family profile. First use `action="profiles"` and inspect `systemProfiles`, then import into a fresh managed profile name. Import asks for one Keychain/Touch ID consent prompt. It copies cookies, not local storage or IndexedDB; device-bound session credentials (DBSC) mean some Google sessions may still require re-authentication.
For `profile="user"` and other existing-session profiles, omit `timeoutMs` on `act:type`, `hover`, `scrollIntoView`, `drag`, `select`, and `fill`; that driver rejects per-call timeout overrides for those actions. `act:evaluate` accepts `timeoutMs`.
## Google Meet Notes
When creating or joining a Meet:
- Treat camera/microphone permission screens as progress, not login failure.
- If asked whether people can hear you, click the microphone option when voice is required.
- If Google asks for sign-in, 2FA, account chooser confirmation, or permission that needs user approval, report the exact manual action.
- Use one labeled tab per meeting flow, for example `label="meet"`, and reuse it during retries.
Pliki w paczce (2)
- SKILL.md 12 KB
- references/page-triage.md 4 KB
Wersje
| Repozytorium | Wynik | Licencja |
|---|---|---|
| Ta strona · krishnashahane/Omnipolymath | 100 | MIT |
| steipete/clawdis | 100 | MIT |
| dengruihan/AgileClaw | 100 | Apache-2.0 |
| openclaw/openclaw | 100 | MIT |
| coco-research/coco | 100 | MIT |
| Sanit-mann/The-Insectoids | 100 | MIT |
| alexei-led/cc-thingz | 100 | MIT |
| different-ai/openwork | 100 | MIT |
| dreamlab-ai/agentbox | 100 | AGPL-3.0 |
Migawka bazy z 13.11.2026 · wersja paczki d974ebe3f480 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100agent-browser
Steruje autoryzowanymi stronami webowymi przez CLI agent-browser, korzystając z migawek dostępności i referencji do elementów oraz weryfikując przepływy w przeglądarce. - 100nodriver-browser
Automatyzuje przeglądarkę Chrome lub Chromium z trwałą sesją przez nodriver: renderowanie JavaScript, autoryzowane logowanie, klikanie, wpisywanie, migawki DOM, zrzuty ekranu i wieloetapowe przepływy. - 100release-omnipolymath-announcement
Przygotowuje ogłoszenie wydania Omnipolymath na Discorda na podstawie changelogu, tagu i wyników walidacji. - 100browser
Automatyzuje przeglądarkę: klikanie przycisków, formularze, wieloetapowe przepływy, wyciąganie danych ze stron renderowanych w JS, logowanie, nagrywanie interakcji i testowanie aplikacji webowych. - 100agent-browser
Obsługuje CLI do automatyzacji przeglądarki dla agentów AI ze zwartym wyjściem tekstowym opartym na referencjach (200-400 tokenów zamiast 3000-5000) i ponad 50 poleceniami: nawigacja, scraping, formularze, zrzuty i testy. - 100agent-browser
Obsługuje CLI do automatyzacji przeglądarki dla agentów AI ze zwartym wyjściem tekstowym opartym na referencjach (200-400 tokenów zamiast 3000-5000) i ponad 50 poleceniami: nawigacja, scraping, formularze, zrzuty i testy.