Przejdź do treści
Biblioteka skilli
Programowanie · Automatyzacja

browser-control

/browser-control

Steruje stronami w istniejącej sesji zdalnego debugowania Chromium za pomocą poleceń sagui lub agenta tylko przeglądarkowego: karty, elementy, nawigacja, klikanie, pisanie, klawisze i przewijanie przez CDP.

Repozytorium
swiftautogui ↗
Licencja
MIT
Ostatnia zmiana repozytorium
18.10.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
2
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
2
Przestarzałość

Brak sygnałów. Repozytorium zmienione 18.10.2026, bez odwołań do wycofanych modeli i API.

Licencja

MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.

Co robi i kiedy

Steruje stronami w istniejącej sesji zdalnego debugowania Chromium za pomocą poleceń sagui lub agenta tylko przeglądarkowego: karty, elementy, nawigacja, klikanie, pisanie, klawisze i przewijanie przez CDP.

Oryginalny opis (EN)

Control web pages in an existing Chromium remote-debugging session with deterministic sagui commands or the browser-only AI Agent. Use when Claude Code needs to inspect tabs, observe semantic elements, navigate, click, type, send keys, scroll, or complete a browser goal through CDP without controlling native macOS UI, Accessibility, or CGEvent input.

Jak działa

  1. Preflight
  2. Connect to Chromium
  3. Choose the domain allowlist
  4. Use deterministic commands
  5. Handle the OpenAI API key
  6. Run a browser-only Agent
  7. Interpret results
  8. Native macOS boundary
Sekcje z nagłówków SKILL.md, w oryginale · 205 linii, 1 087 słów
Podgląd SKILL.md
---
name: browser-control
description: Control web pages in an existing Chromium remote-debugging session with deterministic sagui commands or the browser-only AI Agent. Use when Claude Code needs to inspect tabs, observe semantic elements, navigate, click, type, send keys, scroll, or complete a browser goal through CDP without controlling native macOS UI, Accessibility, or CGEvent input.
---

# browser-control

Use `sagui browser` to inspect and automate Chromium pages through the Chrome DevTools Protocol
(CDP). Treat this as a browser-only environment: it never falls back to native macOS automation.

## Preflight

Run these checks before browser automation:

```bash
uname -s
command -v sagui
sagui browser --help
```

Stop if the platform is not macOS. If `sagui` is missing, or the installed version has no
`browser` command, ask before installing or upgrading it:

```bash
brew update
brew install NakaokaRei/tap/sagui
# Use this instead when sagui is already installed:
brew upgrade NakaokaRei/tap/sagui
```

Do not claim that Accessibility or Screen Recording permission is required for browser-only CDP
actions. Those permissions apply to native `sagui` commands, not `sagui browser`.

## Connect to Chromium

List tabs on the default loopback endpoint:

```bash
sagui browser tabs
```

If no debugging endpoint is available, ask the user to start a dedicated Chromium profile, or
obtain approval before launching it:

```bash
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
  --remote-debugging-address=127.0.0.1 \
  --remote-debugging-port=9222 \
  --user-data-dir=/tmp/swift-auto-gui-browser-profile
```

Keep the endpoint on loopback. Do not expose or connect CDP to a LAN or public address. The browser
session does not launch Chromium, manage profiles, bypass login, solve CAPTCHA, or handle 2FA.

Use the target ID printed by `sagui browser tabs` when a specific tab matters. Re-list tabs after a
tab closes, opens, or navigates unexpectedly because target IDs and page observations may become
stale.

## Choose the domain allowlist

Build the narrowest allowlist that satisfies the user's stated goal:

- `github.com` permits exactly `github.com`.
- `'*.github.com'` permits subdomains but not the parent domain; quote wildcards in the shell.
- Repeat `--domain` for multiple hosts.
- Never use a global wildcard, a public suffix such as `com`, or unrelated domains.
- Ask before adding a domain that is not clearly implied by the goal.

An allowlist entry only makes a destination eligible. Cross-origin navigation is still denied
unless `--allow-cross-origin` is present. Use that flag only when the goal requires navigation from
another origin, such as from `chrome://newtab/` to an allowed website. It never permits navigation
outside the allowlist. Downloads remain denied.

## Use deterministic commands

Prefer direct commands when the requested action is known. They do not call the OpenAI API and do
not require `OPENAI_API_KEY`.

First list tabs and observe the intended tab:

```bash
sagui browser tabs
sagui browser observe --tab-id TARGET_ID
sagui browser observe --tab-id TARGET_ID --screenshot /tmp/page.jpg
```

The observation prints actionable elements as `[#N] role "name"`. Select elements by exact
`--role` and `--name`. When multiple current elements share both values, also pass the observed
`--element-id`; the command verifies all three values against a fresh observation before acting.

```bash
sagui browser click \
  --tab-id TARGET_ID \
  --role link \
  --name "Issues" \
  --domain github.com \
  --allow-cross-origin

sagui browser set-value "SwiftAutoGUI" \
  --tab-id TARGET_ID \
  --role searchbox \
  --name "Search"
```

Other deterministic commands:

```bash
sagui browser activate-tab TARGET_ID

sagui browser open "https://github.com/NakaokaRei/SwiftAutoGUI/issues" \
  --tab-id TARGET_ID \
  --domain github.com \
  --allow-cross-origin

sagui browser type "additional text" --tab-id TARGET_ID
sagui browser key command a --tab-id TARGET_ID
sagui browser key return --tab-id TARGET_ID
sagui browser scroll --vertical -5 --tab-id TARGET_ID
sagui browser scroll --horizontal 3 --tab-id TARGET_ID
```

| Command | Purpose | Required selection |
|---|---|---|
| `tabs` | List page targets | None |
| `observe` | Print tabs and semantic elements | `--tab-id` |
| `activate-tab` | Activate a page target | positional target ID |
| `open` | Navigate to an allowed HTTP(S) URL | URL, `--tab-id`, `--domain` |
| `click` | Click a freshly verified semantic element | `--tab-id`, `--role`, `--name` |
| `set-value` | Replace an editable element value | value, `--tab-id`, `--role`, `--name` |
| `type` | Insert text into the focused element | text, `--tab-id` |
| `key` | Send a key or shortcut | keys, `--tab-id` |
| `scroll` | Scroll the page | non-zero axis, `--tab-id` |

After navigation, click, typing, keys, or scrolling, use the updated semantic map printed by the
command. Do not reuse an old element ID after the page changes. Prefer `set-value` over `type` when
the target textbox is known because `set-value` focuses and verifies the semantic element first.

## Handle the OpenAI API key

The Agent defaults to the on-device Apple model. For `--provider openai`, prefer `OPENAI_API_KEY`; never print the value, request it in chat, or
put it in a command with `--api-key`, where it may enter shell history or process listings.

Check only whether the variable exists:

```bash
if [ -n "${OPENAI_API_KEY:-}" ]; then
  echo "OPENAI_API_KEY is configured"
else
  echo "OPENAI_API_KEY is not configured"
fi
```

When using `--provider openai`, if the key is absent, ask the user to configure it securely in their shell and stop until they confirm. On-device use does not need an API key.

## Run a browser-only Agent

First list tabs, then run a narrowly scoped goal:

```bash
sagui browser tabs

sagui browser agent \
  "Open issue 118 in the NakaokaRei/SwiftAutoGUI repository" \
  --domain github.com \
  --domain '*.github.com' \
  --allow-cross-origin \
  --tab-id TARGET_ID
```

Omit `--tab-id` when the current active page is the intended starting point. Useful optional
controls include:

```bash
--endpoint http://127.0.0.1:9222
--model gpt-5.6-sol
--provider openai
--max-iterations 20
--delay 1.0
--vision-mode automatic
```

Use `automatic` as the normal vision mode. The Agent primarily observes semantic DOM and
Accessibility data and re-observes after navigation, DOM changes, stale elements, or action
failures.

Before goals that submit forms, publish content, purchase items, change permissions, delete data,
or otherwise have meaningful external effects, confirm the final consequential action with the
user unless their request already authorized it explicitly.

## Interpret results

- `unsupportedAction`: The model requested a native-only action. Do not retry it through mouse or
  keyboard fallback.
- `staleElement`: The DOM node, frame, loader, or tab changed. List tabs or run a fresh Agent step;
  never click an old coordinate.
- `navigationNotAllowed`: Add the exact destination host only if it belongs to the user's goal.
- `authorizationDenied`: Cross-origin navigation needs explicit `--allow-cross-origin` approval.
- `disconnected`: Chromium exited or its debugging endpoint closed.

Treat `Completed: true` as the Agent's completion report, then verify any important result from its
printed actions or by listing/observing the relevant tab again.

## Native macOS boundary

This Skill controls page content only. It cannot operate Chromium window chrome, permission
dialogs, file pickers implemented as native macOS UI, Finder, System Settings, or other apps. For
those tasks, use the separate `macos-control` Skill in a distinct phase. Do not represent the two
backends as one Agent run and do not silently fall back from CDP to Accessibility or CGEvent.
Pliki w paczce (2)
  • SKILL.md 8 KB
  • agents/openai.yaml 1 KB

Wersje

RepozytoriumWynikLicencja
Ta strona · nakaokarei/swiftautogui100MIT
ghostyshell/browser-bridge95MIT
anomalyco/browser-control95MIT
inlevel9-com/BrowserControl95Apache-2.0

Migawka bazy z 13.11.2026 · wersja paczki f3890586b415 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 91agent-smoke-test
    Uruchamia ręczne testy działań agenta SwiftAutoGUI przez CLI sagui na macOS, przy sterowaniu aplikacjami i Accessibility.
  • 100web-browser
    Automatyzuje strony internetowe w Chrome lub Chromium przez Chrome DevTools Protocol: nawigacja, klikanie, formularze, inspekcja treści, zrzuty ekranu i debugowanie konsoli oraz sieci, najlepiej w trybie headless.
  • 100browser
    Steruje przeglądarką przez CDP (nawigacja, klikanie, pisanie, zrzuty ekranu), a profile zachowują stan logowania między sesjami, więc agent loguje się raz.
  • 95browser
    Steruje już działającym Chrome przez CDP: scrapuje, testuje i wchodzi w interakcje ze stronami.
  • 100web-browse
    Steruje Chromium krok po kroku z wiersza poleceń: otwiera strony, klika, wpisuje tekst i robi zrzuty ekranu.
  • 100browser-use
    Obsługuje zadania w przeglądarce: widoczny Chrome przez CDP, dedykowany headless CDP lub agent-browser, z ponownym użyciem istniejących sesji logowania i dostępem do stron wewnętrznych oraz dynamicznych.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.