browser-control
/browser-control
Steruje stronami w istniejącej sesji zdalnego debugowania Chromium za pomocą poleceń sagui lub agenta tylko przeglądarkowego: karty, elementy, nawigacja, klikanie, pisanie, klawisze i przewijanie przez CDP.
- Autor
- nakaokarei ↗
- Repozytorium
- swiftautogui ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 18.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 2
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 2
Brak sygnałów. Repozytorium zmienione 18.10.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
Co robi i kiedy
Steruje stronami w istniejącej sesji zdalnego debugowania Chromium za pomocą poleceń sagui lub agenta tylko przeglądarkowego: karty, elementy, nawigacja, klikanie, pisanie, klawisze i przewijanie przez CDP.
Oryginalny opis (EN)
Control web pages in an existing Chromium remote-debugging session with deterministic sagui commands or the browser-only AI Agent. Use when Claude Code needs to inspect tabs, observe semantic elements, navigate, click, type, send keys, scroll, or complete a browser goal through CDP without controlling native macOS UI, Accessibility, or CGEvent input.
Jak działa
- Preflight
- Connect to Chromium
- Choose the domain allowlist
- Use deterministic commands
- Handle the OpenAI API key
- Run a browser-only Agent
- Interpret results
- Native macOS boundary
Podgląd SKILL.md
---
name: browser-control
description: Control web pages in an existing Chromium remote-debugging session with deterministic sagui commands or the browser-only AI Agent. Use when Claude Code needs to inspect tabs, observe semantic elements, navigate, click, type, send keys, scroll, or complete a browser goal through CDP without controlling native macOS UI, Accessibility, or CGEvent input.
---
# browser-control
Use `sagui browser` to inspect and automate Chromium pages through the Chrome DevTools Protocol
(CDP). Treat this as a browser-only environment: it never falls back to native macOS automation.
## Preflight
Run these checks before browser automation:
```bash
uname -s
command -v sagui
sagui browser --help
```
Stop if the platform is not macOS. If `sagui` is missing, or the installed version has no
`browser` command, ask before installing or upgrading it:
```bash
brew update
brew install NakaokaRei/tap/sagui
# Use this instead when sagui is already installed:
brew upgrade NakaokaRei/tap/sagui
```
Do not claim that Accessibility or Screen Recording permission is required for browser-only CDP
actions. Those permissions apply to native `sagui` commands, not `sagui browser`.
## Connect to Chromium
List tabs on the default loopback endpoint:
```bash
sagui browser tabs
```
If no debugging endpoint is available, ask the user to start a dedicated Chromium profile, or
obtain approval before launching it:
```bash
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
--remote-debugging-address=127.0.0.1 \
--remote-debugging-port=9222 \
--user-data-dir=/tmp/swift-auto-gui-browser-profile
```
Keep the endpoint on loopback. Do not expose or connect CDP to a LAN or public address. The browser
session does not launch Chromium, manage profiles, bypass login, solve CAPTCHA, or handle 2FA.
Use the target ID printed by `sagui browser tabs` when a specific tab matters. Re-list tabs after a
tab closes, opens, or navigates unexpectedly because target IDs and page observations may become
stale.
## Choose the domain allowlist
Build the narrowest allowlist that satisfies the user's stated goal:
- `github.com` permits exactly `github.com`.
- `'*.github.com'` permits subdomains but not the parent domain; quote wildcards in the shell.
- Repeat `--domain` for multiple hosts.
- Never use a global wildcard, a public suffix such as `com`, or unrelated domains.
- Ask before adding a domain that is not clearly implied by the goal.
An allowlist entry only makes a destination eligible. Cross-origin navigation is still denied
unless `--allow-cross-origin` is present. Use that flag only when the goal requires navigation from
another origin, such as from `chrome://newtab/` to an allowed website. It never permits navigation
outside the allowlist. Downloads remain denied.
## Use deterministic commands
Prefer direct commands when the requested action is known. They do not call the OpenAI API and do
not require `OPENAI_API_KEY`.
First list tabs and observe the intended tab:
```bash
sagui browser tabs
sagui browser observe --tab-id TARGET_ID
sagui browser observe --tab-id TARGET_ID --screenshot /tmp/page.jpg
```
The observation prints actionable elements as `[#N] role "name"`. Select elements by exact
`--role` and `--name`. When multiple current elements share both values, also pass the observed
`--element-id`; the command verifies all three values against a fresh observation before acting.
```bash
sagui browser click \
--tab-id TARGET_ID \
--role link \
--name "Issues" \
--domain github.com \
--allow-cross-origin
sagui browser set-value "SwiftAutoGUI" \
--tab-id TARGET_ID \
--role searchbox \
--name "Search"
```
Other deterministic commands:
```bash
sagui browser activate-tab TARGET_ID
sagui browser open "https://github.com/NakaokaRei/SwiftAutoGUI/issues" \
--tab-id TARGET_ID \
--domain github.com \
--allow-cross-origin
sagui browser type "additional text" --tab-id TARGET_ID
sagui browser key command a --tab-id TARGET_ID
sagui browser key return --tab-id TARGET_ID
sagui browser scroll --vertical -5 --tab-id TARGET_ID
sagui browser scroll --horizontal 3 --tab-id TARGET_ID
```
| Command | Purpose | Required selection |
|---|---|---|
| `tabs` | List page targets | None |
| `observe` | Print tabs and semantic elements | `--tab-id` |
| `activate-tab` | Activate a page target | positional target ID |
| `open` | Navigate to an allowed HTTP(S) URL | URL, `--tab-id`, `--domain` |
| `click` | Click a freshly verified semantic element | `--tab-id`, `--role`, `--name` |
| `set-value` | Replace an editable element value | value, `--tab-id`, `--role`, `--name` |
| `type` | Insert text into the focused element | text, `--tab-id` |
| `key` | Send a key or shortcut | keys, `--tab-id` |
| `scroll` | Scroll the page | non-zero axis, `--tab-id` |
After navigation, click, typing, keys, or scrolling, use the updated semantic map printed by the
command. Do not reuse an old element ID after the page changes. Prefer `set-value` over `type` when
the target textbox is known because `set-value` focuses and verifies the semantic element first.
## Handle the OpenAI API key
The Agent defaults to the on-device Apple model. For `--provider openai`, prefer `OPENAI_API_KEY`; never print the value, request it in chat, or
put it in a command with `--api-key`, where it may enter shell history or process listings.
Check only whether the variable exists:
```bash
if [ -n "${OPENAI_API_KEY:-}" ]; then
echo "OPENAI_API_KEY is configured"
else
echo "OPENAI_API_KEY is not configured"
fi
```
When using `--provider openai`, if the key is absent, ask the user to configure it securely in their shell and stop until they confirm. On-device use does not need an API key.
## Run a browser-only Agent
First list tabs, then run a narrowly scoped goal:
```bash
sagui browser tabs
sagui browser agent \
"Open issue 118 in the NakaokaRei/SwiftAutoGUI repository" \
--domain github.com \
--domain '*.github.com' \
--allow-cross-origin \
--tab-id TARGET_ID
```
Omit `--tab-id` when the current active page is the intended starting point. Useful optional
controls include:
```bash
--endpoint http://127.0.0.1:9222
--model gpt-5.6-sol
--provider openai
--max-iterations 20
--delay 1.0
--vision-mode automatic
```
Use `automatic` as the normal vision mode. The Agent primarily observes semantic DOM and
Accessibility data and re-observes after navigation, DOM changes, stale elements, or action
failures.
Before goals that submit forms, publish content, purchase items, change permissions, delete data,
or otherwise have meaningful external effects, confirm the final consequential action with the
user unless their request already authorized it explicitly.
## Interpret results
- `unsupportedAction`: The model requested a native-only action. Do not retry it through mouse or
keyboard fallback.
- `staleElement`: The DOM node, frame, loader, or tab changed. List tabs or run a fresh Agent step;
never click an old coordinate.
- `navigationNotAllowed`: Add the exact destination host only if it belongs to the user's goal.
- `authorizationDenied`: Cross-origin navigation needs explicit `--allow-cross-origin` approval.
- `disconnected`: Chromium exited or its debugging endpoint closed.
Treat `Completed: true` as the Agent's completion report, then verify any important result from its
printed actions or by listing/observing the relevant tab again.
## Native macOS boundary
This Skill controls page content only. It cannot operate Chromium window chrome, permission
dialogs, file pickers implemented as native macOS UI, Finder, System Settings, or other apps. For
those tasks, use the separate `macos-control` Skill in a distinct phase. Do not represent the two
backends as one Agent run and do not silently fall back from CDP to Accessibility or CGEvent.
Pliki w paczce (2)
- SKILL.md 8 KB
- agents/openai.yaml 1 KB
Wersje
| Repozytorium | Wynik | Licencja |
|---|---|---|
| Ta strona · nakaokarei/swiftautogui | 100 | MIT |
| ghostyshell/browser-bridge | 95 | MIT |
| anomalyco/browser-control | 95 | MIT |
| inlevel9-com/BrowserControl | 95 | Apache-2.0 |
Migawka bazy z 13.11.2026 · wersja paczki f3890586b415 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 91agent-smoke-test
Uruchamia ręczne testy działań agenta SwiftAutoGUI przez CLI sagui na macOS, przy sterowaniu aplikacjami i Accessibility. - 100web-browser
Automatyzuje strony internetowe w Chrome lub Chromium przez Chrome DevTools Protocol: nawigacja, klikanie, formularze, inspekcja treści, zrzuty ekranu i debugowanie konsoli oraz sieci, najlepiej w trybie headless. - 100browser
Steruje przeglądarką przez CDP (nawigacja, klikanie, pisanie, zrzuty ekranu), a profile zachowują stan logowania między sesjami, więc agent loguje się raz. - 95browser
Steruje już działającym Chrome przez CDP: scrapuje, testuje i wchodzi w interakcje ze stronami. - 100web-browse
Steruje Chromium krok po kroku z wiersza poleceń: otwiera strony, klika, wpisuje tekst i robi zrzuty ekranu. - 100browser-use
Obsługuje zadania w przeglądarce: widoczny Chrome przez CDP, dedykowany headless CDP lub agent-browser, z ponownym użyciem istniejących sesji logowania i dostępem do stron wewnętrznych oraz dynamicznych.