canvas
/canvas
Steruje interfejsami canvas, WebGL i gier bez DOM i węzłów dostępności, jak plansze gier, mapy, przeglądarki 3D i HUD: przechwytuje obraz, klika po współrzędnych i steruje w czasie rzeczywistym.
- Autor
- leeguooooo ↗
- Repozytorium
- chrome-use ↗
- Licencja
- Apache-2.0
- Ostatnia zmiana repozytorium
- 27.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 1
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 27.10.2026, bez odwołań do wycofanych modeli i API.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
Co robi i kiedy
Steruje interfejsami canvas, WebGL i gier bez DOM i węzłów dostępności, jak plansze gier, mapy, przeglądarki 3D i HUD: przechwytuje obraz, klika po współrzędnych i steruje w czasie rzeczywistym.
Oryginalny opis (EN)
Drive canvas / WebGL / game UIs that paint to a <canvas> with no DOM or accessibility node — game boards, voice-room mic seats, map tiles, 3D viewers, design canvases (Figma/mockitt), HUDs. Use when snapshot/find/eval querySelector return no ref for something clearly on screen, when the label shows in `get text` but has no @ref, or when you need real-time coordinate/keyboard control. Covers canvas capture, coordinate clicks, hold-to-move, batch timed sequences, reading engine globals, and the real-time WebSocket stream.
Jak działa
- Canvas / WebGL apps (games, map & 3D viewers, drawing tools)
Podgląd SKILL.md
---
name: canvas
description: Drive canvas / WebGL / game UIs that paint to a <canvas> with no DOM or accessibility node — game boards, voice-room mic seats, map tiles, 3D viewers, design canvases (Figma/mockitt), HUDs. Use when snapshot/find/eval querySelector return no ref for something clearly on screen, when the label shows in `get text` but has no @ref, or when you need real-time coordinate/keyboard control. Covers canvas capture, coordinate clicks, hold-to-move, batch timed sequences, reading engine globals, and the real-time WebSocket stream.
allowed-tools: Bash(chrome-use:*), Bash(npx chrome-use:*)
---
# chrome-use canvas / WebGL / games
**Canvas / WebGL UIs (game boards, voice-room mic seats, map tiles, design
canvases).** These paint to a `<canvas>` — there is **no DOM node and no
accessibility node** behind what you see, so `snapshot`/`find`/`eval
querySelector` will never return a ref for them. This is a hard limitation, not a
missing feature. To work with them:
- **Read** the rendered pixels with `chrome-use canvas list` then `chrome-use
canvas capture [selector] <file>` (extracts the canvas bitmap), or a normal
`screenshot` of the region — then *you* interpret it.
- **Act** by coordinate: compute the target point and `chrome-use click <x> <y>`
(or `box @ref` on a container to get its CSS-px box first). Coordinates are the
*correct* tool here — the snapshot-first rule explicitly carves out canvas.
- On the **relay**, a coordinate click can drift onto the user's foreground tab;
prefer a `--launch`/owned tab for heavy canvas coordinate work, or confirm the
underlying state via the app's backend/API instead of driving the canvas.
## Canvas / WebGL apps (games, map & 3D viewers, drawing tools)
These paint everything to a `<canvas>` and expose **almost no accessibility
tree**, so `snapshot` comes back near-empty and refs are a dead end. `snapshot`
detects this, prints a one-line hint, and also saves a viewport screenshot and
prints `screenshot: <path>`. View that image instead of calling `screenshot`
again (JSON: `data.screenshot`, `screenshotReason: "sparse"`; opt out with
`AGENT_BROWSER_SPARSE_SCREENSHOT=0`). Drive them the screenshot way:
```bash
chrome-use canvas list # enumerate <canvas> elements (size, type)
chrome-use canvas capture out.png # save the canvas's RENDERED pixels to PNG —
# toDataURL (full backing-store res, e.g.
# Figma 2522x1904), screenshot fallback for
# WebGL w/o preserveDrawingBuffer / tainted.
# Gets the RENDER, not hidden source data
# (those live in the app's binary store/API).
chrome-use screenshot /tmp/s.png # SEE the state (your only read path —
# eval/get text return nothing useful)
chrome-use click 640 360 # interact by viewport coordinate
chrome-use press d --hold 800 # hold-to-move, precise (timed in-daemon —
# NOT keydown+shell-sleep+keyup, which
# adds ~250ms jitter per round-trip)
chrome-use press Space # discrete actions (jump/attack/confirm)
```
**Symptom: the label shows in `get text` but has no `@ref`.** Voice-room mic-seats
(Zego/Agora), prototype canvases (mockitt/modao), game HUDs, and some web
components paint their controls, so the text appears in `get text`/`read_page`
("Add Add Add…") yet `snapshot -i` lists nothing and `querySelectorAll` returns 0
— there is no addressable node, so `@ref`/`find` can't reach it. Drive by position:
```bash
chrome-use get text --pierce # FIRST: if it's a CLOSED shadow root (not
# canvas), this reads through it — cheap to try
chrome-use screenshot /tmp/s.png # else SEE where the control sits
chrome-use click <x> <y> # click the pixel (bare numbers = coordinate)
```
Why there's no ref: `<canvas>`/WebGL hit-regions and **closed** shadow roots expose
no DOM/AX node for the painted control, so no amount of snapshot work can mint a
ref — coordinates are the only handle. (Open shadow roots and same-origin /
cross-origin iframes ARE surfaced by `snapshot -i`; only canvas + closed-shadow are
coordinate-only.) On the relay, foreground the agent's own tab first so the
coordinate click can't drift onto the user's other tab.
**Don't drive frame-by-frame with one CLI call per action** — that's the slowest,
lowest-fidelity way (each call is a process spawn + round-trip). Script a *timed
sequence in a single round-trip* with `batch` (it sends each step to the running
daemon; `press --hold` and `wait` block in-daemon, so timing is precise):
```bash
chrome-use batch "press d --hold 900" "press j" "press j" "wait 200" "press d --hold 500"
```
Also try reading real state instead of pixels: `eval` runs in the page's main
world, so for a framework/engine game you can often reach its globals (e.g. a
Phaser/PIXI/Three instance, a store, `window.__GAME__`) and read positions/score
directly — far better than guessing from a screenshot.
**For genuinely real-time driving, drop the CLI entirely and use the WebSocket.**
`chrome-use stream enable` opens a bidirectional WS (`stream status` prints the
`ws://127.0.0.1:<port>`). Connect once and you get a live ~60fps screencast AND
can send input on the same socket — no per-action process spawn, no round-trip,
works over the extension relay:
```js
// node (global WebSocket): live frames + locally-timed input
const ws = new WebSocket("ws://127.0.0.1:PORT")
ws.onmessage = e => { const m = JSON.parse(e.data); if (m.type==="frame") {/* base64 jpeg */} }
const k = (eventType,key,code,vk) => ws.send(JSON.stringify({type:"input_keyboard",eventType,key,code,windowsVirtualKeyCode:vk}))
k("keyDown"," ","Space",32); setTimeout(()=>k("keyUp"," ","Space",32), 80) // a jump
// also: {type:"input_mouse",eventType:"mousePressed",x,y,button:"left",clickCount:1}
```
This is the difference between watching a slideshow and playing the game. Reserve
screenshots for one-off checks; use the WS for any sustained real-time control.
Wersje
| Repozytorium | Wynik | Licencja |
|---|---|---|
| Ta strona · leeguooooo/chrome-use | 100 | Apache-2.0 |
| nimbalyst/nimbalyst | 100 | MIT |
| bitterbot-ai/bitterbot-desktop | 100 | MIT |
| fogelman/local-design-canvas | 100 | MIT |
| kensaurus/cursor-kenji | 100 | MIT |
| agricidaniel/claude-obsidian | 100 | MIT |
| kansoku-trade/kansoku | 100 | AGPL-3.0 |
| spinabot/brigade | 100 | MIT |
| grtninja/skill-arbiter | 100 | MIT |
Migawka bazy z 13.11.2026 · wersja paczki af9b209b54b3 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100web-games
Prowadzi tworzenie gier przeglądarkowych: wybór frameworka (Phaser, PixiJS, Kaplay, Canvas, WebGL, Three.js, Babylon.js), hybrydowy DOM z canvas, WebGPU, optymalizacja, PWA i odblokowanie audio. - 100web-games
Prowadzi tworzenie gier przeglądarkowych: wybór frameworka (Phaser, PixiJS, Kaplay, Canvas, WebGL, Three.js, Babylon.js), hybrydowy DOM z canvas, WebGPU, optymalizacja, PWA i odblokowanie audio. - 100fec-canvas-threejs
Wspiera budowę i przegląd grafiki w Canvas 2D, Three.js, WebGL, React Three Fiber i shaderów GLSL, wizualizacji, renderowania gier, pętli animacji, zwalniania zasobów GPU i wydajności. - 100game-development
Kieruje wyborem platformy, wymiaru i silnika w projekcie gry (web 2D i 3D, DOM z canvas, narzędzia narracyjne) oraz dobiera spośród Phaser, PixiJS, Three.js, Godot, Unity czy Twine. - 100game-development
Kieruje wyborem platformy, wymiaru i silnika w projekcie gry (web 2D i 3D, DOM z canvas, narzędzia narracyjne) oraz dobiera spośród Phaser, PixiJS, Three.js, Godot, Unity czy Twine. - 100game-engine
Wspiera budowę silników i gier webowych w HTML5, Canvas, WebGL i JavaScript: pętle gry, fizyka, kolizje, sprite'y, sterowanie, dźwięk, multiplayer przez WebRTC i publikacja.