vision-tools
/vision-tools
Obsługuje zrzuty ekranu i diagramy: rekonstrukcję, odczyt długich zrzutów przez OCR, przycinanie, różnice pikseli i śledzenie SVG.
- Autor
- saifymatteo ↗
- Repozytorium
- pi-extensions ↗
- Licencja
- AGPL-3.0
- Ostatnia zmiana repozytorium
- 11.11.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 23
- Skrypty
- 8
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 1
Brak sygnałów. Repozytorium zmienione 11.11.2026, bez odwołań do wycofanych modeli i API.
AGPL-3.0, copyleft: przeróbki udostępniane innym muszą mieć tę samą licencję.
- 8 skryptów w paczce. Przejrzyj je przed instalacją, bo Claude może je uruchomić.
Skrypty: detect.py, ground.py, scripts/dominant_colors.py, scripts/extract_fg.py, scripts/html_shot.py, scripts/long_screenshot_ocr.py, scripts/pixel_diff.py, vision_client.py
Co robi i kiedy
Obsługuje zrzuty ekranu i diagramy: rekonstrukcję, odczyt długich zrzutów przez OCR, przycinanie, różnice pikseli i śledzenie SVG.
Oryginalny opis (EN)
Native-first workflows for screenshot reconstruction, diagram extraction, screenshot-driven GUI work, long-screenshot OCR, and image processing (crops, pixel differences, palette measurements, SVG tracing, foreground extraction). Use when a task needs these workflows or concrete image artifacts; ordinary image questions can be answered with native vision.
Jak działa
- Workflow
- Run the packaged scripts
- Local tools: use when their output is needed
- Task playbooks: load only the matching one
- Optional model-backed CLIs
- Image lifetime
Podgląd SKILL.md
---
name: vision-tools
description: >-
Native-first workflows for screenshot reconstruction, diagram extraction,
screenshot-driven GUI work, long-screenshot OCR, and image processing
(crops, pixel differences, palette measurements, SVG tracing, foreground
extraction). Use when a task needs these workflows or concrete image
artifacts; ordinary image questions can be answered with native vision.
---
# Vision tools
Use the main model's native vision first. The tools here supplement it with
image artifacts, measurements, or an optional separate-model opinion; they
are not a required route for seeing images.
## Workflow
1. Inspect the image natively and identify what the task requires. If you only
have a description and need a missing detail, open the original image path.
If the file is gone, report the missing evidence.
2. Zoom only when a relevant detail is unresolved. Crop that region and read
the crop natively; use `ground` only if its estimated box would help locate
the target. Stop inspecting once you have evidence for the requested answer.
3. Use local image processing for numerical claims or reusable artifacts.
Treat model-generated boxes as estimates, not measured boundaries; verify
their target and coordinate space before acting on them.
4. Verify the result against the task: readable text for OCR, nodes and edges
for diagrams, the expected UI state after an action, or a rendered comparison
for visual reconstruction. A low pixel-diff score alone is not correctness.
For browser or accessible GUI work, prefer DOM/accessibility information for
controls and state. Use screenshot coordinates when that information is
unavailable or the task specifically concerns appearance.
## Run the packaged scripts
1. Resolve `<skill-dir>` to the absolute directory containing this `SKILL.md`,
using the skill location supplied by the host. Replace that placeholder in
commands below; keep script paths quoted and keep the project's working
directory.
2. Select a Python 3 interpreter: try `python3 --version`; on Windows, use
`python --version` or `py -3 --version` if needed. Substitute that command
for `python3` in the examples.
3. Invoke `bin/<tool>` and `scripts/<name>.py` through that interpreter. The
extensionless files in `bin/` are Python scripts, not installed binaries.
This works without PATH registration, executable permissions, or agent-home
symlinks. Short tool names in prose refer to these same script paths.
Use `--help` on the selected script for additional options.
## Local tools: use when their output is needed
These tools do not use the vision API; HTML rendering can load resources
referenced by the page.
| Need | Command |
| --- | --- |
| Reusable crop or close-up | `python3 "<skill-dir>/bin/crop" shot.png --region X1,Y1,X2,Y2 -o detail.png` |
| Locate pixel differences | `python3 "<skill-dir>/scripts/pixel_diff.py" before.png after.png` |
| Measure a region's palette | `python3 "<skill-dir>/scripts/dominant_colors.py" shot.png --region X1,Y1,X2,Y2` |
| Vectorize a flat graphic | `python3 "<skill-dir>/bin/trace" icon.png -o icon.svg` |
| Extract a transparent foreground | `python3 "<skill-dir>/scripts/extract_fg.py" shot.png --region X1,Y1,X2,Y2 -o icon.png` |
| Render an HTML file | `python3 "<skill-dir>/scripts/html_shot.py" page.html --width 1440 --height 900 -o page.png` |
- `crop --scale 4` enlarges a crop; it adds no missing detail. Coordinates
measured on that output use the enlarged grid, not the original image.
- `pixel_diff` reports average intensity difference and ranked grid cells,
not the fraction of changed pixels or a semantic comparison. It rescales a
size-mismatched second image and reports that mismatch; match viewports
before judging visual fidelity.
- `dominant_colors` clusters sampled pixels. `--candidates '#F9FAFA,#F5F5F5'`
selects the closest supplied palette value; it does not recover original CSS.
- `trace` produces approximate vector paths from pixels, with filtering,
scaling and curve fitting. It is not an exact size/offset measurement tool.
It suits flat, high-contrast graphics; text becomes paths. Read the graphic
playbook before shipping its output.
- `extract_fg` has manual-region and centered-icon auto modes. Inspect the
alpha edges and complete foreground on light and dark backgrounds before use.
Use existing utilities when they fit. A small Pillow operation is also fine
when it is the simplest way to obtain an output these tools do not provide.
## Task playbooks: load only the matching one
| Task | Read |
| --- | --- |
| Long screenshot or chat-history transcription, chunking and boundary checks | `references/long-screenshot-ocr.md` |
| Rebuild or align a UI from a reference; fast approximation or close fidelity | `references/restore-ui.md` |
| Extract or rebuild an isolated graphic as PNG/SVG | `references/restore-graphic.md` |
| Convert a visual diagram into Mermaid, Graphviz or structured data | `references/restore-structure.md` |
| Operate a GUI from screenshots: calibrate, act, verify | `references/gui.md` |
## Optional model-backed CLIs
Use these only for a requested second opinion, a needed box/list contract,
scripted OCR, or a model without native image input. They send images to the
separately configured vision chain, not to the main agent model automatically.
Structured output does not guarantee better recognition or localization.
```bash
python3 "<skill-dir>/bin/glance" shot.png -q "Read the small status label."
python3 "<skill-dir>/bin/glance" shot.png --region X1,Y1,X2,Y2 --ocr
python3 "<skill-dir>/bin/glance" before.png after.png -q "What changed semantically?"
python3 "<skill-dir>/bin/ground" shot.png "the send button"
python3 "<skill-dir>/bin/detect" shot.png "buttons"
```
`ground` and `detect` return estimated boxes in original-image pixels, including
with `--region`. They can miss objects or select the wrong target, not just
round an edge incorrectly. `detect` is not an exhaustive inventory guarantee.
For multi-image questions, pass the images in one call so the model sees them
together. Native side-by-side inspection is also sufficient for semantic
comparison; use pixel diff when numerical differences matter.
The long-screenshot OCR script invokes this skill's `bin/glance` with its
current Python interpreter, falling back to PATH only if the bundled script
is missing. Its `--split-only` mode makes chunks without API requests.
### Configuration and failures
`extensions/vision-toolkit.json` under the Pi agent directory selects the CLI `model` and
`fallbackModels` (entries `{"provider", "id", "baseUrl", "thinking"}`). For image-capable main models, `imagePolicy` defaults to `native`:
old and new images stay images, with no automatic description request.
`describe-old` and `describe` are legacy, explicit opt-ins for lossy image-to-text
conversion on supported connections. Text-only or unknown-capability models
still use the configured compatibility bridge. This policy does not route CLI
calls through the main model.
On a model-backed CLI configuration/API error, run
`python3 "<skill-dir>/bin/vision-doctor"`. Add `--smoke` only when a real
diagnostic request is needed; it sends a test image. Relay failures honestly.
Continue with native inspection when available; report if the scripted output
cannot be produced. Keep provider
changes user-controlled: do not scrape credentials, hand-export secret overrides,
or silently switch to an unconfigured provider.
Local tools need their optional dependencies (Pillow, NumPy, vtracer or a
Chrome-family browser, depending on the tool). If a command is missing, use an
already available equivalent or report the missing capability; API diagnostics
do not fix local dependency errors.
## Image lifetime
Use temporary image paths directly for immediate inspection. Copy into durable
work storage only when the file must survive its owner or session, or be kept
with deliverables. Clipboard-image attachments are deleted at Pi session
shutdown; preserve them beforehand only if needed beyond that session.
Source: https://github.com/Anionex/agent-vision-toolkit
Pliki w paczce (23)
- .gitignore 1 KB
- LICENSE 1 KB
- SKILL.md 8 KB
- detect.py 2 KB
- ground.py 8 KB
- vision_client.py 25 KB
- agents/openai.yaml 1 KB
- bin/crop 4 KB
- bin/detect 1 KB
- bin/glance 4 KB
- bin/ground 1 KB
- bin/trace 6 KB
- bin/vision-doctor 7 KB
- references/gui.md 4 KB
- references/long-screenshot-ocr.md 4 KB
- references/restore-graphic.md 4 KB
- references/restore-structure.md 2 KB
- references/restore-ui.md 10 KB
- scripts/dominant_colors.py 10 KB
- scripts/extract_fg.py 12 KB
- scripts/html_shot.py 4 KB
- scripts/long_screenshot_ocr.py 44 KB
- scripts/pixel_diff.py 4 KB
Wersje
W bazie nie ma innych skilli o nazwie vision-tools.
Migawka bazy z 13.11.2026 · wersja paczki 2c0635142aa2 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 98vision
Daje agentowi tekstowemu wzrok: opisuje, odczytuje tekst i analizuje zrzuty ekranu, zdjęcia, diagramy i wykresy przez narzędzie analizy obrazu. - 100vision-expert
Analizuje obrazy, diagramy, zrzuty ekranu i wykresy: zamienia interfejs w kod, interpretuje schematy i wykonuje OCR. - 100build-from-screenshot
Prowadzi autonomiczny potok od zrzutów ekranu lub adresu URL do działającej aplikacji React, Vue, Svelte lub React Native, z wnioskowaniem tokenów, testami najpierw, porównaniem pikseli i E2E. - 100testing-visual-regression
Wykrywa zmiany wizualne w komponentach UI przez porównywanie zrzutów ekranu, także niezamierzone zmiany i różnice pikseli. - 100testing-visual-regression
Wykrywa zmiany wizualne w komponentach UI przez porównywanie zrzutów ekranu, także niezamierzone zmiany i różnice pikseli. - 100screenshot-intake
Przechwytuje strony z adresu URL lub plików zrzutów, analizuje ich strukturę i tworzy plik specyfikacji build-spec.json.