Przejdź do treści
Biblioteka skilli
Programowanie · Konwersja i formaty

vision-tools

/vision-tools

Obsługuje zrzuty ekranu i diagramy: rekonstrukcję, odczyt długich zrzutów przez OCR, przycinanie, różnice pikseli i śledzenie SVG.

Repozytorium
pi-extensions ↗
Licencja
AGPL-3.0
Ostatnia zmiana repozytorium
11.11.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
23
Skrypty
8
Komentarze HTML
0
Adresy URL w SKILL.md
1
Przestarzałość

Brak sygnałów. Repozytorium zmienione 11.11.2026, bez odwołań do wycofanych modeli i API.

Licencja

AGPL-3.0, copyleft: przeróbki udostępniane innym muszą mieć tę samą licencję.

Co sprawdzić przed instalacją
  • 8 skryptów w paczce. Przejrzyj je przed instalacją, bo Claude może je uruchomić.

Skrypty: detect.py, ground.py, scripts/dominant_colors.py, scripts/extract_fg.py, scripts/html_shot.py, scripts/long_screenshot_ocr.py, scripts/pixel_diff.py, vision_client.py

Co robi i kiedy

Obsługuje zrzuty ekranu i diagramy: rekonstrukcję, odczyt długich zrzutów przez OCR, przycinanie, różnice pikseli i śledzenie SVG.

Oryginalny opis (EN)

Native-first workflows for screenshot reconstruction, diagram extraction, screenshot-driven GUI work, long-screenshot OCR, and image processing (crops, pixel differences, palette measurements, SVG tracing, foreground extraction). Use when a task needs these workflows or concrete image artifacts; ordinary image questions can be answered with native vision.

Jak działa

  1. Workflow
  2. Run the packaged scripts
  3. Local tools: use when their output is needed
  4. Task playbooks: load only the matching one
  5. Optional model-backed CLIs
  6. Image lifetime
Sekcje z nagłówków SKILL.md, w oryginale · 142 linii, 1 094 słów
Podgląd SKILL.md
---
name: vision-tools
description: >-
  Native-first workflows for screenshot reconstruction, diagram extraction,
  screenshot-driven GUI work, long-screenshot OCR, and image processing
  (crops, pixel differences, palette measurements, SVG tracing, foreground
  extraction). Use when a task needs these workflows or concrete image
  artifacts; ordinary image questions can be answered with native vision.
---

# Vision tools

Use the main model's native vision first. The tools here supplement it with
image artifacts, measurements, or an optional separate-model opinion; they
are not a required route for seeing images.

## Workflow

1. Inspect the image natively and identify what the task requires. If you only
   have a description and need a missing detail, open the original image path.
   If the file is gone, report the missing evidence.
2. Zoom only when a relevant detail is unresolved. Crop that region and read
   the crop natively; use `ground` only if its estimated box would help locate
   the target. Stop inspecting once you have evidence for the requested answer.
3. Use local image processing for numerical claims or reusable artifacts.
   Treat model-generated boxes as estimates, not measured boundaries; verify
   their target and coordinate space before acting on them.
4. Verify the result against the task: readable text for OCR, nodes and edges
   for diagrams, the expected UI state after an action, or a rendered comparison
   for visual reconstruction. A low pixel-diff score alone is not correctness.

For browser or accessible GUI work, prefer DOM/accessibility information for
controls and state. Use screenshot coordinates when that information is
unavailable or the task specifically concerns appearance.

## Run the packaged scripts

1. Resolve `<skill-dir>` to the absolute directory containing this `SKILL.md`,
   using the skill location supplied by the host. Replace that placeholder in
   commands below; keep script paths quoted and keep the project's working
   directory.
2. Select a Python 3 interpreter: try `python3 --version`; on Windows, use
   `python --version` or `py -3 --version` if needed. Substitute that command
   for `python3` in the examples.
3. Invoke `bin/<tool>` and `scripts/<name>.py` through that interpreter. The
   extensionless files in `bin/` are Python scripts, not installed binaries.
   This works without PATH registration, executable permissions, or agent-home
   symlinks. Short tool names in prose refer to these same script paths.

Use `--help` on the selected script for additional options.

## Local tools: use when their output is needed

These tools do not use the vision API; HTML rendering can load resources
referenced by the page.

| Need | Command |
| --- | --- |
| Reusable crop or close-up | `python3 "<skill-dir>/bin/crop" shot.png --region X1,Y1,X2,Y2 -o detail.png` |
| Locate pixel differences | `python3 "<skill-dir>/scripts/pixel_diff.py" before.png after.png` |
| Measure a region's palette | `python3 "<skill-dir>/scripts/dominant_colors.py" shot.png --region X1,Y1,X2,Y2` |
| Vectorize a flat graphic | `python3 "<skill-dir>/bin/trace" icon.png -o icon.svg` |
| Extract a transparent foreground | `python3 "<skill-dir>/scripts/extract_fg.py" shot.png --region X1,Y1,X2,Y2 -o icon.png` |
| Render an HTML file | `python3 "<skill-dir>/scripts/html_shot.py" page.html --width 1440 --height 900 -o page.png` |

- `crop --scale 4` enlarges a crop; it adds no missing detail. Coordinates
  measured on that output use the enlarged grid, not the original image.
- `pixel_diff` reports average intensity difference and ranked grid cells,
  not the fraction of changed pixels or a semantic comparison. It rescales a
  size-mismatched second image and reports that mismatch; match viewports
  before judging visual fidelity.
- `dominant_colors` clusters sampled pixels. `--candidates '#F9FAFA,#F5F5F5'`
  selects the closest supplied palette value; it does not recover original CSS.
- `trace` produces approximate vector paths from pixels, with filtering,
  scaling and curve fitting. It is not an exact size/offset measurement tool.
  It suits flat, high-contrast graphics; text becomes paths. Read the graphic
  playbook before shipping its output.
- `extract_fg` has manual-region and centered-icon auto modes. Inspect the
  alpha edges and complete foreground on light and dark backgrounds before use.

Use existing utilities when they fit. A small Pillow operation is also fine
when it is the simplest way to obtain an output these tools do not provide.

## Task playbooks: load only the matching one

| Task | Read |
| --- | --- |
| Long screenshot or chat-history transcription, chunking and boundary checks | `references/long-screenshot-ocr.md` |
| Rebuild or align a UI from a reference; fast approximation or close fidelity | `references/restore-ui.md` |
| Extract or rebuild an isolated graphic as PNG/SVG | `references/restore-graphic.md` |
| Convert a visual diagram into Mermaid, Graphviz or structured data | `references/restore-structure.md` |
| Operate a GUI from screenshots: calibrate, act, verify | `references/gui.md` |

## Optional model-backed CLIs

Use these only for a requested second opinion, a needed box/list contract,
scripted OCR, or a model without native image input. They send images to the
separately configured vision chain, not to the main agent model automatically.
Structured output does not guarantee better recognition or localization.

```bash
python3 "<skill-dir>/bin/glance" shot.png -q "Read the small status label."
python3 "<skill-dir>/bin/glance" shot.png --region X1,Y1,X2,Y2 --ocr
python3 "<skill-dir>/bin/glance" before.png after.png -q "What changed semantically?"
python3 "<skill-dir>/bin/ground" shot.png "the send button"
python3 "<skill-dir>/bin/detect" shot.png "buttons"
```

`ground` and `detect` return estimated boxes in original-image pixels, including
with `--region`. They can miss objects or select the wrong target, not just
round an edge incorrectly. `detect` is not an exhaustive inventory guarantee.
For multi-image questions, pass the images in one call so the model sees them
together. Native side-by-side inspection is also sufficient for semantic
comparison; use pixel diff when numerical differences matter.

The long-screenshot OCR script invokes this skill's `bin/glance` with its
current Python interpreter, falling back to PATH only if the bundled script
is missing. Its `--split-only` mode makes chunks without API requests.

### Configuration and failures

`extensions/vision-toolkit.json` under the Pi agent directory selects the CLI `model` and
`fallbackModels` (entries `{"provider", "id", "baseUrl", "thinking"}`). For image-capable main models, `imagePolicy` defaults to `native`:
old and new images stay images, with no automatic description request.
`describe-old` and `describe` are legacy, explicit opt-ins for lossy image-to-text
conversion on supported connections. Text-only or unknown-capability models
still use the configured compatibility bridge. This policy does not route CLI
calls through the main model.

On a model-backed CLI configuration/API error, run
`python3 "<skill-dir>/bin/vision-doctor"`. Add `--smoke` only when a real
diagnostic request is needed; it sends a test image. Relay failures honestly.
Continue with native inspection when available; report if the scripted output
cannot be produced. Keep provider
changes user-controlled: do not scrape credentials, hand-export secret overrides,
or silently switch to an unconfigured provider.

Local tools need their optional dependencies (Pillow, NumPy, vtracer or a
Chrome-family browser, depending on the tool). If a command is missing, use an
already available equivalent or report the missing capability; API diagnostics
do not fix local dependency errors.

## Image lifetime

Use temporary image paths directly for immediate inspection. Copy into durable
work storage only when the file must survive its owner or session, or be kept
with deliverables. Clipboard-image attachments are deleted at Pi session
shutdown; preserve them beforehand only if needed beyond that session.

Source: https://github.com/Anionex/agent-vision-toolkit
Pliki w paczce (23)
  • .gitignore 1 KB
  • LICENSE 1 KB
  • SKILL.md 8 KB
  • detect.py 2 KB
  • ground.py 8 KB
  • vision_client.py 25 KB
  • agents/openai.yaml 1 KB
  • bin/crop 4 KB
  • bin/detect 1 KB
  • bin/glance 4 KB
  • bin/ground 1 KB
  • bin/trace 6 KB
  • bin/vision-doctor 7 KB
  • references/gui.md 4 KB
  • references/long-screenshot-ocr.md 4 KB
  • references/restore-graphic.md 4 KB
  • references/restore-structure.md 2 KB
  • references/restore-ui.md 10 KB
  • scripts/dominant_colors.py 10 KB
  • scripts/extract_fg.py 12 KB
  • scripts/html_shot.py 4 KB
  • scripts/long_screenshot_ocr.py 44 KB
  • scripts/pixel_diff.py 4 KB

Wersje

W bazie nie ma innych skilli o nazwie vision-tools.

Migawka bazy z 13.11.2026 · wersja paczki 2c0635142aa2 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 98vision
    Daje agentowi tekstowemu wzrok: opisuje, odczytuje tekst i analizuje zrzuty ekranu, zdjęcia, diagramy i wykresy przez narzędzie analizy obrazu.
  • 100vision-expert
    Analizuje obrazy, diagramy, zrzuty ekranu i wykresy: zamienia interfejs w kod, interpretuje schematy i wykonuje OCR.
  • 100build-from-screenshot
    Prowadzi autonomiczny potok od zrzutów ekranu lub adresu URL do działającej aplikacji React, Vue, Svelte lub React Native, z wnioskowaniem tokenów, testami najpierw, porównaniem pikseli i E2E.
  • 100testing-visual-regression
    Wykrywa zmiany wizualne w komponentach UI przez porównywanie zrzutów ekranu, także niezamierzone zmiany i różnice pikseli.
  • 100testing-visual-regression
    Wykrywa zmiany wizualne w komponentach UI przez porównywanie zrzutów ekranu, także niezamierzone zmiany i różnice pikseli.
  • 100screenshot-intake
    Przechwytuje strony z adresu URL lub plików zrzutów, analizuje ich strukturę i tworzy plik specyfikacji build-spec.json.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.