Przejdź do treści
Biblioteka skilli
AI i agenci · Analiza

meta-sam

/meta-sam

Znajduje, zarysowuje, liczy, mierzy i śledzi obiekty na zdjęciu lub w wideo modelem SAM 3.1 i rysuje maski oraz ramki.

Repozytorium
muse-muse ↗
Licencja
Apache-2.0
Ostatnia zmiana repozytorium
11.11.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
1
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 11.11.2026, bez odwołań do wycofanych modeli i API.

Licencja

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

Co robi i kiedy

Znajduje, zarysowuje, liczy, mierzy i śledzi obiekty na zdjęciu lub w wideo modelem SAM 3.1 i rysuje maski oraz ramki.

Oryginalny opis (EN)

Use to find, outline, count, measure or track objects in a photo or video with SAM 3.1 (sam-3.1), and to draw masks and boxes over them.

Jak działa

  1. Photo with a text concept
  2. Boxes and points (when the user taps or drags)
  3. Video
  4. Drawing results
  5. Counting and measuring
  6. Errors
Sekcje z nagłówków SKILL.md, w oryginale · 109 linii, 943 słów
Podgląd SKILL.md
---
name: meta-sam
description: Use to find, outline, count, measure or track objects in a photo or video with SAM 3.1 (sam-3.1), and to draw masks and boxes over them.
---

# SAM 3.1

All calls run on the server. Parse and draw results with Meta's client libraries, which are already installed.

## Photo with a text concept

This mirrors Meta's own example.

```ts
import { metaClient } from "@/lib/meta/client";
import { imagePart } from "@/lib/meta/parts";
import { parseImageStream, recordsOfKind, decodeMaskToSVGPath } from "@meta-sam/parser";

const meta = metaClient(request); // the route handler's incoming request
const events = await meta.responses.create({
  model: "sam-3.1",
  input: [
    {
      type: "message",
      role: "user",
      content: [
        { type: "input_text", text: "wall" },
        imagePart(photoUrlOrDataUrl),
      ],
    },
  ],
  stream: true,
  metadata: { mask_encoding: "one_bit", include_confidence: "true" },
});

const result = await parseImageStream(events).finalResult;
if (result.outcome.status !== "completed") throw new Error("Segmentation ended early");
const masks = recordsOfKind(result.records, "mask"); // one per instance; boxes via recordsOfKind(result.records, "box")
const count = masks.length;
```

### What comes back

- Always pass `stream: true`. The stream is standard Responses SSE (`response.created`, `response.output_text.delta`, `response.output_text.done`, `response.completed`). The OpenAI SDK's stream goes straight into `parseImageStream` or `parseVideoStream`.
- The result is packed into the output text as tokens, like `<0f>0<|box;x1=510;y1=265;x2=674;y2=408;w=800;h=600;c=0.96|><|mask;...|>,1<|box;...`. Never parse that by hand. Each record has `objectId`, `bounds` and, with `include_confidence`, `confidence` (0 to 1).
- `response.completed` carries `usage: { images_processed: 1, input_tokens: 0, output_tokens: 0 }`. SAM is billed per image or video frame processed, not by tokens.
- If you need the stream for something else too, collect the events into an array first, then parse from it.

### Prompt rules

- Use one short noun phrase per call, like "wall", "yellow school bus" or "dog".
- Lists don't work, and "cats, dogs" returns nothing. Commands, questions and negation don't work either.
- For several things, make one call per concept and run them in parallel.
- Photos return every matching instance.

## Boxes and points (when the user taps or drags)

- Build prompts with `buildObjectPrompt` from `@meta-sam/parser` instead of writing tokens by hand.
- A box token looks like `<|box;x1=444;y1=626;x2=675;y2=849;w=1800;h=1200|>`, with `x2` and `y2` inclusive. A point looks like `<|point;x=..;y=..;w=..;h=..|>`, and a leading `-` marks a negative point. Photos use the frame marker `<0f>`.
- `w` and `h` must be the pixel size of the media you sent. A different aspect ratio returns 400.
- You can prompt at most 16 objects.

## Video

1. In the browser, upload the clip with `uploadMedia` (product-ui skill). Function bodies can't carry video.
2. On the server, send a concept and `videoPart({ url })` from `@/lib/meta/parts`. Meta downloads public https URLs itself. If that fails with `media_url_origin_error`, upload the bytes with `meta.files.create` and use `videoPart({ fileId })`.
3. Stream the results and parse them with `parseVideoStream`. Each frame comes back on its own line. With concept prompts, frames arrive in order. With box or point prompts they can arrive out of order, so use `frameIndexOf` rather than arrival order.

```ts
import { videoPart } from "@/lib/meta/parts";
import { parseVideoStream } from "@meta-sam/parser";

const events = await meta.responses.create({
  model: "sam-3.1",
  input: [{ type: "message", role: "user", content: [{ type: "input_text", text: "bird" }, videoPart({ url: clipUrl })] }],
  stream: true,
} as any); // the OpenAI SDK types don't include input_video
const video = parseVideoStream(events as any);
```

Video limits:

| Limit | Value |
|---|---|
| Containers | MP4 or MOV |
| Codecs | H.264, H.265 or MPEG-4. VP9 and WebM are rejected, so don't record with MediaRecorder |
| Length | Up to 15,000 frames, about 10 minutes |
| Objects | Up to 16 tracked per frame |
| Cost | Billed per frame, with a 50-frame minimum. Keep demo clips to 5 to 20 seconds |

For box and point prompts, `metadata.propagation_direction` can be `both` (the default), `forward` or `backward`.

## Drawing results

- `SegmentationRenderer` from `@meta-sam/graphics` draws masks and boxes on a canvas. Its default colors aren't the theme's: pass an `objectColor` option that gives every object `--signature`, or, only when the result names several kinds of object, each kind one of `--cherry`, `--grape`, `--lemon`, `--lime`, `--pink` and `--teal`. A canvas can't read `var()`, so read each once with `getComputedStyle(document.documentElement).getPropertyValue("--signature").trim()`. Leave `boxLabels` off: it prints object ids and confidence decimals. Say the result in a caption instead ("Found 3 birds").
- `@meta-sam/video` and `@meta-sam/react` (`Video`, `useMediaPlayer`) give frame-accurate playback under overlays.
- `decodeMaskToSVGPath` works well for SVG overlays of organic shapes (a bird, a plant, a person). For rectangular things (charts, tables, screens, documents), draw only the box, with rounded corners: a traced mask looks jagged there. White lines vanish on a page render, a document or a drawing, so stroke those in `var(--signature-line)` at 3 px with a 15% `--signature` fill (meta-ui components.md, Images and overlays). `decodeMaskToRaster` returns a row-major 0/1 `Uint8Array` for pixel math. `decodeMaskToRLE` returns COCO RLE.

## Counting and measuring

- **Count.** In a photo, count mask records for the concept. In a video, count distinct object ids across frames.
- **Size.** Size needs a reference object of known size in the same photo. A sheet of US letter paper is 8.5 by 11 inches. Segment the reference, get pixels per inch from its box, then measure the target. Label the result as an estimate in the UI.

## Errors

| Status | Meaning |
|---|---|
| 400 | Invalid prompt or metadata. The message names the problem. Fix it before retrying |
| 400 `media_url_origin_error` | Meta couldn't download your media URL. Send a data URL (photos) or a file id instead |
| 404 `model_not_found` | Wrong model id, or this product isn't allowed to use SAM |

An empty result is not an error: zero masks means SAM found nothing for that phrase. Say so in the UI ("No birds found. Try a closer photo.").

Wersje

W bazie nie ma innych skilli o nazwie meta-sam.

Migawka bazy z 13.11.2026 · wersja paczki cd216fe14a72 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100sam3-segmentation
    Wyznacza maski i kontury obiektów na zdjęciach modelem SAM 3 uruchamianym lokalnie na GPU NVIDIA, także przy liczeniu i annotacji wielu drobnych elementów.
  • 100muse-media
    Składa wideo ze zdjęć, zapisuje muzykę jako MIDI, zamienia nuconą melodię w nuty i liczy obiekty na zdjęciu.
  • 96sam3-segmentation
    Dodaje do aplikacji segmentację obrazów modelem SAM 3.1, czyli wydzielanie obiektów na zdjęciach.
  • 90recipe-sam-media
    Pracuje z wideo w SAM 3.1: maski na klipach, animowane naklejki wycięte z filmu i napisy w dymkach podążające za mówcą.
  • 95segment-anything-model
    Segmentuje dowolne obiekty na obrazach bez treningu, na podstawie punktów, ramek lub masek, albo generuje wszystkie maski naraz.
  • 100sam-template-builder
    Prawdopodobnie pomaga tworzyć szablony AWS SAM (Serverless Application Model) do wdrażania aplikacji serverless.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.