framefields-render
/framefields-render
Renderuje, sprawdza, testuje i eksportuje wideo WebGPU oraz siatki klatek silnikiem headless framefields, w tym diagnozuje puste klatki, profiluje potok GPU i eksportuje finalny MP4.
- Autor
- gatewai-dev ↗
- Repozytorium
- framefields ↗
- Licencja
- Apache-2.0
- Ostatnia zmiana repozytorium
- 27.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 1
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 3
Brak sygnałów. Repozytorium zmienione 27.10.2026, bez odwołań do wycofanych modeli i API.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
Co robi i kiedy
Renderuje, sprawdza, testuje i eksportuje wideo WebGPU oraz siatki klatek silnikiem headless framefields, w tym diagnozuje puste klatki, profiluje potok GPU i eksportuje finalny MP4.
Oryginalny opis (EN)
Render, inspect, test, and export high-performance WebGPU video and frame grids with the framefields headless engine. Use when verifying compositions, diagnosing blank frames, profiling GPU pipelines, running visual frame grids, or exporting final MP4 video in framefields.
Jak działa
- Setup
- Overview
- The 5-Step Verification Ladder
- Rendering Without a GPU
- Production Render Script Pattern
- Built-in Video QA (renderVideo({ qa }))
- Inspecting a Finished File with ffmpeg
- Common Rendering Failure Modes
Podgląd SKILL.md
---
name: framefields-render
description: Render, inspect, test, and export high-performance WebGPU video and frame grids with the framefields headless engine. Use when verifying compositions, diagnosing blank frames, profiling GPU pipelines, running visual frame grids, or exporting final MP4 video in framefields.
---
# framefields-render
## Setup
Import everything from the [`framefields`](https://www.npmjs.com/package/framefields) npm package. If the project doesn't have framefields set up yet, follow the `framefields` skill first (install, project scaffold, first render).
## Overview
`framefields` renders headlessly in Node.js on top of native WebGPU (Dawn), encoding through FFmpeg with a hardware encoder when one works (NVENC, VideoToolbox) and a software one otherwise. It does not run a browser, Puppeteer, or Chromium.
Because video rendering is resource-intensive and native GPU pipelines run asynchronously in worker pools, framefields workflows enforce a strict **verification ladder**: inspect inexpensive intermediate artifacts first before committing to a full video render.
---
## The 5-Step Verification Ladder
Always proceed through the ladder in order. Catching a layout or timing bug at step 1 or 2 takes seconds; catching it after a 30-second 4K video render wastes minutes.
### Step 1: Static Typecheck & Schema Validation
Before touching the GPU, verify TypeScript types and parse the composition against the strict runtime schema:
```typescript
import { CompositorProgramSchema } from "framefields";
// 1. Compile spec
const spec = comp.toSpec();
// 2. Validate against strict schema
const parsed = CompositorProgramSchema.safeParse(spec);
if (!parsed.success) {
console.error("Schema validation failed:", parsed.error.format());
process.exit(1);
}
// 3. Lint: things that render without error but not as intended
for (const d of comp.lint()) console.log(`${d.severity} ${d.code}: ${d.message}`);
```
Typecheck the project:
```bash
npx tsc --noEmit -p .
```
---
### Step 2: Single Frame Inspection (`renderFrame`)
Render isolated keyframe points (e.g. entry, peak action, exit) directly to PNG files on disk to inspect alignment, typography, colors, and opacity:
```typescript
import fs from "node:fs/promises";
import path from "node:path";
import { HeadlessMediaRenderer } from "framefields";
const renderer = new HeadlessMediaRenderer();
// Render frame 45 (1.5s at 30fps)
const pngBuffer = await comp.renderFrame({
frame: 45,
renderer,
});
await fs.writeFile("output/frame-45.png", pngBuffer);
```
---
### Step 3: Motion & Layout Stability Contact Sheets (`renderFrameGrid`)
To verify easing curves, cut points, kinetic sweeps, or text wrapping without encoding a full video, generate a sequential frame grid image:
```typescript
import fs from "node:fs/promises";
import { renderFrameGrid } from "framefields";
// Samples frames across the intro sweep (frames 0 to 60)
const gridPng = await comp.renderFrameGrid({
frames: [0, 6, 12, 18, 24, 30, 36, 42, 48, 54, 60],
columns: 4,
cellWidth: 480, // Downscaled thumbnail size for rapid visual scanning
showLabels: true, // frame number and timestamp under each cell
});
await fs.writeFile("output/intro-grid.png", gridPng);
```
Contact sheets immediately reveal:
- Motion hitches and non-smooth easing deceleration.
- Text clipping or wrapping jumps across keyframe boundaries.
- Bad layer z-indexing or missing cut transitions.
---
### Step 4: Programmatic Conformance Probes in Vitest
Write automated headless vitest tests that assert pixel colors, transparency, or layout bounding boxes:
```typescript
import { describe, it, expect } from "vitest";
import { HeadlessMediaRenderer } from "framefields";
import { buildFilm } from "../film.js";
describe("Hero Scene Conformance", () => {
it("renders non-black title text on frame 30", async () => {
const comp = await buildFilm();
const renderer = new HeadlessMediaRenderer();
const png = await comp.renderFrame({ frame: 30, renderer });
expect(png.length).toBeGreaterThan(1000);
// Use skia-canvas or pixel probes to assert color at target coordinates
});
});
```
---
### Step 4.5: Show the User Before Exporting (`preview`)
When a person is in the loop, let them watch the composition before you spend time on the final render. `startPreview` serves a localhost player with the soundtrack, a timeline with the waveform, and frame stepping. The page loads the composition's own code and renders it with WebGPU in the browser, in real time: a device that can't draw every frame in time skips some, and the sound keeps its speed; the server only serves that code, the project's files, and the soundtrack mixed by the export engine. Point it at the module that builds the composition:
```typescript
import { startPreview } from "framefields";
const session = await startPreview(
{ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" },
{ title: "Launch film" },
);
console.log(`Preview at ${session.url}`);
await session.closed; // keep the process alive while the user watches
```
- **The export is a function returning the `Composition`** (it may be async), or a `Composition` itself. The browser calls it, so build-time Node code in it must stay simple: `node:path`, `fileURLToPath(import.meta.url)` and synchronous `fs` reads (JSON, text) work through shims; writes are skipped. Assets referenced by absolute path load from the server, limited to the project that holds the entry (its nearest `.git` or `package.json`); pass `root` with one or more paths for assets kept outside it (e.g. a shared fonts folder).
- **Run it in the background, and read its output.** Start the script as a background command with the longest timeout your harness allows, tell the user the URL, and ask for feedback in the chat. The command's output (stderr, prefixed `[framefields preview]`) says when the page opened, reports each distinct error from the page once (a build error, a frame that threw, a font or renderer that failed to load, no WebGPU), and says why it stopped. Check it after the page opens instead of assuming the film drew. `log` redirects or silences these lines, and `session.closed` resolves with the reason: `"idle"`, `"replaced"` or `"closed"`.
- **One tab for the whole loop.** The port is fixed per project directory. Running the preview again (after you've made changes) takes over from the one already running, and a tab left open reconnects and reloads into the new version by itself, at the moment it was showing. Don't ask the user to reopen anything. When the user closes the tab, the preview stops by itself about 5 s later (a reload doesn't stop it; `idleCloseMs` changes the delay). If the preview was stopped (tab closed, time limit, or you killed it), just start it again.
- **Notes pinned on the picture are the user's precise feedback.** Paused, the user clicks the frame (or drags an area) and writes what should change; the note keeps the frame, the spot as fractions of the frame (`x`, `y`, and `w`, `h` for an area) and the text. Each note prints in the output (`note 3 at 0:12.50 (frame 750), spot 42%,31%: …`) with the path of `note-<id>.jpg`, the frame with the spot marked, and all of them are in `.framefields/preview-notes/notes.json` (`.framefields` ignores itself in git; `notesDir` moves it, `false` turns notes off). To act on them: read `notes.json`, open each open note's picture to see exactly what it points at, find the layer in the code from the moment and the spot, make the change, and run the preview again. The tab comes back with the notes, so the user can check them and tick them off. You can also set `"done": true` in `notes.json` yourself. A chat that can't read files gets the same list from the **Copy** button in the Notes panel.
- **Point at a moment.** `#t=12.5` in the URL opens the player at 12.5 s (`#frame=750` by frame), and the address follows the playhead when it stops, so a link you give the user lands where you mean.
- **Drive it from a script, not with clicks.** If you can run JavaScript in the page (an agent browser), use `window.framefieldsPreview`: `await seek(seconds)` or `await seekFrame(n)` resolve once that frame is drawn (take the screenshot then), `play()` and `pause()` control playback, `notes()` lists the notes, and `state()` returns `{ state: "loading" | "ready" | "error", error, frame, time, playing, buffering, audio, connected, playbackFps, renderMs, meta }`. `<html data-preview="ready">` marks the first frame drawn. A hidden tab pauses playback, since browsers throttle it.
- **Show it where the user is.** `startPreview` only serves the page and returns the URL. If your app has a built-in browser, open the URL there. Otherwise give the user the link, or pass `open: true` to open their default browser (skipped when an open tab reconnects on its own). The page needs WebGPU (current Chrome, Edge or Safari).
- **Playback runs in real time, on any device.** Play draws the frame due on the clock and skips the ones it missed, so it starts at once and the sound keeps its speed; a slow device shows fewer frames. Next to the timecode the page shows the frame rate playback reaches, apart from the film's own `fps`; `state()` reports it as `playbackFps`, and `renderMs` is how long the last frame took to draw. Paused, the page draws the exact frame under the playhead. Vision effects analyse each frame from the one drawn before it: while paused, the page analyses up to 20 s ahead so that stretch plays exactly, but past it a device that skips frames shows cutouts and tracking from the last frame it drew. Check them paused, or in the export.
- Audio is the same mix `renderVideo` muxes (`audio: false` skips it). Playback uses the soundtrack as its clock; when a frame takes long (a video seek, a heavy shot), the picture skips ahead to catch up with the sound.
---
### Step 5: Full Video Export (`renderVideo`)
Once static frames and motion grids are verified, encode the final MP4/WebM video:
```typescript
import path from "node:path";
const result = await comp.renderVideo({
outputPath: "output/final-video.mp4",
quality: "high", // "draft" | "medium" | "high" | "lossless"
codec: "avc", // "avc" (H.264) | "hevc" (H.265) | "vp9" | "av1"
audioCodec: "aac",
concurrency: 4, // Parallel frame encoding threads
});
// Always invoke cleanup to release temporary frame cache and file handles
await result.cleanup?.();
```
Pass `qa: true` to have the engine check the output while it renders (see **Built-in Video QA** below) and return the findings as `result.qa`.
---
## Rendering Without a GPU
Agents often run in sandboxes or CI runners with no GPU. framefields then renders on the CPU, with the same output for every feature (text, media, Lottie, effects, 3D, vision; vision inference moves to the CPU too).
- **Linux:** install Mesa lavapipe, `apt-get install -y libvulkan1 mesa-vulkan-drivers`. Without a Vulkan driver at all, nothing can render.
- **Windows:** Direct3D 12's WARP renderer is built in; lavapipe (e.g. from mesa-dist-win) works through Vulkan too.
- **macOS:** always has a Metal GPU, VMs included; framefields' WebGPU build has no Vulkan, so lavapipe isn't used there.
- **Confirm it:** `await describeGpu()` returns `software: true`.
- **Budget for speed:** about 12× slower than an Apple M1 Pro's GPU in our Linux test (the smoke project's three videos: 9.7 min against 49 s); it scales with CPU cores. Frames, grids and short exports are fine. Vision layers are the slowest part.
- **Long films:** render on a GPU machine, or push the project and submit it to the framefields render worker (Docker or RunPod, renders a git commit on a GPU; if authentication is required, ask the user to set `FRAMEFIELDS_API_KEY` in the process environment and use it without printing or embedding it). Its project checklist and job format: https://framefields.dev/docs/guides/render-service/
## Production Render Script Pattern
Production projects structure their `src/render.ts` to support both quick frame probes and full exports from the CLI:
```typescript
// src/render.ts
import fs from "node:fs/promises";
import path from "node:path";
import { HeadlessMediaRenderer, startPreview } from "framefields";
import { buildFilm } from "./film.js";
const OUT = path.resolve(import.meta.dirname, "../output");
const [mode = "video", ...args] = process.argv.slice(2);
if (mode === "preview") {
// The browser runs film.ts itself; no need to build the film here.
const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" });
console.log(`Preview at ${session.url}`);
await session.closed; // serves until its tab closes or the next preview replaces it
process.exit(0);
}
const comp = await buildFilm();
await fs.mkdir(OUT, { recursive: true });
if (mode === "frames") {
const renderer = new HeadlessMediaRenderer();
await fs.mkdir(path.join(OUT, "frames"), { recursive: true });
for (const f of args.map(Number)) {
const png = await comp.renderFrame({ frame: f, renderer });
const file = path.join(OUT, "frames", `f${String(f).padStart(4, "0")}.png`);
await fs.writeFile(file, png);
console.log(`Rendered frame: ${file}`);
}
} else {
const started = Date.now();
const result = await comp.renderVideo({
outputPath: path.join(OUT, "output.mp4"),
quality: "high",
});
await result.cleanup?.();
console.log(`Exported video: ${result.filePath} in ${((Date.now() - started) / 1000).toFixed(1)}s`);
}
// CRITICAL: Always explicitly exit process to prevent native WebGPU/mediabunny thread hangs
process.exit(0);
```
CLI Usage:
```bash
# Render specific keyframes for instant review:
tsx src/render.ts frames 0 18 36 72
# Live player for the user (run in the background; re-run after changes and the open tab reloads):
tsx src/render.ts preview
# Render full master video:
tsx src/render.ts
```
### Reading render progress
`renderVideo` logs its own progress to stderr, one short `[framefields]` line per event. The log stays short however long the film is, about 15 lines for an export:
```text
[framefields] Preparing 1800 frames (60.0s) at 1920x1080, 30 fps, h264
[framefields] Rendering 1800 frames
[framefields] Rendered 10% (180/1800 frames) · 41.2 fps · eta 39.3s
…
[framefields] Rendered 100% (1800/1800 frames) · 41.8 fps · in 43.1s
[framefields] Mixing audio
[framefields] Encoding 60.0s of audio (aac)
[framefields] Muxing mp4
[framefields] Done in 51.4s (14.2 MB): prepare 1.2s · render 43.1s · audio 3.2s · audio-encode 1.1s · mux 0.8s
```
- **Work continues after the last frame.** "Mixing audio" decodes every audio and video clip in the film, then the soundtrack is encoded and "Muxing" writes the file. With many or long clips this takes a while. It is not a hang; wait for `Done in …`.
- **A quiet phase prints `Still …` lines**, e.g. `Still mixing audio (42%, 3/7 clips: music.mp3), 1m 10s in this phase`: after 10 s, 30 s, 70 s, then backing off to one every 5 min. Each says which phase is running and for how long. A render is stuck only when no line arrives for well over the last gap. Killing it at any point leaves no video.
- **A failure names its phase:** `Failed while mixing audio after 12.3s`, with the error.
- **Run long exports in the background** with the longest timeout your harness allows, and read the output when it finishes rather than polling it.
- `LOG_LEVEL=warn` silences these lines; `LOG_LEVEL=debug` adds per-stage detail. Warnings and errors read `[framefields] warn: …` / `error: …`.
- With `NODE_ENV=production` (or `FRAMEFIELDS_LOG_FORMAT=json`) the lines are pino JSON on stdout instead, with the text in `msg`; `FRAMEFIELDS_LOG_FORMAT=text` forces the short lines.
- For your own progress output, `onProgress` gets every event as `{ phase, frame, totalFrames, elapsedMs, phaseProgress?, fps?, etaSec?, detail? }`. `phase` runs `prepare`, `render` (audio-only output skips it), `audio`, `audio-encode`, `mux`, `convert` (GIF only), `done`.
---
## Built-in Video QA (`renderVideo({ qa })`)
The engine measures the video while it renders: every frame as it is read back for the encoder, and the audio as it is mixed. No second decode pass, and the numbers describe exactly what was encoded.
```typescript
import { formatQaReport } from "framefields";
const result = await comp.renderVideo({
outputPath: "output/final-video.mp4",
qa: { targetLufs: -14 }, // or `qa: true` for defaults (no loudness target)
});
console.log(formatQaReport(result.qa!));
if (!result.qa!.passed) process.exitCode = 1; // fail CI / the agent loop
```
`result.qa` is structured, so branch on it instead of parsing logs:
| `issues[].code` | Severity | Meaning |
|---|---|---|
| `frames_missing` | error | Fewer frames encoded than the composition has |
| `black_frames` | warning | ≥ `minBlackSec` (0.5 s) of black frames; has `startSec`/`endSec` |
| `audio_clipping` | warning | Samples at or beyond full scale |
| `loudness_off_target` | warning | Integrated loudness more than `lufsTolerance` (1 LU) from `targetLufs` |
| `animation_truncated` | warning | Keyframes past a layer's clip window never play; has `layerId` |
| `frozen_video` | info | Picture unchanged for ≥ `minFrozenSec` (2 s); fine for held title cards |
| `audio_silence` | info | ≥ `minSilenceSec` (2 s) below `silenceDbfs` (-60) |
| `audio_silent` | info | The whole track is silent |
| `invalid_gradient` | error | A `background` / `fill` gradient string that does not parse: it draws nothing |
| `unsupported_image_format` | error | An image layer receiving a `.json` / `.lottie` vector animation, video, or audio file |
| `unknown_color` | warning | A color string that is not a color: it draws transparent |
| `low_contrast_text` | warning | Text below 3:1 contrast against the solid color behind it (suggests a readable one) |
| `offscreen_layer` | warning | A static absolute layer entirely outside the canvas |
| `shadow_unsupported_light` | warning | `castShadow` on a point or ambient light |
| `pbr_without_environment` | info | `roughness` / `metallic` set but no `Layer.environment()` (no physical shading) |
| `multiple_shadow_lights` | info | Several `castShadow` lights; only the first casts |
| `static_track` | info | Every keyframe on a track holds the same value |
The document checks also run on their own, before any render: `comp.lint()` returns the same `{ severity, code, message, layerId }` entries. Run it in Step 1, right after typechecking.
`passed` is false when any error or warning is present. `qa.audio.integratedLufs` is ITU-R BS.1770 loudness (streaming platforms normalize to about -14 LUFS); `qa.audio.peakDbfs` is the sample peak.
## Inspecting a Finished File with ffmpeg
For files rendered elsewhere, or to compare against a previous render, inspect with `ffmpeg`/`ffprobe`:
```bash
# 1. Check video and audio stream properties
ffprobe -v error -show_entries stream=codec_name,width,height,duration,r_frame_rate output/final-video.mp4
# 2. Detect unexpected black frames or dead transitions
ffmpeg -i output/final-video.mp4 -vf "blackdetect=d=0.5:pix_th=0.10" -f null -
# 3. Verify audio loudness levels (prevent clipping / silence)
ffmpeg -i output/final-video.mp4 -af "volumedetect" -f null -
```
---
## Common Rendering Failure Modes
| Symptom | Root Cause | Solution |
|---|---|---|
| Entire frame renders black | Missing font registration or `NaN` in uniform | Ensure every font used in `Layer.text` is registered via `FontManager.register()` and listed in `comp.fonts`. Verify all signal outputs are finite numbers. |
| Media layer is invisible / blank | Bad input handle binding or zero opacity | Check that the media path exists and `opacity > 0`. If using an effect or section, ensure `fit: "cover"` and valid container dimensions. |
| Audio cuts out early | Missing `durationFrames` on audio layer | Explicitly assign `durationFrames: totalFrames` on `Layer.audio()`. |
| Vision cutout / mask is empty on a single `renderFrame` | Vision reads the previous rendered frame; frame 0 has none | Render two or more consecutive frames with the same `HeadlessMediaRenderer` and keep the last. `renderVideo` handles this automatically. |
| Ghosted double of the subject in a vision cutout | Frames rendered out of order (frame grid, or jumping from frame 3 to 48), so the cutout uses pixels from another frame | Verify vision layers with sequential frames or the exported MP4, not `renderFrameGrid`. |
| Output stops after `Rendered 100%` | Audio is being mixed, encoded or muxed | Not a hang: wait for `Still …` lines and `Done in …` (see **Reading render progress**). |
| Script hangs after `Done in …` | Background worker thread pool held open | Always include `process.exit(0)` at the end of render scripts. |
| Render very slow on a Linux server | Frames drawn on a CPU adapter (lavapipe) | `await describeGpu()` shows `software: true`; fix the GPU's Vulkan driver (`vulkaninfo --summary`). |
| Encoder fails to open (`h264_rkmpp`, NVENC) | framefields < 2.2 uses the hardware encoder untested | Use framefields 2.2+, which falls back to libx264 etc., or set `FRAMEFIELDS_HW_CODECS=off`. |
| Render worker job fails | Project not render-ready (lockfile, export, LFS) | Read the job's `code` and `hint`; checklist at https://framefields.dev/docs/guides/render-service/ |
| Text looks misaligned or truncated | Layout node box smaller than text string | Set explicit wrapping `width` (e.g. `width: 1400`) or use `width: "fill"`. |
| Colors appear washed out / dark edges | Unpremultiplied alpha blending | The engine operates with premultiplied alpha. Ensure custom shaders use `srcFactor: "one"`. |
Wersje
W bazie nie ma innych skilli o nazwie framefields-render.
Migawka bazy z 13.11.2026 · wersja paczki 7bbb8383825f (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100framefields
Zakłada nowy projekt wideo framefields lub dodaje framefields do istniejącego projektu Node.js: instalacja, szkielet kompozycji i skrypt renderu, pierwszy MP4 oraz generowanie obrazów, wideo, muzyki i głosu przez modele AI. - 100framefields-cloud
Renderuje projekty framefields na chmurowych GPU lub własnym workerze, planuje koszt i dzieli długi film między kilka GPU. - 100video-frames
Wyciąga klatki z plików wideo i analizuje je wizualnie, w tym tekst na ekranie i siatki klatek. - 100guidance-webgpu-performance
Diagnozuje i poprawia wydajność renderowania WebGPU w przeglądarce: tempo klatek, narzut rysowania, wysyłanie buforów, shadery i kod JavaScript na ścieżce renderowania. - 90webgpu-capture-analysis
Interpretuje zrzuty klatek WebGPU Inspector: graf obiektów GPU, listę poleceń, przejścia, bufory, tekstury i błędy walidacji. - 86renderer
Wyznacza zasady architektury renderowania GPU w TypeGPU i WebGPU przy dodawaniu lub zmianie przebiegów renderowania, shaderów, orkiestracji klatek, zasobów GPU, cullingu, terenu, cieni i wydajności wizualnej.