Przejdź do treści
Biblioteka skilli
Inne · Przegląd i audyt

workspace-eval

/workspace-eval

Ocenia projekt według własnej rubryki obszaru roboczego i przekazuje niespełnione kryteria do pętli naprawy.

Repozytorium
ralphy ↗
Licencja
Apache-2.0
Ostatnia zmiana repozytorium
04.11.2026

Ocena i bezpieczeństwo

87/ 100Klasa: dobry
Format20 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność17 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
1
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 04.11.2026, bez odwołań do wycofanych modeli i API.

Licencja

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

Co robi i kiedy

Ocenia projekt według własnej rubryki obszaru roboczego i przekazuje niespełnione kryteria do pętli naprawy.

Oryginalny opis (EN)

Thin agent-facing wrapper over `ralphy workspace eval <project>` (#469): score one project against its WORKSPACE's CUSTOM rubric (#468 config), surface the per-criterion scorecard, and hand failing criteria to the repair loop. The runner writes `workspace-eval.json` + `workspace-eval-report.md` (append-only) — deterministic criteria run in code, vision criteria in ONE deep-vision pass folding in the workspace STYLE_LOCK + benchmarks. The `overall.verdict` uses the #427 readiness vocab (ship | repair | needs-user-decision | blocked). Pairs with `/evaluator` (the built-in structural / audio / ca

Jak działa

  1. ALSO FIRE
  2. DO NOT FIRE
  3. HARD INVARIANTS
  4. Run it
  5. The scorecard (workspace-eval.json)
  6. Surface it to the user
  7. Handoff to repair
  8. References
Sekcje z nagłówków SKILL.md, w oryginale · 67 linii, 674 słów
Podgląd SKILL.md
---
name: workspace-eval
namespace: user
description: >-
  Thin agent-facing wrapper over `ralphy workspace eval <project>` (#469): score one project against its WORKSPACE's CUSTOM rubric (#468 config), surface the per-criterion scorecard, and hand failing criteria to the repair loop. The runner writes `workspace-eval.json` + `workspace-eval-report.md` (append-only) — deterministic criteria run in code, vision criteria in ONE deep-vision pass folding in the workspace STYLE_LOCK + benchmarks. The `overall.verdict` uses the #427 readiness vocab (ship | repair | needs-user-decision | blocked). Pairs with `/evaluator` (the built-in structural / audio / caption gates) and `/fixer` (repair) — this one runs the per-workspace universe rubric, NOT the generic gates.
  USE WHEN the user says "score this against the workspace rubric", "run the custom evaluator", "workspace eval", "check it against the universe's criteria", "run my universe rubric on <project>", "does this pass the workspace's bar", or has an active workspace with an `evaluators.json` rubric and wants a standalone score (outside the four-stage `/universe-studio` flow).
  TRIGGER (EN): "workspace eval", "run the universe rubric", "score against the workspace rubric", "run the custom evaluator", "check against the universe's criteria", "does it pass the workspace bar".
  See body for ALSO FIRE / DO NOT FIRE / HARD INVARIANTS.
---

# workspace-eval — run the universe rubric

A thin wrapper over the #469 runner. **Your input is a project id + its workspace's custom rubric; your output is the per-criterion scorecard plus, on a non-clean verdict, a handoff to repair.** The engine is `ralphy workspace eval` — this skill is only the agent-facing surface.

## ALSO FIRE

- After a render, when the user wants a standalone score against the universe's bar without running the full four-stage `/universe-studio` orchestrator.
- When the user drops a `<project>` and asks whether it clears the workspace's own criteria (distinct from the generic `/evaluator` gates).

## DO NOT FIRE

- For the generic structural / audio / caption / vision gates on an arbitrary mp4 — that is `/evaluator`. This skill scores against the per-workspace CUSTOM rubric.
- For applying fixes — that is `/fixer` / the repair loop (#409). This skill only produces the scorecard and routes failures onward.
- For a workspace WITHOUT an `evaluators.json` rubric — there are no custom criteria to score; the runner reports "no rubric configured". Route to `/evaluator` for the built-in gates instead.
- As the per-stage gate inside the four-stage flow — that is `/universe-studio`, which drives this same runner per stage.

## HARD INVARIANTS

Inherited from `AGENTS.md`:

1. **`ralphy` is the only entry-point.** Run the eval through `ralphy workspace eval` — never call the model or probe the mp4 directly.
2. **Append-only.** The runner archives any existing `workspace-eval.json` / `workspace-eval-report.md` to `.vN` before writing the fresh one; never `--force-overwrite` without the user asking.
3. **Run `ralphy models show <id>`** before overriding the vision model (`--model`).

## Run it

```bash
ralphy workspace eval <project>
```

Flags:
- `--no-vision` — deterministic criteria only, NO model call (a free pass; use it to check the code-checked criteria without spending).
- `--model <id>` — override the deep-vision model (default `google/gemini-3.1-pro-preview`; check `ralphy models show <id>` first).
- `--workspace <slug>` — score against a different workspace's rubric (default: the project's registered workspace).
- `--video <path>` — override the scored video (default `<project>/render/final.mp4`).

It writes `<project>/workspace-eval.json` + `<project>/workspace-eval-report.md` (append-only) and prints the verdict + score + paths.

## The scorecard (`workspace-eval.json`)

```
{ schemaVersion, workspace, projectId, evaluatedAt, video,
  criteria: [ { id, label, category, check, score, verdict, threshold, findings[] } ],
  overall: { verdict, score, summary } }
```

- **Per criterion:** `check` is `deterministic` (code) or `vision`; `verdict` is `pass | warn | fail | na`; `score` is 0-100 or `null` when unscored; `findings[]` carries `{ severity, category, message, fixHint }`.
- **`overall.verdict` (#427 readiness vocab):** any criterion `fail` → **blocked**; else any `warn` → **repair**; else any REQUIRED (`severity: fail`) criterion `na`/unscored → **needs-user-decision**; else **ship**.
- `na` means the criterion did not run — an unregistered deterministic validator, or a vision criterion skipped (`--no-vision` / no video to score).

## Surface it to the user

Show the `overall.verdict` + score, then a one-line-per-criterion summary from `criteria[]` (id · verdict · score · findings count), leading with any `fail` / `warn`. The runner's `workspace-eval-report.md` is the ready-made human view — point the user at it or paste its criteria table.

## Handoff to repair

A `repair` / `blocked` verdict — or ANY `fail` / `warn` criterion — routes to **`/fixer`** / the repair loop (#409). Don't fix here: hand off the `workspace-eval.json` so the fixer reads the findings, builds the deterministic `ralphy project repair-plan <id>`, gates paid regeneration on the user, and re-evals. A `needs-user-decision` verdict means a required criterion is unscored (e.g. no render yet, or `--no-vision` left a hard-bar vision criterion `na`) — surface what's missing and let the user decide, don't auto-spend. A clean `ship` is done.

## References

- `cli/commands/workspace.ts` — the `eval` subcommand (flags + append-only persistence).
- `cli/lib/eval/workspace-evaluators.ts` — `runWorkspaceEval` + the `WorkspaceEvalResult` shape + `deriveOverallVerdict` (#427 mapping).
- `.agents/skills/evaluator/SKILL.md` — the built-in structural / audio / caption gates (the generic counterpart).
- `.agents/skills/fixer/SKILL.md` — the eval-to-repair loop this skill hands off to (#409).
- `.agents/skills/universe-studio/SKILL.md` — the four-stage orchestrator that drives this same runner per stage.

Wersje

RepozytoriumWynikLicencja
Ta strona · alecs5am/ralphy87Apache-2.0
alecs5am/ralphy87Apache-2.0

Migawka bazy z 13.11.2026 · wersja paczki 1419d7bb244f (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 87universe-studio
    Prowadzi produkcję w czterech etapach od pomysłu do finalnego montażu, z automatyczną naprawą według rubryki przestrzeni roboczej.
  • 87universe-studio
    Prowadzi produkcję materiału w oznaczonym obszarze roboczym w czterech etapach, od obsady i scenariusza przez kadry po montaż.
  • 90<workspace>
    Uruchamia lidera obszaru roboczego po wywołaniu polecenia albo prośbie o prowadzenie obszaru.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.