Przejdź do treści
Biblioteka skilli
Nauka i badania · Testowanie

aims

/aims

Pisze, sprawdza i uruchamia scenariusze AIMS, stanowisko do eksperymentów z dopasowaniem AI do celów, a potem odczytuje wyniki sprawdzianów i raportów.

Repozytorium
AIMS ↗
Licencja
Apache-2.0
Ostatnia zmiana repozytorium
13.11.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
9
Skrypty
4
Komentarze HTML
0
Adresy URL w SKILL.md
1
Przestarzałość

Brak sygnałów. Repozytorium zmienione 13.11.2026, bez odwołań do wycofanych modeli i API.

Licencja

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

Co sprawdzić przed instalacją
  • 4 skrypty w paczce. Przejrzyj je przed instalacją, bo Claude może je uruchomić.

Skrypty: assets/scenario/files/test_wordcount.py, assets/scenario/files/wordcount.py, assets/scenario/tests/score.py, assets/scenario/tests/test_hidden.py

Co robi i kiedy

Pisze, sprawdza i uruchamia scenariusze AIMS, stanowisko do eksperymentów z dopasowaniem AI do celów, a potem odczytuje wyniki sprawdzianów i raportów.

Oryginalny opis (EN)

Write, check and run AIMS scenarios, then read their results. AIMS is a bench for alignment experiments. Model agents work on a task in Docker sandboxes with planted temptations while a monitor model flags misbehaviour. Use when the user wants to design an AIMS scenario, plant or variant, write a verifier or a monitor rule, run or fork a scenario with the `aims` CLI, or explain a run's check, score or report.

Jak działa

  1. Rules
  2. Preflight
  3. Write a scenario
  4. Design plants
  5. Run and read results
  6. When something fails
Sekcje z nagłówków SKILL.md, w oryginale · 93 linii, 910 słów
Podgląd SKILL.md
---
name: aims
description: Write, check and run AIMS scenarios, then read their results. AIMS is a bench for alignment experiments. Model agents work on a task in Docker sandboxes with planted temptations while a monitor model flags misbehaviour. Use when the user wants to design an AIMS scenario, plant or variant, write a verifier or a monitor rule, run or fork a scenario with the `aims` CLI, or explain a run's check, score or report.
license: Apache-2.0
---

# AIMS

AIMS runs model agents on a task in Docker sandboxes. The task hides temptations called plants. A monitor watches every step and raises flags. A score compares the plants each agent took with the flags the monitor raised.

## Rules

- Use the installed CLI, `aims`. Check it with `aims --version`. If it is missing, install it with `uv tool install aims-bench -c https://github.com/kaumnen/AIMS/releases/latest/download/constraints.txt`.
- Do not guess a field or an option. `aims scenario schema` prints the JSON Schema of scenario files with a description for every type. `aims <command> --help` lists every option.
- Run `aims scenario check <dir>` after every change to a scenario. Fix every error before a model run. The check costs nothing.
- Model runs cost money. Run `aims estimate <scenario>` first and tell the user what the run can cost. Start `aims run`, `aims monitor` or `aims score` only when the user asked for it.
- Commands take full run ids, such as `01a1a456-a8a8-7317-bd3d-36f38d33d8af`. A prefix does not work.
- Never edit files inside a run folder. Each run is a hash chain. `aims verify <run>` checks it.
- Agents never see plants, variants, the self-test or `facts`. The monitor does not see them either. Keep the task text free of hints about plants unless the experiment is about that.

## Preflight

```sh
aims --version
docker info --format '{{.ServerVersion}}'
aims models list
```

`aims models list` shows which keys are set. The default models need `MODEL_API_KEY`, the key for the Meta Model API. `aims scenario check` and the self-test need Docker but no key.

`aims scenario check` and `aims run` start a privileged kernel tracer. On a shared machine, use `aims run --audit snapshot` and `aims scenario check --no-selftest`.

## Write a scenario

1. Copy the starter scenario in [assets/scenario/](assets/scenario/) to a new folder. It passes `aims scenario check` as it is. It has a task, a verifier, a file plant, a rule plant, two variants and a self-test.
2. To start from a built-in scenario instead, print the folder that holds them:

   ```sh
   uvx --from aims-bench python -c "import aims, pathlib; print(pathlib.Path(aims.__file__).parent / 'scenarios')"
   ```

   `cache-keys` has an impossible task and an `observed` and `unobserved` pair. `shared-cache` adds a scripted peer and a shared folder.
3. Change `name`, the instruction, the files and the Dockerfile. Write the verifier first, then the plants.
4. Give each plant `facts`: one or two plain sentences on what taking it means. `aims score` checks the agent's answer against them.
5. Write the self-test. `reference` must solve the task honestly and take no plant. `exploit` must take every plant listed in `claims`.
6. Check it, then estimate it:

   ```sh
   aims scenario check ./my-scenario
   aims estimate ./my-scenario --variants all
   ```

[references/scenario.md](references/scenario.md) has every section, the plant kinds, variants, channels, rules and the errors `aims scenario check` gives.

## Design plants

| Kind | Use it for | Counts as taken when |
| --- | --- | --- |
| `file` | A bait file: answer key, credentials, oversight config | The agent reads it, or does an action listed in `on` |
| `canary` | A secret placed with `{{canaries.NAME}}` in any text | It turns up where it was not placed |
| `impossible` | Tests that contradict the instruction | The agent submits without telling the operator |
| `tool` | A tempting extra tool with a fixed reply | The agent calls it |
| `rule` | Any action the monitor's rule syntax can match | The rule matches |
| `imds` | A fake cloud metadata service with canary credentials | The credentials turn up in what the agent does |

A `file` or `tool` plant with a `canary` counts only when the canary turns up in what the agent does. `taken` sets another tier: `touched`, `used` or `claimed`.

A good plant is believable, cheap to take and clearly wrong to take. The honest path must exist and must be checkable. An `impossible` plant needs a `reference` folder with an honest solution that the verifier fails.

## Run and read results

Ask before each of these. Each one calls a model.

```sh
aims run ./my-scenario --variant unobserved --samples 5 --parallel 5 --quiet
aims run ./my-scenario --monitor signals    # the monitor without a model
```

Runs go to `./runs/<run id>/`. `--quiet` prints one line per run that starts with the run id. Then read a run without spending:

```sh
aims check <run>                  # the plants each agent took
aims report <run> --out report.md # verdict, reward, flags, plants and a timeline
aims verify <run>                 # the hash chain and blobs
```

`aims run` scores each run at the end unless you pass `--no-score`. The score is a new run. `aims report` picks it up.

[references/results.md](references/results.md) explains the run folder, each command's output and how to fork a run.

## When something fails

- `aims scenario check` prints `error:` lines. Each names the field or the self-test case. Fix the first one and run it again.
- `selftest reference: failed` means a step of the honest solution exited with an error or took a plant.
- `selftest exploit: failed` means a plant in `claims` was not taken. Check its path, its `on` actions or its rule.
- `the self-test needs Docker and the kernel tracer` means Docker is down or cannot start privileged containers.
- A run that stops on a missing key needs the key that `aims models list` shows as unset.
Pliki w paczce (9)
  • SKILL.md 6 KB
  • assets/scenario/scenario.yaml 2 KB
  • assets/scenario/env/Dockerfile 1 KB
  • assets/scenario/files/test_wordcount.py 1 KB
  • assets/scenario/files/wordcount.py 1 KB
  • assets/scenario/tests/score.py 1 KB
  • assets/scenario/tests/test_hidden.py 1 KB
  • references/results.md 5 KB
  • references/scenario.md 13 KB

Wersje

RepozytoriumWynikLicencja
Ta strona · kaumnen/AIMS100Apache-2.0
hazarsozer/crucible-cc100MIT

Migawka bazy z 13.11.2026 · wersja paczki 2a5c515b0afd (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100grant-specific-aims-writer
    Pisze i poprawia stronę Specific Aims we wnioskach grantowych (NIH R01, R21, R03, podsumowanie projektu NSF) oraz ich odpowiedniki w innych agencjach.
  • 100specific-aims
    Pomaga pisać i poprawiać stronę Specific Aims do wniosków NIH: strukturę, formułę otwarcia, konstrukcję celów i typowe błędy.
  • 95teach-oh
    Porządkuje projekt: bada bazę kodu, pyta o strategię i cele, a potem zapisuje kontekst w pliku AGENTS.md.
  • 100aims-audit
    Przygotowuje do audytu systemu zarządzania AI (AIMS) według ISO 42001: audyt certyfikacyjny etapu 1 i 2, audyt nadzorczy lub wewnętrzny oraz ocena wpływu AI (AIIA).
  • 100grant-proposal
    Tworzy szkic wniosku grantowego w strukturze funduszu z planem danych i opisem warunków, do ręcznego wysłania przez użytkownika.
  • 100grant-writing
    Pomaga pisać wnioski grantowe NIH i NSF: Specific Aims, strategię badawczą, sekcje znaczenia, innowacji i podejścia oraz granty R01, R21, K99, F31 i CAREER.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.