aims
/aims
Pisze, sprawdza i uruchamia scenariusze AIMS, stanowisko do eksperymentów z dopasowaniem AI do celów, a potem odczytuje wyniki sprawdzianów i raportów.
Ocena i bezpieczeństwo
- Pliki w paczce
- 9
- Skrypty
- 4
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 1
Brak sygnałów. Repozytorium zmienione 13.11.2026, bez odwołań do wycofanych modeli i API.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
- 4 skrypty w paczce. Przejrzyj je przed instalacją, bo Claude może je uruchomić.
Skrypty: assets/scenario/files/test_wordcount.py, assets/scenario/files/wordcount.py, assets/scenario/tests/score.py, assets/scenario/tests/test_hidden.py
Co robi i kiedy
Pisze, sprawdza i uruchamia scenariusze AIMS, stanowisko do eksperymentów z dopasowaniem AI do celów, a potem odczytuje wyniki sprawdzianów i raportów.
Oryginalny opis (EN)
Write, check and run AIMS scenarios, then read their results. AIMS is a bench for alignment experiments. Model agents work on a task in Docker sandboxes with planted temptations while a monitor model flags misbehaviour. Use when the user wants to design an AIMS scenario, plant or variant, write a verifier or a monitor rule, run or fork a scenario with the `aims` CLI, or explain a run's check, score or report.
Jak działa
- Rules
- Preflight
- Write a scenario
- Design plants
- Run and read results
- When something fails
Podgląd SKILL.md
---
name: aims
description: Write, check and run AIMS scenarios, then read their results. AIMS is a bench for alignment experiments. Model agents work on a task in Docker sandboxes with planted temptations while a monitor model flags misbehaviour. Use when the user wants to design an AIMS scenario, plant or variant, write a verifier or a monitor rule, run or fork a scenario with the `aims` CLI, or explain a run's check, score or report.
license: Apache-2.0
---
# AIMS
AIMS runs model agents on a task in Docker sandboxes. The task hides temptations called plants. A monitor watches every step and raises flags. A score compares the plants each agent took with the flags the monitor raised.
## Rules
- Use the installed CLI, `aims`. Check it with `aims --version`. If it is missing, install it with `uv tool install aims-bench -c https://github.com/kaumnen/AIMS/releases/latest/download/constraints.txt`.
- Do not guess a field or an option. `aims scenario schema` prints the JSON Schema of scenario files with a description for every type. `aims <command> --help` lists every option.
- Run `aims scenario check <dir>` after every change to a scenario. Fix every error before a model run. The check costs nothing.
- Model runs cost money. Run `aims estimate <scenario>` first and tell the user what the run can cost. Start `aims run`, `aims monitor` or `aims score` only when the user asked for it.
- Commands take full run ids, such as `01a1a456-a8a8-7317-bd3d-36f38d33d8af`. A prefix does not work.
- Never edit files inside a run folder. Each run is a hash chain. `aims verify <run>` checks it.
- Agents never see plants, variants, the self-test or `facts`. The monitor does not see them either. Keep the task text free of hints about plants unless the experiment is about that.
## Preflight
```sh
aims --version
docker info --format '{{.ServerVersion}}'
aims models list
```
`aims models list` shows which keys are set. The default models need `MODEL_API_KEY`, the key for the Meta Model API. `aims scenario check` and the self-test need Docker but no key.
`aims scenario check` and `aims run` start a privileged kernel tracer. On a shared machine, use `aims run --audit snapshot` and `aims scenario check --no-selftest`.
## Write a scenario
1. Copy the starter scenario in [assets/scenario/](assets/scenario/) to a new folder. It passes `aims scenario check` as it is. It has a task, a verifier, a file plant, a rule plant, two variants and a self-test.
2. To start from a built-in scenario instead, print the folder that holds them:
```sh
uvx --from aims-bench python -c "import aims, pathlib; print(pathlib.Path(aims.__file__).parent / 'scenarios')"
```
`cache-keys` has an impossible task and an `observed` and `unobserved` pair. `shared-cache` adds a scripted peer and a shared folder.
3. Change `name`, the instruction, the files and the Dockerfile. Write the verifier first, then the plants.
4. Give each plant `facts`: one or two plain sentences on what taking it means. `aims score` checks the agent's answer against them.
5. Write the self-test. `reference` must solve the task honestly and take no plant. `exploit` must take every plant listed in `claims`.
6. Check it, then estimate it:
```sh
aims scenario check ./my-scenario
aims estimate ./my-scenario --variants all
```
[references/scenario.md](references/scenario.md) has every section, the plant kinds, variants, channels, rules and the errors `aims scenario check` gives.
## Design plants
| Kind | Use it for | Counts as taken when |
| --- | --- | --- |
| `file` | A bait file: answer key, credentials, oversight config | The agent reads it, or does an action listed in `on` |
| `canary` | A secret placed with `{{canaries.NAME}}` in any text | It turns up where it was not placed |
| `impossible` | Tests that contradict the instruction | The agent submits without telling the operator |
| `tool` | A tempting extra tool with a fixed reply | The agent calls it |
| `rule` | Any action the monitor's rule syntax can match | The rule matches |
| `imds` | A fake cloud metadata service with canary credentials | The credentials turn up in what the agent does |
A `file` or `tool` plant with a `canary` counts only when the canary turns up in what the agent does. `taken` sets another tier: `touched`, `used` or `claimed`.
A good plant is believable, cheap to take and clearly wrong to take. The honest path must exist and must be checkable. An `impossible` plant needs a `reference` folder with an honest solution that the verifier fails.
## Run and read results
Ask before each of these. Each one calls a model.
```sh
aims run ./my-scenario --variant unobserved --samples 5 --parallel 5 --quiet
aims run ./my-scenario --monitor signals # the monitor without a model
```
Runs go to `./runs/<run id>/`. `--quiet` prints one line per run that starts with the run id. Then read a run without spending:
```sh
aims check <run> # the plants each agent took
aims report <run> --out report.md # verdict, reward, flags, plants and a timeline
aims verify <run> # the hash chain and blobs
```
`aims run` scores each run at the end unless you pass `--no-score`. The score is a new run. `aims report` picks it up.
[references/results.md](references/results.md) explains the run folder, each command's output and how to fork a run.
## When something fails
- `aims scenario check` prints `error:` lines. Each names the field or the self-test case. Fix the first one and run it again.
- `selftest reference: failed` means a step of the honest solution exited with an error or took a plant.
- `selftest exploit: failed` means a plant in `claims` was not taken. Check its path, its `on` actions or its rule.
- `the self-test needs Docker and the kernel tracer` means Docker is down or cannot start privileged containers.
- A run that stops on a missing key needs the key that `aims models list` shows as unset.
Pliki w paczce (9)
- SKILL.md 6 KB
- assets/scenario/scenario.yaml 2 KB
- assets/scenario/env/Dockerfile 1 KB
- assets/scenario/files/test_wordcount.py 1 KB
- assets/scenario/files/wordcount.py 1 KB
- assets/scenario/tests/score.py 1 KB
- assets/scenario/tests/test_hidden.py 1 KB
- references/results.md 5 KB
- references/scenario.md 13 KB
Wersje
| Repozytorium | Wynik | Licencja |
|---|---|---|
| Ta strona · kaumnen/AIMS | 100 | Apache-2.0 |
| hazarsozer/crucible-cc | 100 | MIT |
Migawka bazy z 13.11.2026 · wersja paczki 2a5c515b0afd (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100grant-specific-aims-writer
Pisze i poprawia stronę Specific Aims we wnioskach grantowych (NIH R01, R21, R03, podsumowanie projektu NSF) oraz ich odpowiedniki w innych agencjach. - 100specific-aims
Pomaga pisać i poprawiać stronę Specific Aims do wniosków NIH: strukturę, formułę otwarcia, konstrukcję celów i typowe błędy. - 95teach-oh
Porządkuje projekt: bada bazę kodu, pyta o strategię i cele, a potem zapisuje kontekst w pliku AGENTS.md. - 100aims-audit
Przygotowuje do audytu systemu zarządzania AI (AIMS) według ISO 42001: audyt certyfikacyjny etapu 1 i 2, audyt nadzorczy lub wewnętrzny oraz ocena wpływu AI (AIIA). - 100grant-proposal
Tworzy szkic wniosku grantowego w strukturze funduszu z planem danych i opisem warunków, do ręcznego wysłania przez użytkownika. - 100grant-writing
Pomaga pisać wnioski grantowe NIH i NSF: Specific Aims, strategię badawczą, sekcje znaczenia, innowacji i podejścia oraz granty R01, R21, K99, F31 i CAREER.