monitor-experiment
/monitor-experiment
Monitoruje trwające eksperymenty, sprawdza postęp i zbiera wyniki, gdy użytkownik pyta, czy coś się już skończyło.
- Repozytorium
- Athena ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 13.11.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 1
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 2
Brak sygnałów. Repozytorium zmienione 13.11.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
Co robi i kiedy
Monitoruje trwające eksperymenty, sprawdza postęp i zbiera wyniki, gdy użytkownik pyta, czy coś się już skończyło.
Wyzwalacze z opisu
Argumenty
/monitor-experiment [server-alias or screen-name]
Oryginalny opis (EN)
Monitor running experiments, check progress, collect results. Use when user says "check results", "is it done", "monitor", or wants experiment output.
Jak działa
- Workflow
- Key Rules
- Mirror description (additional to this open warehouse)
Podgląd SKILL.md
---
name: monitor-experiment
description: Monitor running experiments, check progress, collect results. Use when user says "check results", "is it done", "monitor", or wants experiment output.
argument-hint: "[server-alias or screen-name]"
allowed-tools: Bash(ssh *), Bash(echo *), Read, Write, Edit
---
# Monitor Experiment Results
> ⏱ **External cadence is appropriate here.** This skill waits on an external
> fact (job completion / progress), so it is a natural `/loop` / `CronCreate`
> surface: the wake reads status and self-judges only **machine-checkable**
> completion (exit code, file exists, epoch logged) — never quality. This is
> the additive external-wait shape in
> [`shared-references/external-cadence.md`](../shared-references/external-cadence.md).
> If a scheduled wait here ends in a verdict step (e.g. then audit results),
> run that verdict **once** after the wait clears — not re-entered per tick.
Monitor: $ARGUMENTS
## Workflow
### Step 1: Check What's Running
**SSH server:**
```bash
ssh <server> "screen -ls"
```
**Vast.ai instance** (read `ssh_host`, `ssh_port` from `vast-instances.json`):
```bash
ssh -p <PORT> root@<HOST> "screen -ls"
```
Also check vast.ai instance status:
```bash
vastai show instances
```
**Modal** (when `gpu: modal` in CLAUDE.md):
```bash
modal app list # List running/recent apps
modal app logs <app> # Stream logs from a running app
```
Modal apps auto-terminate when done — if it's not in the list, it already finished. Check results via `modal volume ls <volume>` or local output.
### Step 2: Collect Output from Each Screen
For each screen session, capture the last N lines:
```bash
ssh <server> "screen -S <name> -X hardcopy /tmp/screen_<name>.txt && tail -50 /tmp/screen_<name>.txt"
```
If hardcopy fails, check for log files or tee output.
### Step 3: Check for JSON Result Files
```bash
ssh <server> "ls -lt <results_dir>/*.json 2>/dev/null | head -20"
```
If JSON results exist, fetch and parse them:
```bash
ssh <server> "cat <results_dir>/<latest>.json"
```
### Step 3.5: Pull W&B Metrics (when `wandb: true` in CLAUDE.md)
**Skip this step entirely if `wandb` is not set or is `false` in CLAUDE.md.**
Pull training curves and metrics from Weights & Biases via Python API:
```bash
# List recent runs in the project
ssh <server> "python3 -c \"
import wandb
api = wandb.Api()
runs = api.runs('<entity>/<project>', per_page=10)
for r in runs:
print(f'{r.id} {r.state} {r.name} {r.summary.get(\"eval/loss\", \"N/A\")}')
\""
# Pull specific metrics from a run (last 50 steps)
ssh <server> "python3 -c \"
import wandb, json
api = wandb.Api()
run = api.run('<entity>/<project>/<run_id>')
history = list(run.scan_history(keys=['train/loss', 'eval/loss', 'eval/ppl', 'train/lr'], page_size=50))
print(json.dumps(history[-10:], indent=2))
\""
# Pull run summary (final metrics)
ssh <server> "python3 -c \"
import wandb, json
api = wandb.Api()
run = api.run('<entity>/<project>/<run_id>')
print(json.dumps(dict(run.summary), indent=2, default=str))
\""
```
**What to extract:**
- **Training loss curve** — is it converging? diverging? plateauing?
- **Eval metrics** — loss, PPL, accuracy at latest checkpoint
- **Learning rate** — is the schedule behaving as expected?
- **GPU memory** — any OOM risk?
- **Run status** — running / finished / crashed?
**W&B dashboard link** (include in summary for user):
```
https://wandb.ai/<entity>/<project>/runs/<run_id>
```
> This gives the auto-review-loop richer signal than just screen output — training dynamics, loss curves, and metric trends over time.
### Step 4: Summarize Results
Present results in a comparison table:
```
| Experiment | Metric | Delta vs Baseline | Status |
|-----------|--------|-------------------|--------|
| Baseline | X.XX | — | done |
| Method A | X.XX | +Y.Y | done |
```
### Step 5: Interpret
- Compare against known baselines
- Flag unexpected results (negative delta, NaN, divergence)
- Suggest next steps based on findings
### Step 6: Feishu Notification (if configured)
After results are collected, check `~/.claude/feishu.json`:
- Send `experiment_done` notification: results summary table, delta vs baseline
- If config absent or mode `"off"`: skip entirely (no-op)
## Key Rules
- Always show raw numbers before interpretation
- Compare against the correct baseline (same config)
- Note if experiments are still running (check progress bars, iteration counts)
- If results look wrong, check training logs for errors before concluding
- **Vast.ai cost awareness**: When monitoring vast.ai instances, report the running cost (hours * $/hr from `vast-instances.json`). If all experiments on an instance are done, remind the user to run `/vast-gpu destroy <instance_id>` to stop billing
- **Modal cost awareness**: Modal auto-scales to zero — no idle billing. When reporting results from Modal runs, note the actual execution time and estimated cost (time * $/hr from the GPU tier used). No cleanup action needed
## Mirror description (additional to this open warehouse)
- This document is considered upstream. ARIS mirrors in original;support files (scripts/、references/、../shared-references/)Not distributed with mirrors, full originals can be found at the upstream warehouse https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep 。
- If supporting documents are cited for implementation skills, then the relative path should be parsed from the upstream warehouse.
Wersje
| Repozytorium | Wynik | Licencja |
|---|---|---|
| Ta strona · jiangfeng-strathclyde/Athena | 100 | MIT |
| FOURTEEN1416/academic-agent-toolkit | 100 | CC-BY-4.0 |
| allenai/open-instruct | 100 | MIT |
| llv22/autoresearchwitheyes | 100 | MIT |
| zjunlp/mechanist | 100 | MIT |
Migawka bazy z 13.11.2026 · wersja paczki c27b13b74562 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100aris-monitor-experiment
Monitoruje uruchomione eksperymenty, sprawdza postęp i zbiera wyniki. - 95monitor-experiments
Monitoruje aktywne i zakończone eksperymenty w Amplitude, sortuje je według wagi i przygotowuje raport analizy. - 95auto-experiment
Wdraża eksperymenty z pliku EXPERIMENT_PLAN.md, uruchamia je na GPU i zbiera pierwsze wyniki, gotowe do dalszej weryfikacji. - 100monitor-experiments
Monitoruje aktywne i niedawno zakończone eksperymenty w projektach Amplitude, ocenia ich wagę, analizuje najważniejsze i przygotowuje okresowy raport o stanie. - 100monitor-experiments
Monitoruje aktywne i niedawno zakończone eksperymenty w projektach Amplitude, ocenia ich wagę, a następnie głębiej analizuje najważniejsze i przygotowuje raport. - 95analyze-experiment-consolidated
Projektuje testy A/B z metrykami i wariantami, analizuje trwające lub zakończone eksperymenty i interpretuje wyniki, wspierając decyzje o wdrożeniu.