Skip to content
Skills Library
AI & agents · Analysis

train

/train

HSLM training analytics dashboard — live process status, checkpoint loss curve, model architecture, platform benchmarks, FPGA status, and scientific metrics. Use when checking training progress, loss convergence, or preparing paper-ready results.

Repository
trinity ↗
Licence
MIT
Last repository change
27.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
1
Deprecation

No signals detected. Repository changed on 27.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

HSLM training analytics dashboard — live process status, checkpoint loss curve, model architecture, platform benchmarks, FPGA status, and scientific metrics. Use when checking training progress, loss convergence, or preparing paper-ready results.

Arguments

/train [focus] (status, loss, bench, fpga, paper, all)
Original author description

HSLM training analytics dashboard — live process status, checkpoint loss curve, model architecture, platform benchmarks, FPGA status, and scientific metrics. Use when checking training progress, loss convergence, or preparing paper-ready results.

How it works

  1. Step 0: Auto-Monitoring Check (REQUIRED)
  2. Step 1: Show the Dashboard (REQUIRED)
  3. Step 2: Wave 9 Status (REQUIRED) — S3 MultiObj
  4. Step 3: Your Analysis (after dashboard)
  5. Step 4: Additional Data (if $ARGUMENTS specified)
  6. Step 5: Telegram Broadcast (REQUIRED)
Original SKILL.md headings · 153 lines, 755 words
SKILL.md preview
---
name: train
description: HSLM training analytics dashboard — live process status, checkpoint loss curve, model architecture, platform benchmarks, FPGA status, and scientific metrics. Use when checking training progress, loss convergence, or preparing paper-ready results.
argument-hint: [focus] (status, loss, bench, fpga, paper, all)
allowed-tools: Bash(tri *), Bash(cat *), Bash(ls *), Bash(tail *), Bash(date *), Read, Grep, Glob
context: fork
---

# HSLM Training Observatory v2

## Step 0: Auto-Monitoring Check (REQUIRED)

First, check if monitoring loop is running:

```bash
# Check if 15-min monitoring loop is active
crontab -l 2>/dev/null | grep -q "tri train" && echo "✅ Мониторинг активен" || echo "⚠️ Мониторинг НЕ настроен"
```

If NOT active, offer to set up:
```
⚠️ Автомониторинг не запущен. Для мониторинга каждые 15 минут:
/loop 15m /train
```

## Step 1: Show the Dashboard (REQUIRED)

Run and **display the FULL output to the user as-is**:

!`cd /Users/playra/trinity-w1 && ./zig-out/bin/tri train dashboard 2>&1`

**CRITICAL**: The output of `tri train dashboard` IS the report. Show it COMPLETELY. Do NOT summarize, do NOT rephrase, do NOT hide it. The user wants to SEE the ANSI dashboard output directly.

### Sacred Workers Status (ALWAYS show)

After dashboard, show sacred workers explicitly:

```bash
cd /Users/playra/trinity-w1 && ./zig-out/bin/tri train dashboard 2>&1 | grep -E "(hslm-r6|hslm-r33|hslm-r5|hslm-r12|hslm-r13|hslm-r11|hslm-r18|hslm-w7-50)" | head -10
```

## Step 2: Wave 9 Status (REQUIRED) — S3 MultiObj

**CRITICAL**: Wave 9 is the current active wave with S3 MultiObj profile.

### S3 MultiObj Configuration (ALL 48 services MUST use):

```
HSLM_PROFILE = s3-multiobj
HSLM_CTX = 81
HSLM_NTP_WEIGHT = 0.50
HSLM_JEPA_WEIGHT = 0.25
HSLM_NCA_WEIGHT = 0.25
HSLM_CRASH_TOLERANCE = 0.05
HSLM_WAVE = 9
```

Query ALL 8 Railway accounts (FARM-2 through FARM-12) for wave 9 deployment:
```bash
set -a && source /Users/playra/trinity-w1/.env && set +a
for ACCT in 7 8; do
  TOKEN_VAR="RAILWAY_API_TOKEN_$ACCT"
  TOKEN="${!TOKEN_VAR}"
  PID_VAR="RAILWAY_PROJECT_ID_$ACCT"
  PID="${!PID_VAR}"
  echo "=== FARM-$ACCT ==="
  curl -s https://railway.com/graphql/v2 \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d "{\"query\":\"query { project(id: \\\"$PID\\\") { services { edges { node { name serviceInstances { edges { node { startCommand builder latestDeployment { status } } } } } } } } }\"}" 2>/dev/null | python3 -c "
import sys,json
d=json.load(sys.stdin)
svcs=d['data']['project']['services']['edges']
building=running=failed=idle=0
for s in svcs:
    n=s['node']
    inst=n['serviceInstances']['edges']
    st='NO_DEPLOY'
    if inst:
        dep=inst[0]['node'].get('latestDeployment')
        if dep: st=dep.get('status','?')
    if st in ('BUILDING','DEPLOYING','INITIALIZING'): building+=1
    elif st in ('SUCCESS',): running+=1
    elif st in ('FAILED','CRASHED'): failed+=1
    else: idle+=1
    print(f'  {n[\"name\"]}: {st}')
print(f'  --- {len(svcs)} total: {running} running, {building} building, {failed} failed, {idle} other')
" 2>/dev/null
done
```

Display wave 9 table:
```
### Wave 9 — 48 Workers (ALL 8 Railway accounts)
| Account | Workers | Building | Running | Failed | S3-Ready |
|---------|---------|----------|---------|--------|----------|
| FARM-2  | 6       | ...      | ...     | ...    | ... |
| FARM-3  | 6       | ...      | ...     | ...    | ... |
| FARM-8  | 6       | ...      | ...     | ...    | ... |
| FARM-9  | 6       | ...      | ...     | ...    | ... |
| FARM-10 | 6       | ...      | ...     | ...    | ... |
| FARM-11 | 6       | ...      | ...     | ...    | ... |
| FARM-12 | 6       | ...      | ...     | ...    | ... |
| ...     | ...     | ...      | ...     | ...    | ... |
| TOTAL   | 48      | ...      | ...     | ...    | ... |

Config: S3 MultiObj (NTP 50%, JEPA 25%, NCA 25%), ctx=81, crash_tol=5%
```

## Step 3: Your Analysis (after dashboard)

After showing the full dashboard + wave 9 table, add a SHORT (5-10 lines) analytical block in Russian:

```
### 📊 Аналитика [UTC: $(date -u +%H:%M)]

{emoji} **Лидер**: {name} PPL={val} @ {step} — {insight}
{emoji} **S3 MultiObj**: {count}/{total} активны — {status} (NTP 50%, JEPA 25%, NCA 25%)
{emoji} **Stalled**: {count} воркеров — {action if any}
{emoji} **Эволюция**: Step {step} — {kills} kills за сессию
{emoji} **Волна 7**: {best} w7-50 PPL={val} — {insight}

**Вердикт**: {one sentence — act or wait}
```

Rules:
- Compare leader with R6 PPL=28.07 (current king)
- Flag S3 workers that are STALLED
- Count stalled total vs previous run
- Note which objective (NTP/NCA/JEPA) is winning
- Say the phase: EARLY (<10K) / MIDDLE (10-50K) / LATE (50K+)
- Report wave 9 build progress (how many building/running/failed)
- Max 10 lines. Dense. No fluff.

### S3 Workers Alert

If ANY S3 MultiObj worker is STALLED, add:

```
🚨 **СТОП! S3 воркер {name} stalled на {step} шагов!**
   Действие: railway restart --service {name}
   Профиль: S3 MultiObj (NTP 50%, JEPA 25%, NCA 25%)
```

## Step 4: Additional Data (if $ARGUMENTS specified)

### If focus=fpga:
!`ls -lh /Users/playra/trinity-w1/fpga/openxc7-synth/hslm_full_top.bit 2>/dev/null && echo "Bitstream ready" || echo "No bitstream"`

### If focus=paper:
Reference: R33 PPL=4.6 (verified), R18 PPL=6.1 (MIRAGE), R19 PPL=2.04 (UNVERIFIED)

## Step 5: Telegram Broadcast (REQUIRED)

Compose a SHORT narration in Russian (3-5 sentences, Slack tone).

Set `TG_TEXT` to the narration.
Set `TG_MODE=dedup`, `TG_DEDUP_FILE=.trinity/tg_dedup_train.hash`.
Then execute the shared Telegram template from `.claude/skills/_shared/telegram.md`.

Versions

RepositoryScoreLicence
This page · trinity-s-ai-dna/trinity100MIT
octavehq/lfgtm100MIT
Kiran-980/1ml90MIT

Database snapshot: 13.11.2026 · package version e9072b2ff777 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 90fit-train
    Design a model training pipeline — algorithm selection, cross-validation, and serialization. Use when asked to "train a model for this", "design a training pipeline", or "which algorithm should we use".
  • 100training-check
    Periodically check WandB metrics during training to catch problems early (NaN, loss divergence, idle GPUs). Avoids wasting GPU hours on broken runs. Use when training is running and you want automated health checks.
  • 100training-check
    Periodically check WandB metrics during training to catch problems early (NaN, loss divergence, idle GPUs). Avoids wasting GPU hours on broken runs. Use when training is running and you want automated health checks.
  • 87jackframework
    How to build on, launch, and debug JackFramework (JF) — a PyTorch training framework that owns the training loop, DDP launcher, checkpoint I/O and progress bar while you supply a model-interface class, a dataloader class and a launcher. Use this whenever a task touches JF hooks…
  • 100mlops-mlflow-experiment-tracking
    Use this skill when logging hyperparameters, training loss metrics, evaluation curves, and model artifacts with MLflow.
  • 100model-checkpoint-manager
    Manage model checkpoint manager operations. Auto-activating skill for ML Training. Triggers on: model checkpoint manager, model checkpoint manager Part of the ML Training skill category. Use when working with model checkpoint manager functionality. Trigger with phrases like…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.