Skip to content
Skills Library
AI & agents · Testing

cognitive-testing

/cognitive-testing

Test cognitive and agentic loops by their mechanisms and invariants rather than by output-equality. Use when: cognitive testing, invariant testing, convergence testing, episodic memory integrity, shadow testing, loop guard, brain mode testing.

Repository
gabbe ↗
Licence
CC-BY-4.0
Last repository change
24.10.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 24.10.2026, without references to retired models or APIs.

Licence

CC-BY-4.0, otwarta.

What it does and when to use it

Test cognitive and agentic loops by their mechanisms and invariants rather than by output-equality. Use when: cognitive testing, invariant testing, convergence testing, episodic memory integrity, shadow testing, loop guard, brain mode testing.

Original author description

Test cognitive and agentic loops by their mechanisms and invariants rather than by output-equality. Use when: cognitive testing, invariant testing, convergence testing, episodic memory integrity, shadow testing, loop guard, brain mode testing.

How it works

  1. Goal
  2. Steps
  3. Constraints
  4. Output Format
  5. Security & Guardrails
Original SKILL.md headings · 55 lines, 693 words
SKILL.md preview
---
name: cognitive-testing
description: "Test cognitive and agentic loops by their mechanisms and invariants rather than by output-equality. Use when: cognitive testing, invariant testing, convergence testing, episodic memory integrity, shadow testing, loop guard, brain mode testing."
license: CC-BY-SA-4.0
metadata:
  source: "agents/skills/brain/cognitive-testing.skill.md"
  tags: "brain"
  core: "false"
  context_cost: "medium"
---
# Cognitive Testing Skill

## Goal
Test cognitive/agentic loops (Brain Mode active-inference, Loki swarm orchestration) by their MECHANISMS and INVARIANTS rather than by output-equality. You cannot predict what an autonomous loop decides, but you can BOUND it: assert the invariants it must never violate, the convergence it should exhibit, the memory it must retrieve faithfully, and the guards that must fire under stress.

## Steps

1. **Invariant-bounded state verification**
   - Assert mutually-exclusive states never co-exist (e.g. a workspace cannot be both `DEV_READY` and `INFRA_MISSING`).
   - Assert budget/cost invariants always hold (spend never exceeds the cap).
   - Assert illegal state transitions never occur (no jump that skips a required gate).

2. **Convergence testing**
   - For an active-inference / error-minimizing loop, assert prediction error is non-increasing / converges over N iterations.
   - Test the toy demo HONESTLY: GABBE's production `brain.py` uses epsilon-greedy gene selection plus a monotonic `success_rate`, NOT literal free-energy math. Convergence claims therefore apply to the CONCEPTUAL / toy loop only, never to the production engine.

3. **Episodic-memory integrity**
   - Under rapid context switching, assert the resume pointer retrieves the EXACT last state — no fabricated or hallucinated history, no drift.

4. **Shadow / sandbox testing**
   - Run the orchestrator against a MOCK `state.db`. Inject a failure and assert it is detected, that self-heal is triggered, and that baseline state is restored.
   - Shadow runs never touch real state.

5. **Resource-exhaustion / loop guards**
   - Assert the agent escalates to a human if it repeats an identical tool call 3 times or if a confidence score drops below threshold.
   - Assert hard iteration / depth / timeout caps fire as designed.

## Constraints
- These tests BOUND behavior; they do not prove the cognition is "correct."
- Keep convergence claims scoped to the conceptual loop, not the production engine (epsilon-greedy + monotonic success_rate, not free-energy).
- Shadow tests MUST use mocks and MUST NEVER touch real state.
- Cross-link `pbt-strategy.skill` for expressing invariants via Hypothesis and `chaos-fault-injection.skill` for fault recipes used in shadow/failure-injection tests.

## Output Format
A cognitive-test report containing:
- **Invariants checked**: which state, budget, and transition invariants were asserted and their pass/fail.
- **Convergence**: the prediction-error delta over N steps (scoped to the conceptual loop).
- **Episodic integrity**: whether the resume pointer retrieved the exact last state.
- **Shadow-test outcome**: failure injected, detected, self-healed, baseline restored.
- **Guard-trigger results**: loop-detection, confidence-threshold, and hard-cap firings.

## Security & Guardrails

### 1. Skill Security (Cognitive Testing)
- **Episodic-memory poisoning**: Episodic memory is an injection sink — a poisoned memory replayed on resume becomes a persistent instruction the loop trusts as its own history. Memories MUST be sanitized before write, and the integrity test in Step 3 MUST assert the resume pointer rejects fabricated history rather than merely matching the happy path.
- **Convergence-claim overreach**: A test that asserts "free-energy convergence" against the production engine misrepresents what `brain.py` does and gives false assurance. The skill MUST keep convergence assertions scoped to the conceptual/toy loop and label them as such in the report, never implying the production engine minimizes free energy.

### 2. System Integration Security
- **Shadow tests hitting production resources**: A shadow test misconfigured against the real `state.db`, live APIs, or production budget can corrupt state or spend real money while injecting failures. Mock isolation MUST be enforced structurally (separate connection/credentials, no production endpoints reachable from the test harness), failing closed if a real resource is detected.
- **Fault-injection blast radius**: Faults injected in shadow runs (per `chaos-fault-injection.skill`) must be contained to the sandbox. The harness MUST verify the mock boundary before injecting and MUST NOT propagate injected failures to shared or live infrastructure.

### 3. LLM & Agent Guardrails
- **Runaway loops exhausting budget**: An autonomous loop without guards can spin indefinitely, exhausting token/cost budget — a self-inflicted denial of service. The skill MUST assert hard iteration/depth/timeout caps and identical-call loop detection (escalate to human after 3 repeats), and these guards MUST fail closed.
- **Bounding is not correctness**: Passing invariant, convergence, and guard tests bounds behavior but does NOT prove the cognition is correct or safe to run unsupervised. High-impact autonomous actions still require human approval per `ai-safety-guardrails.skill` and `prompt-injection-defense.skill`.

Versions

The database has no other skills named cognitive-testing.

Database snapshot: 13.11.2026 · package version d35418807010 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100brain-mode
    Meta-cognitive orchestrator — adds Active Inference, cost-routing, and episodic memory to the SDLC loop (System-2 planning). Use when: brain activate, supermode, solve everything, brain mode, meta orchestrate.
  • 100cognitive-update
    Update GOAL_STACK, BELIEF_STATE, SELF_MODEL, and PROCEDURES from explicit evidence. Use when updating cognitive state or runtime state.
  • 100cognitive-architectures
    Patterns from SOAR, ACT-R, and LIDA for advanced agent cognitive cycles. Use when: cognitive, architecture, soar, act-r, lida, reasoning cycle.
  • 100cognitive-routing
    Use when Cognitive Routing is active: the Coordinator plans and delegates work to Helper or Executor profiles, and workers carry out and report their assignments.
  • 100testing-llm
    LLM testing patterns: mocked responses, DeepEval and RAGAS evaluation, structured output checks, agentic tests. Use when testing AI features.
  • 90syncolab-cognitive-memory
    Guides Syncolab Cognitive Memory Layer (CML)—Qdrant + Mem0 composite, tenant scope, REST memory endpoints, and TS agent recall/persist. Use for long-term memory hydration and external runtime memory_layer contracts.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.