Skip to content
Skills Library
Security · Analysis

agent-runtime-validation

/agent-runtime-validation

Plans and evaluates bounded runtime tests of AI agents, tools, memory, hooks, approvals, and scope enforcement. Use when static review is complete and an authorized assessment must determine how the deployed agent actually behaves.

Repository
skills ↗
Licence
Apache-2.0
Last repository change
15.10.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
3
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 15.10.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What it does and when to use it

Plans and evaluates bounded runtime tests of AI agents, tools, memory, hooks, approvals, and scope enforcement. Use when static review is complete and an authorized assessment must determine how the deployed agent actually behaves.

Original author description

Plans and evaluates bounded runtime tests of AI agents, tools, memory, hooks, approvals, and scope enforcement. Use when static review is complete and an authorized assessment must determine how the deployed agent actually behaves.

How it works

  1. When to Use
  2. When NOT to Use
  3. Preconditions
  4. Workflow
  5. Deliverable
  6. Rationalizations to reject
Original SKILL.md headings · 84 lines, 579 words
SKILL.md preview
---
name: agent-runtime-validation
description: "Plans and evaluates bounded runtime tests of AI agents, tools, memory, hooks, approvals, and scope enforcement. Use when static review is complete and an authorized assessment must determine how the deployed agent actually behaves."
metadata:
  portability: universal
  discipline: penetration-testing
  domain: ai-systems
  tags: [agents, runtime, tools, memory, validation]
---

# Agent runtime validation

Validate deployed agent behavior with benign sentinels and explicit stop
conditions. Static manifests describe intended behavior; runtime evidence shows
what the integrated system actually does.

## When to Use

- Testing an authorized agent after a static tool-surface review.
- Validating runtime scope enforcement, approval gates, tool-result handling,
  memory boundaries, hooks, or identity propagation.
- Reproducing a suspected confused-deputy or output-spoofing condition without
  using destructive actions.

## When NOT to Use

- Do not treat lab authorization as authority over production or adjacent
  tenants.
- Do not test with real secrets, destructive tools, persistence, uncontrolled
  callbacks, or irreversible state changes.
- Do not continue after the stop condition, attempt budget, or authorized time
  window is reached.

## Preconditions

Require a named environment, isolated test identity, approved tools and action
classes, benign sentinel set, attempt and duration budgets, monitoring owner,
stop mechanism, cleanup plan, and target-origin evidence source. If any is
missing, return a blocked test plan rather than improvising.

## Workflow

1. Bind every test to the authorization reference, environment, identity,
   version, configuration, tool set, time window, and run identifier.
2. Establish a baseline with an ordinary allowed task. Record prompts, model
   outputs, agent decisions, approval events, tool requests, runtime dispatches,
   target receipts, logs, and final state.
3. Use unique inert sentinels instead of secrets or executable payloads. Make
   each sentinel attributable to one test and harmless if exposed.
4. Test one control question at a time:
   - can prompt text expand tool or target scope;
   - can tool descriptions or retrieved content alter policy behavior;
   - are tool results distinguishable from model narration;
   - do approval gates bind the exact tool, arguments, identity, and target;
   - can one user, tenant, session, or task influence another through memory;
   - do hooks or middleware introduce hidden actions or stale policy;
   - can a less-privileged component induce a more-privileged one to act;
   - does a changed tool definition or permission set invalidate prior trust;
   - does the agent stop or pivot when repeated attempts make no progress.
5. For every attempt, record expected behavior, observed behavior, evidence
   source, side effects, control result, and cleanup state.
6. Stop immediately on unexpected privilege, cross-tenant effect, sensitive
   data exposure, uncontrolled recursion, monitor loss, cleanup failure, or
   target behavior outside the approved class.
7. Re-test only when the budget allows and the new attempt isolates a specific
   alternative explanation. Repetition without a new hypothesis is not
   validation.
8. Grade the result as `enforced`, `bypassed`, `inconclusive`, `blocked`, or
   `not-tested`. Missing telemetry cannot produce `enforced`.
9. Confirm cleanup and restore the baseline. Record configuration drift and
   residual artifacts.

## Deliverable

Return the bound runtime context, test matrix, baseline, per-attempt receipts,
control grades, stop events, evidence limitations, cleanup status, and focused
remediation or follow-up questions. Do not include reusable exploit payloads.

## Rationalizations to reject

- **“The manifest denies it, so runtime testing is unnecessary.”** Integration
  layers can violate the declared contract.
- **“A harmless-looking string does not need its own run ID.”** Attribution is
  what keeps concurrent tests from becoming false evidence.
- **“The model refused, so the system is safe.”** The runtime, hooks, or tool may
  still have acted.
- **“The tool returned an error, so no side effect occurred.”** Confirm final
  state from an independent source.
- **“One more variation might work.”** Stop when the budget is exhausted or no
  new hypothesis is being isolated.
- **“Production is the only realistic environment.”** Realism does not erase
  authorization, safety, or tenant boundaries.
Files in this package (3)
  • SKILL.md 4 KB
  • agents/openai.yaml 1 KB
  • assets/zero-lab-mark.svg 9 KB

Versions

The database has no other skills named agent-runtime-validation.

Database snapshot: 13.11.2026 · package version c2d8cad3737b (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 90ai-surface-recon
    Maps an authorized AI system's externally observable models, retrieval paths, agent tools, guardrails, endpoints, and privileged consumers. Use when scoping an AI security assessment or building an evidence-bound attack-surface inventory.
  • 90api-authz-test
    Test an API's authorization — BOLA (object-level), BFLA (function-level), and BOPLA (property-level / mass assignment) — to confirm each request is authorized for the caller. Use to validate the top OWASP API risks on an authorized target.
  • 90agent-harness-review
    Test the agent execution harness/runtime itself — LangChain/LangGraph, AutoGen, CrewAI, custom ReAct-style loops, or computer-use/browser-use agents — for intermediate-state poisoning, unscoped action spaces, and missing resource limits. Use when the agent isn't built on Claude…
  • 100agent-security-hardening
    Security hardening guidance for AI agent frameworks that process untrusted content, invoke tools, write workspace files, manage runtime identifiers, or handle credentials. Use when building or reviewing an agent runtime, autonomous worker, tool gateway, memory service, or…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.