Skip to content
Skills Library
Finance & accounting · Testing

risk-control-unit-testing-framework

/risk-control-unit-testing-framework

Use before promoting a change to a pre-trade risk control, to run positive, boundary, breach and malformed-input cases and check the engine still rejects exactly what it should, matching triggered rule ids.

Licence
Apache-2.0
Last repository change
14.09.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
6
Scripts
2
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 14.09.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What to check before installing
  • 2 scripts in this package. Review them before installing, as Claude may execute them.

Scripts: scripts/risk_unit_test_harness.py, scripts/test_risk_unit_test_harness.py

What it does and when to use it

Use before promoting a change to a pre-trade risk control, to run positive, boundary, breach and malformed-input cases and check the engine still rejects exactly what it should, matching triggered rule ids.

Original author description

Use before promoting a change to a pre-trade risk control, to run positive, boundary, breach and malformed-input cases and check the engine still rejects exactly what it should, matching triggered rule ids.

How it works

  1. When to Use
  2. When NOT to Use
  3. Prerequisites
  4. Workflow
  5. Common Pitfalls
  6. Verification
  7. Related Skills
Original SKILL.md headings · 76 lines, 1,681 words
SKILL.md preview
---
name: risk-control-unit-testing-framework
description: >-
  Use before promoting a change to a pre-trade risk control, to run positive, boundary,
  breach and malformed-input cases and check the engine still rejects exactly what it
  should, matching triggered rule ids.
license: Apache-2.0
metadata:
  domain: algorithmic-trading
  subdomain: risk-management
  tags: risk-testing, unit-test-harness, pre-trade-risk, limit-breach, fat-finger-collar, fail-closed-gate, rts-6-art-15, sec-rule-15c3-5
  brokers_frameworks: "MiFID II RTS 6 (EU 2017/589) Art. 5, 7, 11, 15; SEC Rule 15c3-5 (17 CFR 240.15c3-5); FINRA Regulatory Notice 15-09; FIA Guide to the Development and Operation of Automated Trading Systems (Mar 2015); Python Dataclasses; unittest"
  version: "2.0.0"
  author: algo-trading-skills-contributors
---

## When to Use

Use this skill when a change to a pre-trade risk control (order-size cap, position cap, price collar, daily loss limit) is about to be promoted, and you need a machine-checkable answer to one question: **does this risk engine still reject exactly what it is supposed to reject, and nothing else?** The framework runs a suite of `RiskTestCase` objects through the engine, compares the triggered rule ids **as an exact set**, verifies that every rule in `required_rule_coverage` was actually fired by some case, and returns an auditable `RiskControlTestExecutionReport` whose `status` you branch on.

The rules modelled here are the ones named in primary sources: MiFID II **RTS 6 Art. 15** requires price collars, maximum order values, maximum order volumes and maximum message limits as pre-trade controls on order entry; **17 CFR 240.15c3-5(c)(1)(ii)** requires controls reasonably designed to prevent erroneous orders "that exceed appropriate price or size parameters". Neither rule prescribes how to test those controls — that gap is what this skill fills, and the testing expectation itself comes from RTS 6 Art. 5(1)/(4) and Art. 11, FINRA Regulatory Notice 15-09 (guidance), and FIA's March 2015 guide.

## When NOT to Use

- **As a production risk gate.** `PreTradeRiskEngine` is a *test fixture*: single-threaded, stateless between orders, and with no throttle, no credit check and no self-match prevention. Copying it into a live order path gives you a risk engine that has never seen concurrency. Point the framework at your real engine instead — it accepts any object exposing `evaluate_order(order) -> RiskCheckResult`.
- **For cumulative or rate-based controls.** 15c3-5(c)(1)(ii) covers erroneous orders "on an order-by-order basis **or over a short period of time**", and RTS 6 Art. 15(d) requires maximum message limits. This framework evaluates one order at a time against a stateless engine, so it cannot exercise either limb. A suite that is green here has said nothing about your throttle.
- **As evidence of RTS 6 or 15c3-5 conformance.** 15c3-5 contains no algorithm-testing mandate at all; its testing-adjacent requirement is the annual review under §(e)(1) and the CEO certification under §(e)(2). A passing gate is an input to Art. 5(1)/Art. 11 evidence, not a discharge of either.
- **As a latency benchmark.** The report carries p50/p99 of the engine's own evaluation time, but an interpreted engine timed on a shared CI runner measures the runner. Across repeated fresh-process runs of the shipped standard suite the p99 sits around $2.5\times$ the p50 at single-digit microseconds, with occasional outliers beyond $100\,\mu\text{s}$ from cold code paths alone. `enforce_latency_budget` is off by default for that reason; budget work belongs in `risk-control-latency-budget`.
- **As a substitute for the kill switch.** RTS 6 Art. 12 requires the ability to cancel any or all unexecuted orders immediately. Nothing here tests that — see `kill-switch-and-drawdown-circuit-breakers`.
- **When the ruleset under test is disabled.** A `RiskRuleConfig` with `enabled=False` allows every order. The framework does not special-case it; every negative test case simply fails and the gate reports `RISK_TEST_FAILURES_DETECTED`.

## Prerequisites

- A risk engine exposing `evaluate_order(order) -> RiskCheckResult`, or the shipped `PreTradeRiskEngine` plus a `RiskRuleConfig` (`max_order_size`, `max_position_size`, `max_daily_loss_usd`, `max_price_collar_pct`). Limits must be finite and positive — a zero or absent limit raises rather than meaning "unlimited" (FIA §1.1).
- Test cases written against **your desk's** limits. The shipped standard suite is written against the library defaults (size $1000$, position $5000$, daily loss $10{,}000$, collar $5\%$); those are engineering placeholders, not thresholds any regulator publishes.
- Order state per case: `side` ∈ {BUY, SELL}, `quantity`, `price`, `current_mid_price`, `current_position`, `accumulated_daily_pnl_usd`, and — for a faithful position-cap test — `working_buy_quantity` / `working_sell_quantity`.
- Sign conventions: `accumulated_daily_pnl_usd` is signed (a loss is negative) while `max_daily_loss_usd` is a positive magnitude; `max_price_collar_pct` is a fraction ($0.05 = 5\%$).
- Threshold convention, applied uniformly: **the configured limit value is itself permitted; a breach requires exceeding it.** An order of exactly `max_order_size` passes.

## Workflow

1. **Scenario Suite Construction** — for *each* rule, write four cases: a valid order, one at the exact threshold (must be allowed), one just past the threshold, and the gross breach.
   - **Decision point — the boundary case is the one that finds bugs.** A suite that tests $500$ against a $1000$ cap and $2000$ against a $1000$ cap passes against an engine that uses `>=` and against one that uses `>`. Only the case at exactly $1000$ distinguishes them.
   - Add malformed-input cases: NaN/negative quantity, non-positive price, an unrecognised `side`, and an absent reference price.
2. **Expectation Specification**:
   - **Decision point — name the exact rule set, not one member.** `expected_triggered_rules` is compared with set equality. A case that expects `POSITION_CAP` and gets `{POSITION_CAP, MAX_ORDER_SIZE}` **fails**, because the extra rejection means a limit is firing where it should not — an over-tight risk control rejects good orders in production and no membership assertion would ever surface it.
   - `expected_allowed=True` combined with an expected rule is self-contradictory and raises; it is not silently resolved.
3. **Order Evaluation** — each case is passed to `evaluate_order`; the framework records `is_allowed`, the triggered rule ids, the rejection reasons and the per-evaluation latency.
   - **Decision point — malformed order data must be a rejection, not an exception.** A NaN quantity reaches a live gate for real, and every limit comparison against NaN is `False`. The engine returns `INVALID_ORDER` and stops, rather than evaluating on and allowing the order.
4. **Coverage Evaluation**:
   - **Decision point — a suite that never fired a rule has not tested it, however green it looks.** A suite of nothing but valid orders passes every case; `failed_tests` is `0` and `status` is `RISK_TEST_COVERAGE_INCOMPLETE`. Read `status`, never the count.
5. **CI/CD Gate Evaluation** — precedence is empty suite → case failures → missing coverage → (if enforced) latency budget → pass. Only `ALL_RISK_TESTS_PASSED` promotes the build.
6. **Report Generation** — persist the `RiskControlTestExecutionReport` (`ruleset_id`, `rules_exercised`, `missing_rule_coverage`, per-case `actual_triggered_rules`) against the change record, as the "means used for testing" evidence RTS 6 Art. 5(1) and Art. 11 contemplate.

> Full procedure: see `references/workflows.md`.
> Standards reference: see `references/standards.md`.
> Printable pre-flight checklist: see `assets/checklist.md`.

## Common Pitfalls

- **Asserting rule membership instead of the exact rule set**: `expected_rule in triggered_rules` passes when the engine *also* fires two unrelated rules. The over-tight limit that silently rejects valid flow is invisible to every such assertion. Compare sets.
- **Letting NaN through the gate**: `float('nan') > max_order_size` is `False`, and so is every other comparison. An unvalidated NaN quantity therefore breaches no rule and the order is **allowed** — the classic fail-open. Validate first, reject on malformed input.
- **Treating an unrecognised side as a SELL**: `side.upper() == "BUY"` sends `"BUYY"`, `"Buy Ltd"` and `""` down the SELL branch, flipping the sign of the position projection so a long-side breach is projected short and passes. Whitelist the side.
- **Skipping the price collar when the reference price is missing**: `if mid_price > 0:` disables the fat-finger check exactly when a stale or absent feed makes a fat finger likeliest. FIA §1.3 requires the opposite — aberrant market data blocks orders while it is investigated. A collar that cannot be evaluated must reject.
- **Projecting the position from holdings alone**: FIA §1.2 requires working orders to be included, "such that limits would not be breached if that order is filled". A cap checked against `position + new_quantity` is breached by five individually-compliant orders resting at once.
- **Dividing to test a percentage collar**: `abs(price - mid) / mid > 0.05` spuriously rejects an order priced at exactly the collar for roughly $1\%$ of reference prices — e.g. mid $402.69$, price $422.8245$ evaluates to $0.05000000000000001$. Compare `abs(price - mid) > collar * mid`.
- **Reading `failed_tests == 0` as a pass**: it is also `0` for an empty suite and for an all-positive suite that exercised no rule at all. Both are fail-closed statuses; a pipeline branching on the count ships them.
- **Testing only in a backtest**: a backtest exercises the risk rule only on the paths the strategy happened to take. Boundary and malformed-input cases essentially never occur there.
- **Asserting a microsecond SLA in CI**: a single-shot timing of an interpreted engine on a shared runner is noise-dominated — the shipped suite's own p99 sits around $2.5\times$ its p50 run to run, and single evaluations beyond $100\,\mu\text{s}$ show up from cold code paths alone. Report the percentiles; enforce a budget only against the real engine on representative hardware.
- **Mutating limits mid-suite**: `RiskRuleConfig` is frozen precisely so case A cannot loosen a limit that case B then "passes" under.

## Verification

- Instantiate `RiskControlUnitTestFrameworkEngine()` and run `run_standard_suite()` $\implies$ `status == "ALL_RISK_TESTS_PASSED"`, `total_tests == 16`, `pass_rate_pct == 100.0`, `coverage_satisfied` true, and `rules_exercised` equal to all six rule ids including `INVALID_ORDER` and `REFERENCE_PRICE_UNAVAILABLE`.
- Boundary checks (must be **allowed**): quantity exactly $1000$; position $4000$ + BUY $1000$ against a $5000$ cap; `accumulated_daily_pnl_usd == -10000.0` against a $10{,}000$ limit; price $422.8245$ against mid $402.69$ at a $5\%$ collar. One increment past each must be rejected by exactly one rule.
- Fail-closed checks (must be **rejected** with `INVALID_ORDER`): NaN, infinite, zero and negative quantity; NaN and non-positive price; NaN position; NaN daily PnL; negative working quantity; `side` of `"BUYY"`, `""` and `None`. A mid price of `0.0` or NaN must reject with `REFERENCE_PRICE_UNAVAILABLE` — and must not mask a simultaneous `MAX_ORDER_SIZE` breach.
- Working-order check: position $4000$ + $900$ resting buys + BUY $200$ $\implies$ `POSITION_CAP`; the same case with $100$ instead of $200$ is allowed.
- Assertion-strength check: an order breaching both size and collar, asserted as `expected_triggered_rule=MAX_ORDER_SIZE`, must **fail** the test case.
- Gate checks: `run_suite([])` $\implies$ `RISK_TEST_SUITE_EMPTY`; a suite of three valid orders $\implies$ `RISK_TEST_COVERAGE_INCOMPLETE` with `failed_tests == 0`; a permissive engine that allows everything $\implies$ `RISK_TEST_FAILURES_DETECTED`.
- Mis-wiring checks (must **raise**): blank or duplicate `test_name`; `expected_allowed=True` with an expected rule; both rule arguments supplied; a bare string passed as `expected_triggered_rules`; `RiskRuleConfig` with a zero, negative, infinite or NaN limit; `enforce_latency_budget=True` without a budget.
- Run `python -m unittest discover -s skills/risk-control-unit-testing-framework/scripts` and confirm a 100% pass rate.

## Related Skills

- `risk-control-bypass-audit-logging`
- `risk-control-latency-budget`
- `execution-algorithm-regression-testing-suite`
- `kill-switch-and-drawdown-circuit-breakers`
- `sec-rule-15c3-5-risk-controls-us`
- `position-limit-breach-simulation-fire-drills`
Files in this package (6)
  • SKILL.md 12 KB
  • assets/checklist.md 4 KB
  • references/standards.md 11 KB
  • references/workflows.md 7 KB
  • scripts/risk_unit_test_harness.py 29 KB
  • scripts/test_risk_unit_test_harness.py 23 KB

Versions

The database has no other skills named risk-control-unit-testing-framework.

Database snapshot: 13.11.2026 · package version 02875766f03d (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100risk-control-bypass-audit-logging
    Use when a human can override a pre-trade or intra-trade risk control and the override must leave evidence: who requested, authorised and verified it, what was bypassed and until when. Parameter changes are audit-logging-for-configuration-changes.
  • 90risk-metric-recalculation-frequency-tuning
    Use when a real-time risk engine cannot recompute every metric on every tick; assigns metrics to cadence tiers and accelerates them under stress. Never use it to defer a per-order pre-trade check.
  • 100pre-trade-checklist
    Run a planned trade through a rule gate before the order is placed - is there a stop on the right side, is the risk within the per-trade limit, is the position too large, does reward justify risk, is the loss circuit breaker clear, is there room under the open-risk cap, is this…
  • 100pre-trade-risk-controls
    Specify, review, or test pre-trade and real-time controls for automated trading systems, including order size/notional limits, price collars, position/exposure limits, message throttles, stale-data checks, disconnect handling, and kill switches. Use for risk-control architecture…
  • 95risk-control-matrix
    构建风险控制矩阵(Risk Control Matrix, RCM)——系统性地将流程风险与控制活动对应, 评价控制设计有效性和执行有效性。在内审现场工作阶段使用, 通常接在 process-mapping 之后。
  • 100stop-reward-risk-skill
    用户在买卖候选定型后、下单前问"止损设哪、这单值不值得做、盈亏比划不划算"时使用;或持仓浮亏问"该不该止损"。四步:①止损位由形态客观给定(乌云盖顶/空头吞噬高点上方、锤子线/启明星低点下方、缺口底部下方、趋势线下方,收市价为准)→②目标位由西方工具供给(方法须固定并声明)→③风险÷报偿核算→④诱人则做、不诱人则放弃(可等回抽改善比值,但"许多时候市场并不回调")。触发词:止损位、风险报偿比/盈亏比、risk-reward…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.