Skip to content
Skills Library
Programming · Testing

software-test-execution

/software-test-execution

Run scoped tests for a TDD slice, determine the appropriate test framework and command, and report structured pass/fail results. Use when tests need to be executed after writing, implementing, or refactoring code. Reads test metadata, discovers the test framework if needed, executes the scoped test command, parses output into structured results, and reports whether tests passed or failed. Handles compilation errors, missing dependencies, timeouts, and works with any language and test framework.

Repository
stencila ↗
Licence
Apache-2.0
Last repository change
28.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
3
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 28.10.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What it does and when to use it

Run scoped tests for a TDD slice, determine the appropriate test framework and command, and report structured pass/fail results. Use when tests need to be executed after writing, implementing, or refactoring code. Reads test metadata, discovers the test framework if needed, executes the scoped test command, parses output into structured results, and reports whether tests passed or failed. Handles compilation errors, missing dependencies, timeouts, and works with any language and test framework.

Original author description

Run scoped tests for a TDD slice, determine the appropriate test framework and command, and report structured pass/fail results. Use when tests need to be executed after writing, implementing, or refactoring code. Reads test metadata, discovers the test framework if needed, executes the scoped test command, parses output into structured results, and reports whether tests passed or failed. Handles compilation errors, missing dependencies, timeouts, and works with any language and test framework.

How it works

  1. Overview
  2. Required Inputs
  3. Outputs
  4. Steps
  5. Examples
  6. Edge Cases
Original SKILL.md headings · 219 lines, 1,600 words
SKILL.md preview
---
name: software-test-execution
title: Software Test Execution Skill
description: Run scoped tests for a TDD slice, determine the appropriate test framework and command, and report structured pass/fail results. Use when tests need to be executed after writing, implementing, or refactoring code. Reads test metadata, discovers the test framework if needed, executes the scoped test command, parses output into structured results, and reports whether tests passed or failed. Handles compilation errors, missing dependencies, timeouts, and works with any language and test framework.
keywords:
  - test execution
  - test runner
  - run tests
  - test results
  - pass fail
  - TDD
  - red green refactor
  - scoped tests
  - test command
  - test framework discovery
  - compilation errors
  - test output parsing
  - test verification
  - cargo test
  - pytest
  - vitest
  - go test
  - not test creation
  - not test writing
  - not implementation
  - not refactoring
allowed-tools: read_file glob grep shell
---

## Overview

Execute scoped tests for a TDD slice and report structured results. This skill runs only the tests relevant to the current slice, parses the output, and reports a clear pass/fail result with details.

This skill does not write or modify any code or test files. It only reads, executes, and reports.

## Required Inputs

This skill requires the following information to operate:

| Input          | Required | Description                                                          |
|----------------|----------|----------------------------------------------------------------------|
| Test command   | No       | Command to run the scoped tests (discovered if not provided)         |
| Test files     | No       | List of test file paths (used to scope the command if needed)        |
| Slice scope    | No       | Description of what the slice covers (for context in the report)     |
| Target packages| No       | Packages, crates, or directories involved (for command discovery)    |
| Slice name     | No       | Name or identifier of the current slice (for the report header)      |

When used standalone, these inputs come from the user or the agent's prompt. When used within a workflow, the workflow's stage prompt will specify how to obtain them.

## Outputs

After completing its work, this skill reports:

| Output       | Description                                                    |
|--------------|----------------------------------------------------------------|
| Result       | Whether the tests passed or failed (Pass / Fail)               |
| Report       | A structured report with counts, failure details, and notes    |

## Steps

### 1. Gather test metadata

Ensure the available inputs are collected:

- The test command is the primary input — if present, use it directly
- Test files and target packages are used as fallbacks for constructing a test command if no command is provided
- Slice name and scope provide context for the report

### 2. Determine the test command

If a test command is provided and non-empty, use it. Otherwise, discover the command:

#### 2a. Identify the build system

- Use `glob` to search for build files in the target directories:
  - `Cargo.toml`, `go.mod`, `package.json`, `pyproject.toml`, `setup.py`, `pom.xml`, `build.gradle*`, `Gemfile`, `mix.exs`, `Makefile`
- Read the relevant config files to understand the test configuration

#### 2b. Check for canonical test commands

- Look in `Makefile` for `test:` targets
- Check `package.json` `scripts.test` for JS/TS projects
- Check CI config files (`.github/workflows/*.yml`) for test commands
- Consult `references/framework-detection.md` for the mapping from build file to test command

#### 2c. Construct a scoped command

- If test files list specific files, scope the command to those files
- If target packages name specific packages, scope to those packages
- Never run the full project test suite when scoping is possible
- See `references/framework-detection.md` for scoping patterns per framework

### 3. Execute the test command

- Run the test command via `shell` with an appropriate timeout (default 120 seconds; increase to 300 seconds for compiled languages like Rust, Go, or C++ where compilation may be slow)
- Capture both stdout and stderr — test output may appear on either stream
- Record the exit code

#### Timeout handling

- If the command times out, report it as a failure with an explanation that the test suite exceeded the time limit
- Suggest possible causes: infinite loops, network-dependent tests, excessive test scope

#### Compilation or collection errors

- If the output contains compilation errors (e.g., `error[E...]` in Rust, `SyntaxError` in Python, `Cannot find module` in JS), report this as a failure
- Include the compilation error details in the structured output so the upstream agent can fix them
- Do not attempt to fix the errors — this skill only executes and reports

#### Missing dependencies

- If tests fail because a test framework or dependency is not installed (e.g., `ModuleNotFoundError: No module named 'pytest'`, `error: no matching package named`), report this as a failure
- Include the missing dependency in the output so the upstream agent can address it

### 4. Parse the test output

Analyze the combined stdout/stderr output to extract:

- **Number of tests passed**
- **Number of tests failed**
- **Number of tests skipped or ignored**
- **Names and failure details for each failing test** (assertion messages, expected vs actual values, stack traces)
- **Names of passing tests** (when available in the output)

Use `references/output-parsing.md` for framework-specific parsing patterns. If the output format is unfamiliar, extract what you can and include the raw output in the report.

### 5. Determine the overall result

The result is **Pass** if and only if:

1. The exit code is 0
2. No test failures are reported in the output
3. At least one test actually ran (zero tests executed is suspicious and should be flagged as a potential issue, but still reported as Pass if exit code is 0)

The result is **Fail** if:

1. The exit code is non-zero, OR
2. Any test failures appear in the output, OR
3. A compilation or collection error prevented tests from running

When the exit code and output disagree (e.g., exit code 0 but failures in output), trust the output — some frameworks have bugs or configurations that mask failures in exit codes.

### 6. Report structured results

Output a clear structured report:

````
## Test Results: [PASS | FAIL]

**Slice**: <slice name>
**Command**: `<the command that was run>`
**Exit code**: <code>
**Duration**: <time if available>

### Summary
- Passed: N
- Failed: N
- Skipped: N

### Failed Tests
1. `<test_name>` — <brief failure reason>
   ```
   <assertion message or error detail>
   ```

### Passed Tests
1. `<test_name>`

### Notes
- <any observations: warnings, slow tests, suspicious patterns>
````

If no tests failed, omit the "Failed Tests" section. If the list of passing tests is very long (>20), summarize rather than listing each one.

## Examples

### Example 1: Rust — tests pass

Test command: `cargo test -p my-auth`, scope: "token validation"

```
$ cargo test -p my-auth
   Compiling my-auth v0.1.0
    Finished test target(s)
     Running unittests src/lib.rs
running 4 tests
test token::tests::test_valid_token_parses ... ok
test token::tests::test_expired_token_rejected ... ok
test token::tests::test_malformed_token_returns_error ... ok
test token::tests::test_missing_roles_uses_empty_vec ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
```

Report: PASS, 4 passed, 0 failed. Result: Pass.

### Example 2: Python — tests fail

Test command: `pytest tests/test_parser.py`, scope: "CSV parser error handling"

```
$ pytest tests/test_parser.py
FAILED tests/test_parser.py::test_empty_file_raises_error - ImportError: cannot import name 'parse_csv'
FAILED tests/test_parser.py::test_malformed_row_skipped - ImportError: cannot import name 'parse_csv'
====== 0 passed, 2 failed in 0.12s ======
```

Report: FAIL, 0 passed, 2 failed. Failures are import errors because the implementation does not exist yet — expected in Red phase. Result: Fail.

### Example 3: No test command provided — discovery needed

Test command: absent. Target packages: "frontend". Test files: "frontend/src/components/__tests__/Button.test.tsx"

Discovery:
- `glob` finds `frontend/package.json`
- Read `package.json` → `"scripts": { "test": "vitest run" }`
- Construct scoped command: `cd frontend && npx vitest run src/components/__tests__/Button.test.tsx`

Execute, parse, report as normal.

### Example 4: Compilation error

Test command: `cargo test -p my-codec`

```
$ cargo test -p my-parser
error[E0433]: failed to resolve: could not find `Parser` in `parser`
 --> src/lib.rs:15:22
   |
15 |     use crate::parser::Parser;
   |                        ^^^^^^^ not found in `parser`
```

Report: FAIL, 0 passed, 0 failed (compilation error prevented tests from running). Include the full compiler error. Result: Fail.

## Edge Cases

- **No test command provided and no build system detected**: Search for a `Makefile` with a test target or CI config with a test step. If nothing is found, report the failure clearly — "Could not determine how to run tests for this project" — and report Fail.
- **Test command runs but produces no output**: Some frameworks produce no output on success. If exit code is 0 and no failures are detected, report Pass with a note that no detailed output was available. If exit code is non-zero with no output, report Fail and suggest checking stderr or increasing verbosity.
- **Flaky tests**: If a test sometimes passes and sometimes fails, report the observed result and note in the report that the test may be flaky (e.g., if the failure is timing-related or network-dependent). Do not re-run automatically — let the upstream agent decide.
- **Very large test output**: If the output exceeds what can be reasonably included in a report (thousands of lines), summarize: include the summary line, all failure details, and the first few passing tests. Truncate with a note about the total count.
- **Multiple test commands needed**: If test files span multiple packages that require different test commands (e.g., both a Rust crate and a Python package), run each command separately and combine the results. The overall result is Pass only if all commands pass.
- **Tests pass but with warnings**: Warnings do not affect the Pass/Fail result. Include notable warnings in the Notes section of the report (e.g., deprecation warnings, compiler warnings about unused code).
- **Zero tests executed**: If the test runner reports 0 tests run and exits with code 0, report as Pass but flag this prominently — it usually means the test filter matched nothing, which may indicate stale test file paths or an incorrect scoping pattern.
- **Test command not found**: If the `shell` execution fails because the test runner binary is not installed (e.g., `command not found: pytest`), report Fail with the specific error so the upstream agent can install the dependency.
- **Workspace-level vs package-level test commands**: Some projects run tests from the workspace root (e.g., `cargo test -p <crate>` from the repo root) while others require `cd <dir> && <command>`. Check whether the test command includes a directory change or package selector, and if it fails with a "not found" error, try running from the workspace root with a package flag.
Files in this package (3)
  • SKILL.md 11 KB
  • references/framework-detection.md 4 KB
  • references/output-parsing.md 3 KB

Versions

The database has no other skills named software-test-execution.

Database snapshot: 13.11.2026 · package version e3830f833f10 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100software-test-creation
    Write failing tests for a TDD slice based on acceptance criteria and codebase conventions. Use when the "red" phase of red-green-refactor requires tests that define expected behavior before implementation exists. Discovers codebase test conventions first, writes test files that…
  • 100software-implementation
    Write the minimal production code needed to make failing TDD tests pass (Green phase of red-green-refactor). Use when failing tests exist and production code must be written or modified to satisfy them. Reads and interprets failing test output, discovers codebase conventions…
  • 100testing
    Creates and manages unit and integration tests: detects the test framework, follows project conventions, runs the fix-loop. Use when the user mentions "write tests", "unit tests", "integration tests", "test coverage", "tests missing", "TDD", or after implementation when tests…
  • 100tdd
    Test-driven development workflow. Use when the user asks to "write tests first", "TDD", "add test coverage", "test this function", or wants to develop a feature test-first.
  • 100testing
    This skill should be used when the user asks about "testing", "test coverage", "unit tests", "integration tests", "test organization", "test best practices", "flaky tests", "test isolation", "TDD", "regression tests", or needs guidance on test structure, coverage requirements, and test execution.
  • 90TDD Methodology
    This skill should be used when the user asks to "write tests first", "use TDD", "test-driven development", "red green refactor", "test first", "add unit tests before code", "write regression test first", "safe refactor with tests", or when TDD mode is active and the user makes…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.