Skip to content
Skills Library
DevOps & cloud · Testing

testing-operators

/testing-operators

Use when writing or updating tests for an operator built on the operator-component-framework - the test layers, mutation unit tests, golden snapshot tests (pkg/testing/golden), version-matrix golden generation (pkg/testing/goldengen), the YAML matrix loader, and controller-level tests with envtest against a third-party operator's CRDs.

Licence
Apache-2.0
Last repository change
17.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
2
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 17.10.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What it does and when to use it

Use when writing or updating tests for an operator built on the operator-component-framework - the test layers, mutation unit tests, golden snapshot tests (pkg/testing/golden), version-matrix golden generation (pkg/testing/goldengen), the YAML matrix loader, and controller-level tests with envtest against a third-party operator's CRDs.

Original author description

Use when writing or updating tests for an operator built on the operator-component-framework - the test layers, mutation unit tests, golden snapshot tests (pkg/testing/golden), version-matrix golden generation (pkg/testing/goldengen), the YAML matrix loader, and controller-level tests with envtest against a third-party operator's CRDs.

How it works

  1. The testing layers
  2. Mutation tests
  3. Golden snapshots
  4. goldengen version matrices
  5. YAML matrix loader
  6. Controller layer: envtest with a third-party operator's CRDs
  7. Anti-patterns
  8. Ground truth
  9. References
Original SKILL.md headings · 281 lines, 2,062 words
SKILL.md preview
---
name: testing-operators
description:
  Use when writing or updating tests for an operator built on the operator-component-framework - the test layers,
  mutation unit tests, golden snapshot tests (pkg/testing/golden), version-matrix golden generation
  (pkg/testing/goldengen), the YAML matrix loader, and controller-level tests with envtest against a third-party
  operator's CRDs.
---

# Testing Operators

## The testing layers

Test an operator from the inside out. Each layer asserts something the layer below cannot:

| Layer          | What you assert                                                        | Tool                                                       |
| -------------- | ---------------------------------------------------------------------- | ---------------------------------------------------------- |
| **Mutation**   | one mutation makes the field changes you intend, on a baseline         | testify, against `Preview()`                               |
| **Resource**   | the right mutations fire for a spec, and the rendered output is pinned | `golden` for a snapshot, `goldengen.Resource` for coverage |
| **Component**  | the whole component renders the resources you expect, applied together | `golden.AssertComponentYAML`, or `goldengen.Component`     |
| **Controller** | reconciliation against a real API server writes the status you expect  | `envtest`, plus a harness you write                        |

Two framework packages back the first three layers: `pkg/testing/golden` for single-build snapshot tests, and
`pkg/testing/goldengen` for declarative coverage across versions and specs. Both are opt-in and import nothing into the
reconcile path, so a consumer that does not test against them pays nothing. The controller layer has no framework
package at all: it is the recipe below, assembled in your own suite.

## Mutation tests

A mutation is a pure function: given a baseline object, it makes a specific, isolated set of field changes. Test it as
an input/output pair, not against a golden file. Build a minimal baseline primitive with only the mutation under test,
preview it, and assert the fields it changed:

```go
func TestDebugLoggingMutation(t *testing.T) {
    res, err := deployment.NewBuilder(baseDeployment()).
        WithMutation(features.DebugLoggingMutation(true)).
        Build()
    require.NoError(t, err)

    dep, err := res.Preview()
    require.NoError(t, err)

    container := dep.(*appsv1.Deployment).Spec.Template.Spec.Containers[0]
    assert.Contains(t, container.Env, corev1.EnvVar{Name: "LOG_LEVEL", Value: "debug"})
}
```

There is no golden file at this layer: the assertion states intent directly, against the specific fields the mutation is
documented to change. Share a minimal `baseDeployment()` / `baseConfigMap()` baseline across a package's mutation tests
in a `helpers_test.go` so each test declares only what it exercises.

## Golden snapshots

`golden` renders a built primitive or component to canonical YAML and compares it against a checked-in file. What gets
snapshotted is the full rendered desired state, not the mutation list or internal builder state: serialization resolves
`TypeMeta` (from the object or a supplied scheme) and strips zero-value noise, so the golden reflects only meaningful
desired state.

Typed Kubernetes objects (built-in primitives and standard `k8s.io/api` types) do not populate `TypeMeta` on their own.
Serializing one without a scheme fails with an incomplete-`TypeMeta` error, so pass `golden.WithScheme(scheme)` to every
`AssertYAML` and `AssertComponentYAML` call; the scheme only needs to register the types being serialized.

`AssertYAML` accepts a `golden.Previewer` (`Preview() (client.Object, error)`); `AssertComponentYAML` accepts a
`golden.ComponentPreviewer` (`Preview() ([]client.Object, error)`). All built-in primitives satisfy `Previewer` through
`generic.BaseResource`, and a built `*component.Component` satisfies `ComponentPreviewer` directly.

`Preview` runs no declared data extraction, so every data cell is unset during a golden render. A mutation that calls
`Get` quietly omits the enriched field; one that calls `Require` fails the preview with an error wrapping
`concepts.ErrDataNotExtracted`. Seed the cell before rendering (`dbHost.Set("postgres.default.svc")`), and have the
component assembly function return its cells so tests can reach them.

```go
var update = flag.Bool("update", false, "update golden files")

func TestDeploymentGolden(t *testing.T) {
    res, err := resources.NewDeploymentResource(owner)
    require.NoError(t, err)

    previewer, ok := res.(golden.Previewer)
    require.True(t, ok)
    golden.AssertYAML(t, "testdata/deployment.yaml", previewer,
        golden.WithScheme(scheme), golden.Update(*update))
}
```

### Regeneration

`golden.Update(*update)` overwrites the golden file instead of comparing. Generate once, inspect the diff, then commit:

```bash
go test ./path/to/pkg -run TestDeploymentGolden -update
go test ./path/to/pkg -run TestDeploymentGolden
```

The `-update` flag goes after the package path: `go test -update ./...` passes `-update` to `go test` itself, which
rejects it. Golden files live in a `testdata/` directory next to the test file (Go excludes `testdata/` from the build).

Non-`testing.T` variants (`CompareYAML`, `CompareComponentYAML`) return a `*MismatchError` carrying a unified diff
instead of failing a test, for use outside a test body. `golden.Serialize` and `golden.SerializeComponent` produce the
canonical YAML bytes directly when you need them out of band; `goldengen` is built on exactly these two functions.

### Why mutation names matter

`goldengen` (and golden introspection generally) classifies which mutations fired by reading a resource's
`RegisteredMutations()` and `FiringSet()`, the `concepts.MutationInspector` interface every built resource and component
implements. Those sets are keyed by mutation name, and the name is what `Requires`/`Forbids` assertions, `Exclude`
entries, and the completeness check all refer to. A mutation with an empty name is itself a completeness violation. Give
every mutation a stable, descriptive name (for example `PeerDiscovery/V2`): it is the identifier the whole coverage and
gating story is built on, and it is what shows up in the generated manifest when a reviewer reads a gating diff.

## goldengen version matrices

A resource with version-gated mutations behaves differently across versions, but only where a gate actually flips.
`goldengen` sweeps a declared set of versions and specs, groups the versions by which mutations fire (a "regime"), and
writes one golden per distinct regime instead of one golden per version, then proves through `AssertComplete` that every
registered mutation was asserted somewhere.

A matrix is warranted once a resource or component has version-gated mutations whose behavior needs to be pinned across
more than one version: asserting one golden per version at that point is wasteful (versions inside the same regime
produce identical output) and, more importantly, does not prove where the behavior boundary actually is. A single golden
snapshot is enough when there is nothing version-gated to sweep, or you only care about one fixed version's output;
reach for a matrix specifically to lock down a gate boundary, and pin both sides of it (`Requires` at the version after
the boundary, `Forbids` at the version before) so the boundary itself is asserted, not just "fires somewhere".

Declare the sweep with `goldengen.Config[T]` (`Dir`, `Versions`, `Fixtures`, `Exclude`, `Build`), where `Build` adapts a
version-and-spec pair into a `goldengen.Unit` via `goldengen.Resource(res, scheme)` or
`goldengen.Component(comp, scheme)`:

```go
var gen = goldengen.New(goldengen.Config[*app.ExampleApp]{
    Dir:      "testdata/version_matrix",
    Versions: []string{"1.0.0", "1.5.0", "2.0.0"},
    Fixtures: []goldengen.Fixture[*app.ExampleApp]{{
        Name: "default",
        Spec: defaultCluster(),
        Requires: []goldengen.Expect{
            {Name: "ContainerImage"},
            {Name: "PeerDiscovery/PreV2", For: "1.5.0"},
            {Name: "PeerDiscovery/V2", For: "2.0.0"},
        },
        Forbids: []goldengen.Expect{
            {Name: "PeerDiscovery/V2", For: "1.5.0"},
        },
    }},
    Build: func(version string, spec *app.ExampleApp) (goldengen.Unit, error) {
        c := spec.DeepCopyObject().(*app.ExampleApp)
        c.Spec.Version = version
        res, err := resources.NewStatefulSetResource(c)
        if err != nil {
            return nil, err
        }
        return goldengen.Resource(res, scheme), nil
    },
})
```

`Build` must deep-copy the incoming spec before setting the version: it is called once per version for the same fixture,
and the spec is shared across that sweep. List `Versions` ascending; the representative golden for a regime is named
after the first version (in supplied order) that belongs to it, so ascending order puts each golden's filename on the
lower inclusive boundary of its gating range.

Wire a sweep into a normal test:

```go
func TestVersionMatrix(t *testing.T) {
    gen.WithUpdate(*update)
    gen.Run(t)
}

func TestMain(m *testing.M) {
    os.Exit(gen.AssertComplete(m.Run()))
}
```

`Run` validates the config, builds every fixture at every version, checks each `Requires`/`Forbids` during the sweep,
then writes (under `-update`) or compares one golden per regime plus a reviewable `manifest.yaml` (per fixture, each
regime's representative version, the versions it covers, and its firing set). `AssertComplete`, called from `TestMain`,
is a separate, registration-based check: it fails when a registered mutation is named in neither `Requires` nor
`Exclude`, or when a `Requires`/`Exclude` name matches nothing registered. Registering a new version-gated mutation
therefore fails the suite until it is asserted or deliberately excluded.

## YAML matrix loader

`goldengen.LoadMatrix[T](path, newSpec, build)` loads a matrix's `Dir`, `Versions`, `Fixtures` (with their `Requires` /
`Forbids` / `Exclude`) from a YAML file instead of Go source, keeping the version universe and fixture data separate
from the build logic, which still lives in the `build` callback passed to `LoadMatrix`. Each fixture supplies its spec
either inline under `spec:` or from an external file under `specFile:`, exactly one of the two. Reach for the YAML
loader when the matrix data (versions, fixtures, gating expectations) is more naturally maintained as data files than as
a Go literal, for example when non-Go-writing maintainers curate fixtures, or the same matrix shape is reused across
several resources with only the data changing. `LoadMatrix` returns a validated `Config[T]`; wrap it with
`goldengen.New(cfg)` and call `Run` exactly as with a Go-declared config. It errors if a fixture sets both `spec` and
`specFile` (or neither), if a `for` value is not in `versions`, or if a spec fails to unmarshal into `T`.

## Controller layer: envtest with a third-party operator's CRDs

The rendering layers prove what the operator would apply. Only a real API server proves what it writes to the owner's
status. `sigs.k8s.io/controller-runtime/pkg/envtest` supplies one. **The framework ships no envtest helper**;
`pkg/testing` holds `golden` and `goldengen` only, so what follows is a recipe you assemble yourself. The framework's
own suites show the shape: `pkg/component/suite_test.go` starts an envtest environment with an in-Go CRD, and
`e2e/framework/` holds a test owner CRD (`crd.go`) and a factory-driven reconciler (`reconciler.go`).

Three steps, each with a trap.

**Load the third-party CRDs from the module cache.** The CRD YAML ships inside the operator's Go module, so the version
your `go.mod` resolves is already on disk. Ask the toolchain for it instead of vendoring a copy that drifts:

```go
out, err := exec.Command("go", "list", "-m", "-f", "{{.Dir}}", "example.com/broker-operator").Output()
require.NoError(t, err)

testEnv = &envtest.Environment{
    CRDDirectoryPaths: []string{
        filepath.Join("..", "..", "config", "crd", "bases"),
        filepath.Join(strings.TrimSpace(string(out)), "config", "crd", "bases"),
    },
    ErrorIfCRDPathMissing: true,
}
```

Register the third-party scheme too, or the client cannot decode the kind. Keep `ErrorIfCRDPathMissing` true so a
dependency bump that moves the directory fails the suite instead of testing against a missing kind.

**Drive the external CR's status from the test.** envtest runs the API server and etcd and no controllers, so the
third-party operator is not there and the CR your wrapper applies keeps an empty status forever. The test plays its
part, through `Status().Update` because status is a subresource:

```go
require.NoError(t, k8sClient.Get(ctx, brokerKey, broker)) // fresh object, current resourceVersion
broker.Status.ObservedGeneration = broker.Generation      // the wrapper's handlers gate on this
broker.Status.ReadyReplicas = *broker.Spec.Replicas
require.NoError(t, k8sClient.Status().Update(ctx, broker))
```

Stamp `observedGeneration` deliberately. A wrapper's status handler compares it against `metadata.generation` before it
trusts any readiness field, which is what `concepts.StaleGenerationStatus` does, so a test that sets `ReadyReplicas` and
leaves `observedGeneration` at zero pins the component in its stale-generation branch for a reason unrelated to the code
under test.

**Assert the owner's condition.** Call the reconciler directly rather than starting a manager: one `Reconcile` per state
change keeps the assertion deterministic.

```go
_, err := reconciler.Reconcile(ctx, reconcile.Request{NamespacedName: ownerKey})
require.NoError(t, err)
require.NoError(t, k8sClient.Get(ctx, ownerKey, app))

ready := meta.FindStatusCondition(app.Status.Conditions, "Ready")
require.NotNil(t, ready)
assert.Equal(t, metav1.ConditionTrue, ready.Status)
assert.Equal(t, string(component.Healthy), ready.Reason)
```

`"Ready"` is your own aggregate condition type here, not a framework-fixed name; a single component's condition type
reads the same way. `meta.FindStatusCondition` is right because the test is a consumer of the persisted object; inside
the controller, aggregation reads component conditions with `(*Component).GetCondition`. Assert the reason as well as
the status: the reason is a `component.Status` value, so it pins which component governed the result. Two assertions
belong at this layer and nowhere else: a not-yet-reconciled component holds the owner at `False`, and a component built
with `Suspend(true)` reports `True` while its resource is gone from the cluster.

## Anti-patterns

- **Regenerating goldens to make a failing test pass without reading the diff.** A golden mismatch means the rendered
  output changed; run with `-update` only after confirming the new output is the output you intend, not as a reflex to
  clear a red test.
- **Asserting on implementation internals instead of rendered output.** Golden and goldengen tests exist to pin what
  gets applied to the cluster; asserting on builder state or mutation call counts instead of the previewed object misses
  the thing that actually matters to a consumer.
- **Skipping the version matrix for version-gated mutations.** A single golden at one version cannot prove where a gate
  boundary sits; a version-gated mutation without a matrix (or without `Requires`/`Forbids` pinning both sides of its
  boundary) is unverified at the versions that matter most.

## Ground truth

The consumer's resolved module version is the source of truth, not these docs. Before asserting an exact signature,
method name, or option:

1. Read the framework version from the consumer's `go.mod` entry for
   `github.com/sourcehawk/operator-component-framework`.
2. Verify the symbol with `go doc github.com/sourcehawk/operator-component-framework/pkg/<package> <Symbol>`.

The reference files bundled with this skill match the framework version this plugin shipped with. When they disagree
with `go doc`, `go doc` wins.

## References

- `references/testing.md`: full testing documentation. Read when you need exact `goldengen` config field semantics, the
  completeness accounting rules, or the YAML matrix file format.
Files in this package (2)
  • SKILL.md 16 KB
  • references/testing.md 28 KB

Versions

The database has no other skills named testing-operators.

Database snapshot: 13.11.2026 · package version c7851fcc5c30 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 90devops-kubernetes-operators
    Use when the user asks about Kubernetes operators, custom controllers, operator pattern, Kubebuilder, Operator SDK, custom resource definitions (CRD), reconciliation loops, or extending Kubernetes with custom resources. Covers: Kubebuilder project structure, controller-runtime…
  • 90mobile-testing
    Use this skill when the user asks about mobile testing strategies, unit tests, widget tests, component tests, integration tests, E2E, golden/snapshot tests, mocking, or CI test integration.
  • 100kubernetes-operator-development
    Guides building Kubernetes Operators with Kubebuilder or the Operator SDK — CustomResourceDefinition (CRD) API design, controller scaffolding, the reconciliation loop pattern, status/conditions reporting, finalizers for cleanup, and testing with envtest. Use when a user asks to…
  • 100kubernetes-operator
    Design, build, and operate Kubernetes operators. Use when extending Kubernetes with a custom controller, choosing a framework, designing CRDs, implementing reconciliation loops, or auditing an operator for anti-patterns.
  • 100operators-and-crds
    Covers extending Kubernetes with CustomResourceDefinitions and controllers — the reconciliation pattern, a CRD as an API contract, why controllers must be level-triggered not edge-triggered, and when to build an operator versus buy one versus not bother. Use this whenever the…
  • 100mobile-testing
    Mobile E2E testing patterns — Flutter integration_test, Patrol, Maestro, golden testing, device matrix, and test data management. Use when writing E2E tests for mobile applications.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.