testing-operators
/testing-operators
Pisze testy operatora zbudowanego na operator-component-framework: testy mutacyjne, golden snapshoty, macierz wersji i testy z envtest.
- Autor
- sourcehawk ↗
- Repozytorium
- operator-component-framework ↗
- Licencja
- Apache-2.0
- Ostatnia zmiana repozytorium
- 17.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 2
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 17.10.2026, bez odwołań do wycofanych modeli i API.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
Co robi i kiedy
Pisze testy operatora zbudowanego na operator-component-framework: testy mutacyjne, golden snapshoty, macierz wersji i testy z envtest.
Oryginalny opis (EN)
Use when writing or updating tests for an operator built on the operator-component-framework - the test layers, mutation unit tests, golden snapshot tests (pkg/testing/golden), version-matrix golden generation (pkg/testing/goldengen), the YAML matrix loader, and controller-level tests with envtest against a third-party operator's CRDs.
Jak działa
- The testing layers
- Mutation tests
- Golden snapshots
- goldengen version matrices
- YAML matrix loader
- Controller layer: envtest with a third-party operator's CRDs
- Anti-patterns
- Ground truth
- References
Podgląd SKILL.md
---
name: testing-operators
description:
Use when writing or updating tests for an operator built on the operator-component-framework - the test layers,
mutation unit tests, golden snapshot tests (pkg/testing/golden), version-matrix golden generation
(pkg/testing/goldengen), the YAML matrix loader, and controller-level tests with envtest against a third-party
operator's CRDs.
---
# Testing Operators
## The testing layers
Test an operator from the inside out. Each layer asserts something the layer below cannot:
| Layer | What you assert | Tool |
| -------------- | ---------------------------------------------------------------------- | ---------------------------------------------------------- |
| **Mutation** | one mutation makes the field changes you intend, on a baseline | testify, against `Preview()` |
| **Resource** | the right mutations fire for a spec, and the rendered output is pinned | `golden` for a snapshot, `goldengen.Resource` for coverage |
| **Component** | the whole component renders the resources you expect, applied together | `golden.AssertComponentYAML`, or `goldengen.Component` |
| **Controller** | reconciliation against a real API server writes the status you expect | `envtest`, plus a harness you write |
Two framework packages back the first three layers: `pkg/testing/golden` for single-build snapshot tests, and
`pkg/testing/goldengen` for declarative coverage across versions and specs. Both are opt-in and import nothing into the
reconcile path, so a consumer that does not test against them pays nothing. The controller layer has no framework
package at all: it is the recipe below, assembled in your own suite.
## Mutation tests
A mutation is a pure function: given a baseline object, it makes a specific, isolated set of field changes. Test it as
an input/output pair, not against a golden file. Build a minimal baseline primitive with only the mutation under test,
preview it, and assert the fields it changed:
```go
func TestDebugLoggingMutation(t *testing.T) {
res, err := deployment.NewBuilder(baseDeployment()).
WithMutation(features.DebugLoggingMutation(true)).
Build()
require.NoError(t, err)
dep, err := res.Preview()
require.NoError(t, err)
container := dep.(*appsv1.Deployment).Spec.Template.Spec.Containers[0]
assert.Contains(t, container.Env, corev1.EnvVar{Name: "LOG_LEVEL", Value: "debug"})
}
```
There is no golden file at this layer: the assertion states intent directly, against the specific fields the mutation is
documented to change. Share a minimal `baseDeployment()` / `baseConfigMap()` baseline across a package's mutation tests
in a `helpers_test.go` so each test declares only what it exercises.
## Golden snapshots
`golden` renders a built primitive or component to canonical YAML and compares it against a checked-in file. What gets
snapshotted is the full rendered desired state, not the mutation list or internal builder state: serialization resolves
`TypeMeta` (from the object or a supplied scheme) and strips zero-value noise, so the golden reflects only meaningful
desired state.
Typed Kubernetes objects (built-in primitives and standard `k8s.io/api` types) do not populate `TypeMeta` on their own.
Serializing one without a scheme fails with an incomplete-`TypeMeta` error, so pass `golden.WithScheme(scheme)` to every
`AssertYAML` and `AssertComponentYAML` call; the scheme only needs to register the types being serialized.
`AssertYAML` accepts a `golden.Previewer` (`Preview() (client.Object, error)`); `AssertComponentYAML` accepts a
`golden.ComponentPreviewer` (`Preview() ([]client.Object, error)`). All built-in primitives satisfy `Previewer` through
`generic.BaseResource`, and a built `*component.Component` satisfies `ComponentPreviewer` directly.
`Preview` runs no declared data extraction, so every data cell is unset during a golden render. A mutation that calls
`Get` quietly omits the enriched field; one that calls `Require` fails the preview with an error wrapping
`concepts.ErrDataNotExtracted`. Seed the cell before rendering (`dbHost.Set("postgres.default.svc")`), and have the
component assembly function return its cells so tests can reach them.
```go
var update = flag.Bool("update", false, "update golden files")
func TestDeploymentGolden(t *testing.T) {
res, err := resources.NewDeploymentResource(owner)
require.NoError(t, err)
previewer, ok := res.(golden.Previewer)
require.True(t, ok)
golden.AssertYAML(t, "testdata/deployment.yaml", previewer,
golden.WithScheme(scheme), golden.Update(*update))
}
```
### Regeneration
`golden.Update(*update)` overwrites the golden file instead of comparing. Generate once, inspect the diff, then commit:
```bash
go test ./path/to/pkg -run TestDeploymentGolden -update
go test ./path/to/pkg -run TestDeploymentGolden
```
The `-update` flag goes after the package path: `go test -update ./...` passes `-update` to `go test` itself, which
rejects it. Golden files live in a `testdata/` directory next to the test file (Go excludes `testdata/` from the build).
Non-`testing.T` variants (`CompareYAML`, `CompareComponentYAML`) return a `*MismatchError` carrying a unified diff
instead of failing a test, for use outside a test body. `golden.Serialize` and `golden.SerializeComponent` produce the
canonical YAML bytes directly when you need them out of band; `goldengen` is built on exactly these two functions.
### Why mutation names matter
`goldengen` (and golden introspection generally) classifies which mutations fired by reading a resource's
`RegisteredMutations()` and `FiringSet()`, the `concepts.MutationInspector` interface every built resource and component
implements. Those sets are keyed by mutation name, and the name is what `Requires`/`Forbids` assertions, `Exclude`
entries, and the completeness check all refer to. A mutation with an empty name is itself a completeness violation. Give
every mutation a stable, descriptive name (for example `PeerDiscovery/V2`): it is the identifier the whole coverage and
gating story is built on, and it is what shows up in the generated manifest when a reviewer reads a gating diff.
## goldengen version matrices
A resource with version-gated mutations behaves differently across versions, but only where a gate actually flips.
`goldengen` sweeps a declared set of versions and specs, groups the versions by which mutations fire (a "regime"), and
writes one golden per distinct regime instead of one golden per version, then proves through `AssertComplete` that every
registered mutation was asserted somewhere.
A matrix is warranted once a resource or component has version-gated mutations whose behavior needs to be pinned across
more than one version: asserting one golden per version at that point is wasteful (versions inside the same regime
produce identical output) and, more importantly, does not prove where the behavior boundary actually is. A single golden
snapshot is enough when there is nothing version-gated to sweep, or you only care about one fixed version's output;
reach for a matrix specifically to lock down a gate boundary, and pin both sides of it (`Requires` at the version after
the boundary, `Forbids` at the version before) so the boundary itself is asserted, not just "fires somewhere".
Declare the sweep with `goldengen.Config[T]` (`Dir`, `Versions`, `Fixtures`, `Exclude`, `Build`), where `Build` adapts a
version-and-spec pair into a `goldengen.Unit` via `goldengen.Resource(res, scheme)` or
`goldengen.Component(comp, scheme)`:
```go
var gen = goldengen.New(goldengen.Config[*app.ExampleApp]{
Dir: "testdata/version_matrix",
Versions: []string{"1.0.0", "1.5.0", "2.0.0"},
Fixtures: []goldengen.Fixture[*app.ExampleApp]{{
Name: "default",
Spec: defaultCluster(),
Requires: []goldengen.Expect{
{Name: "ContainerImage"},
{Name: "PeerDiscovery/PreV2", For: "1.5.0"},
{Name: "PeerDiscovery/V2", For: "2.0.0"},
},
Forbids: []goldengen.Expect{
{Name: "PeerDiscovery/V2", For: "1.5.0"},
},
}},
Build: func(version string, spec *app.ExampleApp) (goldengen.Unit, error) {
c := spec.DeepCopyObject().(*app.ExampleApp)
c.Spec.Version = version
res, err := resources.NewStatefulSetResource(c)
if err != nil {
return nil, err
}
return goldengen.Resource(res, scheme), nil
},
})
```
`Build` must deep-copy the incoming spec before setting the version: it is called once per version for the same fixture,
and the spec is shared across that sweep. List `Versions` ascending; the representative golden for a regime is named
after the first version (in supplied order) that belongs to it, so ascending order puts each golden's filename on the
lower inclusive boundary of its gating range.
Wire a sweep into a normal test:
```go
func TestVersionMatrix(t *testing.T) {
gen.WithUpdate(*update)
gen.Run(t)
}
func TestMain(m *testing.M) {
os.Exit(gen.AssertComplete(m.Run()))
}
```
`Run` validates the config, builds every fixture at every version, checks each `Requires`/`Forbids` during the sweep,
then writes (under `-update`) or compares one golden per regime plus a reviewable `manifest.yaml` (per fixture, each
regime's representative version, the versions it covers, and its firing set). `AssertComplete`, called from `TestMain`,
is a separate, registration-based check: it fails when a registered mutation is named in neither `Requires` nor
`Exclude`, or when a `Requires`/`Exclude` name matches nothing registered. Registering a new version-gated mutation
therefore fails the suite until it is asserted or deliberately excluded.
## YAML matrix loader
`goldengen.LoadMatrix[T](path, newSpec, build)` loads a matrix's `Dir`, `Versions`, `Fixtures` (with their `Requires` /
`Forbids` / `Exclude`) from a YAML file instead of Go source, keeping the version universe and fixture data separate
from the build logic, which still lives in the `build` callback passed to `LoadMatrix`. Each fixture supplies its spec
either inline under `spec:` or from an external file under `specFile:`, exactly one of the two. Reach for the YAML
loader when the matrix data (versions, fixtures, gating expectations) is more naturally maintained as data files than as
a Go literal, for example when non-Go-writing maintainers curate fixtures, or the same matrix shape is reused across
several resources with only the data changing. `LoadMatrix` returns a validated `Config[T]`; wrap it with
`goldengen.New(cfg)` and call `Run` exactly as with a Go-declared config. It errors if a fixture sets both `spec` and
`specFile` (or neither), if a `for` value is not in `versions`, or if a spec fails to unmarshal into `T`.
## Controller layer: envtest with a third-party operator's CRDs
The rendering layers prove what the operator would apply. Only a real API server proves what it writes to the owner's
status. `sigs.k8s.io/controller-runtime/pkg/envtest` supplies one. **The framework ships no envtest helper**;
`pkg/testing` holds `golden` and `goldengen` only, so what follows is a recipe you assemble yourself. The framework's
own suites show the shape: `pkg/component/suite_test.go` starts an envtest environment with an in-Go CRD, and
`e2e/framework/` holds a test owner CRD (`crd.go`) and a factory-driven reconciler (`reconciler.go`).
Three steps, each with a trap.
**Load the third-party CRDs from the module cache.** The CRD YAML ships inside the operator's Go module, so the version
your `go.mod` resolves is already on disk. Ask the toolchain for it instead of vendoring a copy that drifts:
```go
out, err := exec.Command("go", "list", "-m", "-f", "{{.Dir}}", "example.com/broker-operator").Output()
require.NoError(t, err)
testEnv = &envtest.Environment{
CRDDirectoryPaths: []string{
filepath.Join("..", "..", "config", "crd", "bases"),
filepath.Join(strings.TrimSpace(string(out)), "config", "crd", "bases"),
},
ErrorIfCRDPathMissing: true,
}
```
Register the third-party scheme too, or the client cannot decode the kind. Keep `ErrorIfCRDPathMissing` true so a
dependency bump that moves the directory fails the suite instead of testing against a missing kind.
**Drive the external CR's status from the test.** envtest runs the API server and etcd and no controllers, so the
third-party operator is not there and the CR your wrapper applies keeps an empty status forever. The test plays its
part, through `Status().Update` because status is a subresource:
```go
require.NoError(t, k8sClient.Get(ctx, brokerKey, broker)) // fresh object, current resourceVersion
broker.Status.ObservedGeneration = broker.Generation // the wrapper's handlers gate on this
broker.Status.ReadyReplicas = *broker.Spec.Replicas
require.NoError(t, k8sClient.Status().Update(ctx, broker))
```
Stamp `observedGeneration` deliberately. A wrapper's status handler compares it against `metadata.generation` before it
trusts any readiness field, which is what `concepts.StaleGenerationStatus` does, so a test that sets `ReadyReplicas` and
leaves `observedGeneration` at zero pins the component in its stale-generation branch for a reason unrelated to the code
under test.
**Assert the owner's condition.** Call the reconciler directly rather than starting a manager: one `Reconcile` per state
change keeps the assertion deterministic.
```go
_, err := reconciler.Reconcile(ctx, reconcile.Request{NamespacedName: ownerKey})
require.NoError(t, err)
require.NoError(t, k8sClient.Get(ctx, ownerKey, app))
ready := meta.FindStatusCondition(app.Status.Conditions, "Ready")
require.NotNil(t, ready)
assert.Equal(t, metav1.ConditionTrue, ready.Status)
assert.Equal(t, string(component.Healthy), ready.Reason)
```
`"Ready"` is your own aggregate condition type here, not a framework-fixed name; a single component's condition type
reads the same way. `meta.FindStatusCondition` is right because the test is a consumer of the persisted object; inside
the controller, aggregation reads component conditions with `(*Component).GetCondition`. Assert the reason as well as
the status: the reason is a `component.Status` value, so it pins which component governed the result. Two assertions
belong at this layer and nowhere else: a not-yet-reconciled component holds the owner at `False`, and a component built
with `Suspend(true)` reports `True` while its resource is gone from the cluster.
## Anti-patterns
- **Regenerating goldens to make a failing test pass without reading the diff.** A golden mismatch means the rendered
output changed; run with `-update` only after confirming the new output is the output you intend, not as a reflex to
clear a red test.
- **Asserting on implementation internals instead of rendered output.** Golden and goldengen tests exist to pin what
gets applied to the cluster; asserting on builder state or mutation call counts instead of the previewed object misses
the thing that actually matters to a consumer.
- **Skipping the version matrix for version-gated mutations.** A single golden at one version cannot prove where a gate
boundary sits; a version-gated mutation without a matrix (or without `Requires`/`Forbids` pinning both sides of its
boundary) is unverified at the versions that matter most.
## Ground truth
The consumer's resolved module version is the source of truth, not these docs. Before asserting an exact signature,
method name, or option:
1. Read the framework version from the consumer's `go.mod` entry for
`github.com/sourcehawk/operator-component-framework`.
2. Verify the symbol with `go doc github.com/sourcehawk/operator-component-framework/pkg/<package> <Symbol>`.
The reference files bundled with this skill match the framework version this plugin shipped with. When they disagree
with `go doc`, `go doc` wins.
## References
- `references/testing.md`: full testing documentation. Read when you need exact `goldengen` config field semantics, the
completeness accounting rules, or the YAML matrix file format.
Pliki w paczce (2)
- SKILL.md 16 KB
- references/testing.md 28 KB
Wersje
W bazie nie ma innych skilli o nazwie testing-operators.
Migawka bazy z 13.11.2026 · wersja paczki c7851fcc5c30 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 90devops-kubernetes-operators
Buduje operatory i własne kontrolery Kubernetes: CRD, pętle uzgadniania, finalizatory, webhooki i testy envtest. - 90mobile-testing
Planuje testy aplikacji mobilnych: jednostkowe, widgetów, integracyjne, E2E, snapshoty i integrację z CI. - 100kubernetes-operator-development
Prowadzi budowę Operatorów Kubernetes w Kubebuilder lub Operator SDK: projekt API CRD, szkielet kontrolera, pętla uzgadniania, status i conditions, finalizery oraz testy z envtest. - 100kubernetes-operator
Projektuje, buduje i obsługuje operatory Kubernetes: własny kontroler, wybór frameworka, projekt CRD, pętle uzgadniania i audyt operatora pod kątem antywzorców. - 100operators-and-crds
Wyjaśnia rozszerzanie Kubernetes przez CRD i kontrolery: wzorzec uzgadniania, CRD jako kontrakt API, kontrolery poziomowe oraz kiedy zbudować, kupić albo pominąć operatora. - 100mobile-testing
Opisuje wzorce testów E2E aplikacji mobilnych: Flutter integration_test, Patrol, Maestro, testy golden, macierz urządzeń i zarządzanie danymi testowymi.