Skip to content
Skills Library
Management & product · Analysis

content-based-validation

/content-based-validation

Use when building validity evidence by linking selection-procedure content to the work domain — demonstrating that the procedure samples important work behaviors, activities, and/or worker KSAOs defined by an analysis of work. Covers defining the content domain, SME qualifications and linkage judgments, sampling the domain, fidelity/specificity, competency-model foundations, and evaluating content evidence. Triggers: "content validity", "content-based strategy", "work sample", "job knowledge test", "link test content to the job", "SME linkage ratings", "no criterion data available".

Licence
MIT
Last repository change
01.06.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
2
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 01.06.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Use when building validity evidence by linking selection-procedure content to the work domain — demonstrating that the procedure samples important work behaviors, activities, and/or worker KSAOs defined by an analysis of work. Covers defining the content domain, SME qualifications and linkage judgments, sampling the domain, fidelity/specificity, competency-model foundations, and evaluating content evidence. Triggers: "content validity", "content-based strategy", "work sample", "job knowledge test", "link test content to the job", "SME linkage ratings", "no criterion data available".

Triggers from the description

content validitycontent-based strategywork samplejob knowledge testlink test content to the jobSME linkage ratingsno criterion data available
Original author description

Use when building validity evidence by linking selection-procedure content to the work domain — demonstrating that the procedure samples important work behaviors, activities, and/or worker KSAOs defined by an analysis of work. Covers defining the content domain, SME qualifications and linkage judgments, sampling the domain, fidelity/specificity, competency-model foundations, and evaluating content evidence. Triggers: "content validity", "content-based strategy", "work sample", "job knowledge test", "link test content to the job", "SME linkage ratings", "no criterion data available".

How it works

  1. When content evidence is and isn't the right tool
  2. SMEs define the domain — qualify them
  3. Process
  4. Evaluating content evidence
  5. Experience and training requirements as content
  6. Competency modeling
  7. Pitfalls
  8. Checklist
  9. See also
Original SKILL.md headings · 111 lines, 891 words
SKILL.md preview
---
name: content-based-validation
description: >-
  Use when building validity evidence by linking selection-procedure content to
  the work domain — demonstrating that the procedure samples important work
  behaviors, activities, and/or worker KSAOs defined by an analysis of work.
  Covers defining the content domain, SME qualifications and linkage judgments,
  sampling the domain, fidelity/specificity, competency-model foundations, and
  evaluating content evidence. Triggers: "content validity", "content-based
  strategy", "work sample", "job knowledge test", "link test content to the job",
  "SME linkage ratings", "no criterion data available".
version: "1.0.0"
author: OpenMatter-Network
license: MIT
category: research
tags: ["Community", "io-psychology", "personnel-selection"]
permissions: []
---

# Content-based validation

Evidence that **the content of a selection procedure is a representative sample of important work
behaviors, activities, and/or worker KSAOs** defined by an analysis of work. The inference rests on
the **fidelity of the procedure's content to the work** — not on an empirical predictor–criterion
correlation. Often the strategy of choice when a job is unique, samples are too small for a
criterion study, or the procedure is built directly from the work (e.g., work samples, job-knowledge
tests).

## When content evidence is and isn't the right tool

- Strong fit when procedure content is **closely linked to job content**.
- A construct labeled an *ability or personality* characteristic does **not** by itself rule out a
  content strategy — but when content is **less clearly linked to job content**, other sources of
  evidence take precedence (e.g., criterion-related, internal structure).
- Content evidence supports **construct interpretation** and generalization to job performance/
  training when the procedure samples the work well.

## SMEs define the domain — qualify them

The study's success hinges on SME qualifications. SMEs:
- define the work domain (important behaviors, activities, worker KSAOs),
- participate in the analysis of work, and
- judge the link between selection-procedure content and work/worker requirements.

SMEs must be **competent to perform the work** and knowledgeable about relevant organizational
characteristics (shift, location, equipment, software). For behavior definition, include people
fully knowledgeable about the work. **Standardize and document** the method for translating SME
judgments into the procedure; if SME ratings evaluate the content–work match, standardize the rating
procedures and criteria for each aspect.

## Process

1. **Confirm feasibility.** Are the work and worker requirements reasonably **stable**? Can
   irrelevant content be minimized? Are qualified, unbiased SMEs available? Do cost/time allow the
   needed fidelity?
2. **Define the content domain** from accurate, thorough work information (behaviors, activities,
   responsibilities, prerequisite KSAOs). The domain need **not** include everything done on the
   job. State which important elements are included, how each links to the procedure, and **why any
   parts were excluded**. Ground the definition in organizational needs, labor markets, and
   specifications.
3. **Decide what the candidate must already have vs. will be trained.** Base the procedure on the
   analysis that defines the **balance** between KSAOs expected at entry and training the
   organization provides. (Read-the-manual example: if employees are trained on the technical
   manuals, test *reading ability* to the required level — not manual-specific knowledge.)
4. **Develop or choose the procedure; sample the domain.** Random sampling of the domain is usually
   neither feasible nor appropriate. Instead, sample the **most important** behaviors/KSAOs or those
   **prerequisite** to others, or a smaller set predictive of critical outcomes. Document the
   rationale in a **test plan** specifying which KSAOs are measured by which methods.
5. **Set the level of specificity deliberately.** More fidelity to exact job components → stronger
   content evidence, but **less resilience to change** and less transferability. If the work is
   likely to change, build **less job-specific** content (e.g., software *principles/techniques*
   rather than one program). Measure skills and knowledge that are job-relevant, never irrelevant
   content, even when a full simulation isn't feasible (a single-step problem may substitute for a
   multistep work sample).
6. **Establish administration and scoring guidelines** in advance — especially when the procedure
   includes work behaviors or activities that raise administration/scoring challenges.
7. **Evaluate the content evidence** and document everything.

## Evaluating content evidence

Content evidence rests on showing the procedure **adequately samples and is linked to** the
important work behaviors/activities/KSAOs from the work analysis. The documented development methods
**are** the primary evidence. The sufficiency of the match is a matter of **professional judgment**
based on evidence collected in the effort. Determine **reliability** of performance on the procedure
when feasible, choosing a reliability type that fits the measurement design and intended use (e.g.,
pass/fail vs. rank ordering).

## Experience and training requirements as content

A procedure may include evidence of specific prior training, experience, or achievement — judged by
the **relationship between the content of that experience and the content of the work**. Superficial
resemblance is not enough: course titles and job titles may poorly indicate actual content or
proficiency. Evaluate the **similarity of behaviors/activities/processes/KSAOs**, not labels.

## Competency modeling

A competency model can found a content study **only if it is detailed and rigorous** — as rigorous
as a traditional work-analysis project. Judge that rigor before relying on it. See `work-analysis`.

## Pitfalls

- Including content not part of the work domain (standardization artifacts aside) or, worse,
  irrelevant content.
- Over-specifying content for work that is about to change.
- Resting on a thin competency model.
- Treating face validity as content evidence (acceptability ≠ validity).
- Failing to document *why* domain elements were excluded.

## Checklist

- [ ] Work/worker requirements reasonably stable; feasibility confirmed
- [ ] Content domain defined from thorough work information
- [ ] Entry-vs.-trained KSAOs distinguished
- [ ] Sampling rationale (importance/prerequisite) documented in a test plan
- [ ] Specificity level chosen against likelihood of work change
- [ ] SMEs qualified; linkage method standardized and documented
- [ ] Administration/scoring guidelines set in advance
- [ ] Reliability assessed where feasible
- [ ] Excluded domain elements and their rationale recorded

## See also

`work-analysis` (defines the domain) · `validation-planning` ·
`internal-structure-validation` · `generalizing-validity-evidence` ·
`technical-validation-report`

*Source: Principles (5th ed., 2018), "Sources of Validity Evidence → Evidence for Validity Based on
Content."*
Files in this package (2)
  • README.md 1 KB
  • SKILL.md 7 KB

Versions

The database has no other skills named content-based-validation.

Database snapshot: 13.11.2026 · package version fdcb78e8562d (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 90work-analysis
    Use when analyzing work to support a selection effort — to identify worker requirements (KSAOs/competencies), define the work content domain, or develop job-relevant criteria. Covers choosing methods, level of detail, SME sampling, competency modeling, and documenting results.…
  • 90ai-validity-evidence
    Use when determining what validity evidence an AI/ML selection tool needs and whether it has it — Concern 5 of Tippins, Oswald & McPhail (2021). Covers validity as the legal/business sine qua non when adverse impact exists, AI tools as "tests" requiring validity,…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.