content-based-validation
/content-based-validation
Use when building validity evidence by linking selection-procedure content to the work domain — demonstrating that the procedure samples important work behaviors, activities, and/or worker KSAOs defined by an analysis of work. Covers defining the content domain, SME qualifications and linkage judgments, sampling the domain, fidelity/specificity, competency-model foundations, and evaluating content evidence. Triggers: "content validity", "content-based strategy", "work sample", "job knowledge test", "link test content to the job", "SME linkage ratings", "no criterion data available".
- Author
- openmatter-network ↗
- Repository
- agent-io-skills ↗
- Licence
- MIT
- Last repository change
- 01.06.2026
Assessment and security
- Files in this package
- 2
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 01.06.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Use when building validity evidence by linking selection-procedure content to the work domain — demonstrating that the procedure samples important work behaviors, activities, and/or worker KSAOs defined by an analysis of work. Covers defining the content domain, SME qualifications and linkage judgments, sampling the domain, fidelity/specificity, competency-model foundations, and evaluating content evidence. Triggers: "content validity", "content-based strategy", "work sample", "job knowledge test", "link test content to the job", "SME linkage ratings", "no criterion data available".
Triggers from the description
Original author description
Use when building validity evidence by linking selection-procedure content to the work domain — demonstrating that the procedure samples important work behaviors, activities, and/or worker KSAOs defined by an analysis of work. Covers defining the content domain, SME qualifications and linkage judgments, sampling the domain, fidelity/specificity, competency-model foundations, and evaluating content evidence. Triggers: "content validity", "content-based strategy", "work sample", "job knowledge test", "link test content to the job", "SME linkage ratings", "no criterion data available".
How it works
- When content evidence is and isn't the right tool
- SMEs define the domain — qualify them
- Process
- Evaluating content evidence
- Experience and training requirements as content
- Competency modeling
- Pitfalls
- Checklist
- See also
SKILL.md preview
--- name: content-based-validation description: >- Use when building validity evidence by linking selection-procedure content to the work domain — demonstrating that the procedure samples important work behaviors, activities, and/or worker KSAOs defined by an analysis of work. Covers defining the content domain, SME qualifications and linkage judgments, sampling the domain, fidelity/specificity, competency-model foundations, and evaluating content evidence. Triggers: "content validity", "content-based strategy", "work sample", "job knowledge test", "link test content to the job", "SME linkage ratings", "no criterion data available". version: "1.0.0" author: OpenMatter-Network license: MIT category: research tags: ["Community", "io-psychology", "personnel-selection"] permissions: [] --- # Content-based validation Evidence that **the content of a selection procedure is a representative sample of important work behaviors, activities, and/or worker KSAOs** defined by an analysis of work. The inference rests on the **fidelity of the procedure's content to the work** — not on an empirical predictor–criterion correlation. Often the strategy of choice when a job is unique, samples are too small for a criterion study, or the procedure is built directly from the work (e.g., work samples, job-knowledge tests). ## When content evidence is and isn't the right tool - Strong fit when procedure content is **closely linked to job content**. - A construct labeled an *ability or personality* characteristic does **not** by itself rule out a content strategy — but when content is **less clearly linked to job content**, other sources of evidence take precedence (e.g., criterion-related, internal structure). - Content evidence supports **construct interpretation** and generalization to job performance/ training when the procedure samples the work well. ## SMEs define the domain — qualify them The study's success hinges on SME qualifications. SMEs: - define the work domain (important behaviors, activities, worker KSAOs), - participate in the analysis of work, and - judge the link between selection-procedure content and work/worker requirements. SMEs must be **competent to perform the work** and knowledgeable about relevant organizational characteristics (shift, location, equipment, software). For behavior definition, include people fully knowledgeable about the work. **Standardize and document** the method for translating SME judgments into the procedure; if SME ratings evaluate the content–work match, standardize the rating procedures and criteria for each aspect. ## Process 1. **Confirm feasibility.** Are the work and worker requirements reasonably **stable**? Can irrelevant content be minimized? Are qualified, unbiased SMEs available? Do cost/time allow the needed fidelity? 2. **Define the content domain** from accurate, thorough work information (behaviors, activities, responsibilities, prerequisite KSAOs). The domain need **not** include everything done on the job. State which important elements are included, how each links to the procedure, and **why any parts were excluded**. Ground the definition in organizational needs, labor markets, and specifications. 3. **Decide what the candidate must already have vs. will be trained.** Base the procedure on the analysis that defines the **balance** between KSAOs expected at entry and training the organization provides. (Read-the-manual example: if employees are trained on the technical manuals, test *reading ability* to the required level — not manual-specific knowledge.) 4. **Develop or choose the procedure; sample the domain.** Random sampling of the domain is usually neither feasible nor appropriate. Instead, sample the **most important** behaviors/KSAOs or those **prerequisite** to others, or a smaller set predictive of critical outcomes. Document the rationale in a **test plan** specifying which KSAOs are measured by which methods. 5. **Set the level of specificity deliberately.** More fidelity to exact job components → stronger content evidence, but **less resilience to change** and less transferability. If the work is likely to change, build **less job-specific** content (e.g., software *principles/techniques* rather than one program). Measure skills and knowledge that are job-relevant, never irrelevant content, even when a full simulation isn't feasible (a single-step problem may substitute for a multistep work sample). 6. **Establish administration and scoring guidelines** in advance — especially when the procedure includes work behaviors or activities that raise administration/scoring challenges. 7. **Evaluate the content evidence** and document everything. ## Evaluating content evidence Content evidence rests on showing the procedure **adequately samples and is linked to** the important work behaviors/activities/KSAOs from the work analysis. The documented development methods **are** the primary evidence. The sufficiency of the match is a matter of **professional judgment** based on evidence collected in the effort. Determine **reliability** of performance on the procedure when feasible, choosing a reliability type that fits the measurement design and intended use (e.g., pass/fail vs. rank ordering). ## Experience and training requirements as content A procedure may include evidence of specific prior training, experience, or achievement — judged by the **relationship between the content of that experience and the content of the work**. Superficial resemblance is not enough: course titles and job titles may poorly indicate actual content or proficiency. Evaluate the **similarity of behaviors/activities/processes/KSAOs**, not labels. ## Competency modeling A competency model can found a content study **only if it is detailed and rigorous** — as rigorous as a traditional work-analysis project. Judge that rigor before relying on it. See `work-analysis`. ## Pitfalls - Including content not part of the work domain (standardization artifacts aside) or, worse, irrelevant content. - Over-specifying content for work that is about to change. - Resting on a thin competency model. - Treating face validity as content evidence (acceptability ≠ validity). - Failing to document *why* domain elements were excluded. ## Checklist - [ ] Work/worker requirements reasonably stable; feasibility confirmed - [ ] Content domain defined from thorough work information - [ ] Entry-vs.-trained KSAOs distinguished - [ ] Sampling rationale (importance/prerequisite) documented in a test plan - [ ] Specificity level chosen against likelihood of work change - [ ] SMEs qualified; linkage method standardized and documented - [ ] Administration/scoring guidelines set in advance - [ ] Reliability assessed where feasible - [ ] Excluded domain elements and their rationale recorded ## See also `work-analysis` (defines the domain) · `validation-planning` · `internal-structure-validation` · `generalizing-validity-evidence` · `technical-validation-report` *Source: Principles (5th ed., 2018), "Sources of Validity Evidence → Evidence for Validity Based on Content."*
Files in this package (2)
- README.md 1 KB
- SKILL.md 7 KB
Versions
The database has no other skills named content-based-validation.
Database snapshot: 13.11.2026 · package version fdcb78e8562d (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 90work-analysis
Use when analyzing work to support a selection effort — to identify worker requirements (KSAOs/competencies), define the work content domain, or develop job-relevant criteria. Covers choosing methods, level of detail, SME sampling, competency modeling, and documenting results.… - 90ai-validity-evidence
Use when determining what validity evidence an AI/ML selection tool needs and whether it has it — Concern 5 of Tippins, Oswald & McPhail (2021). Covers validity as the legal/business sine qua non when adverse impact exists, AI tools as "tests" requiring validity,…