# Standards framework

This is the evaluation backbone of the review. Every assessment is reviewed against the
same core areas, grouped under pillars, with context modules switched on only when the use
case or instrument type calls for them. The same framework and the same bar apply to every
assessment regardless of publisher.

## Source standards

The framework synthesizes four widely recognized standards. Cite these by name when a
finding rests on them. Do not quote them at length; refer to the principle.

- AERA, APA, and NCME Standards for Educational and Psychological Testing (2014). The
  foundational standard. Frames validity as a unitary concept supported by multiple
  sources of evidence, and sets expectations for reliability, fairness, and documentation.
- SIOP Principles for the Validation and Use of Personnel Selection Procedures (5th ed.,
  2018). The applied employment-selection standard. Covers validation strategies, criterion
  measurement, validity generalization and transportability, and fairness.
- EFPA Test Review Model (current edition). The European framework used to describe
  documentation, norms, reliability, and validity.
- ITC Guidelines (International Test Commission), in particular on test use, on translating
  and adapting tests, and on computer-based and internet-delivered testing. The reference
  point for cross-cultural equivalence, language adaptation, and remote delivery.

These standards are mutually reinforcing. When they overlap (for example, all four expect
reliability evidence), treat the area as a single finding and name the standard most
specific to the context.

## Pillars and core areas (always assessed)

Group findings under these pillars in the findings table. Assess every core area for every
review.

### Pillar: Theoretical rationale
- Construct definition and theoretical grounding: are the constructs clearly defined and
  linked to a model or framework?
- Scientific consensus on the model: is the underlying model broadly accepted, or novel and
  unreplicated? Report the state of the literature neutrally, with citations. A model can
  be commercially popular and still be debated in the literature; state that neutrally and
  never as a label on the tool.

### Pillar: Reliability
- Internal consistency (Cronbach's alpha or McDonald's omega), reported per scale.
- Test-retest stability over a stated interval.
- Inter-rater agreement, where the method involves rater judgment.
- Standard error of measurement and whether score bands support individual interpretation.

### Pillar: Validity
- Construct validity: convergent and discriminant evidence; factor structure, ideally
  replicated.
- Content validity: a documented link to a job analysis for the target domain.
- Criterion validity: concurrent and predictive evidence; whether local or transported;
  sample sizes and any range-restriction corrections.
- Incremental validity: added value over other tools in use, or over the rest of a battery.
- Consequential validity: documented effects of use, including across subgroups.

### Pillar: Fairness
- Measurement invariance and differential item functioning (DIF).
- Subgroup mean differences and adverse impact (four-fifths rule).
- Differential prediction.
- Accessibility and accommodations.

### Pillar: Norms and scoring
- Norm recency, size, representativeness, and relevance to the target population.
- The basis for any cut scores or bands.

## Context modules (switch on by use case or instrument type)

Switch a module on only when it is relevant, so the report does not bloat. When a module
is not relevant, omit it rather than marking every area "Not applicable".

- Susceptibility to faking, coaching, and response distortion, plus any documented controls
  (relevant for self-report personality tools used in selection).
- Administration and standardization (proctoring, security, conditions of use).
- Applicant reactions and face validity (candidate experience, relevant for selection).
- Data privacy and automated-decision governance (relevant where the tool feeds automated
  or semi-automated decisions).
- Practicalities (time to complete, cost, languages available, vendor support, and whether
  local validation is offered).

## Use-case sensitivity: what the standards emphasize (not gating)

The intended use changes which areas the standards tend to emphasize. Use this to decide
which areas to foreground in the narrative and in "Considerations for your intended use".
It never produces a verdict and never gates the report.

- Selection (hiring, promotion, succession): the standards tend to emphasize criterion
  validity for the target role and population, a documented content link to a job analysis,
  and fairness evidence including adverse-impact monitoring in the setting of use. Reliability
  should support individual-level interpretation. Frame these as considerations the reader
  may wish to confirm, not as requirements the tool passes or fails.
- Development (coaching, team effectiveness, leadership development): the standards tend to
  emphasize construct validity, reliability, and the quality and clarity of feedback.
  Criterion validity and adverse-impact evidence are less central because no selection
  decision is being made. Still report what was and was not found, neutrally.

Decision weight (a hard screen, a decisive input, or one of several non-decisive inputs)
further shapes which standards are most relevant. A hard screen draws more attention to
criterion validity and fairness; a non-decisive input draws less. Use this only to frame
relevance, never to rule.

## Population sensitivity

- Geography: a test normed in one region used in another may call for local norms or
  evidence of transportability. Note single-region norms used more broadly, neutrally.
- Level: norms collected on a general working population may not match an executive or
  graduate population. Note a mismatch between the norm sample and the stated level.
- Language: a translated or adapted version calls for its own validation and equivalence
  evidence per ITC. Note any language requirement the materials do not cover.

## Deeper Signals evidence file

For Deeper Signals instruments, the authoritative source is
references/deeper_signals_factsheet.md. Use it to populate the core areas and relevant
context modules for Core Drivers, Core Values, the Core Reasoning Suite, and the competency
and job-profile layer. Do not use it to make claims about any other vendor's tool.
