# Assessment Validity Coefficients and Combination Logic

These are the operational validity coefficients used by the skill. They are the single
source of truth for the predictive validity (r and combined R) that feeds the performance
stream of the model. The same table and combination logic are used for BOTH the current
process and the proposed battery. Value comes from the difference between the two (the
added validity), never from the absolute level of either one.

## Source

Sackett, P. R., Zhang, C., Berry, C. M., and Lievens, F. (2022). Revisiting meta-analytic
estimates of validity in personnel selection: Addressing systematic overcorrection for
restriction of range. Journal of Applied Psychology, 107(11), 2040 to 2068.

These are the values from the "Simplified" worksheet of the Assessment Validity workbook.
Use these exact values.

## Validity table (single-method operational validity)

| Selection procedure | Validity (r) |
|---|---|
| Structured interviews | 0.42 |
| Job knowledge tests | 0.40 |
| Empirically keyed biodata | 0.38 |
| Work sample tests | 0.33 |
| Cognitive ability tests | 0.31 |
| Integrity tests | 0.31 |
| Personality-based EI | 0.30 |
| Assessment centers | 0.29 |
| SJT (knowledge) | 0.26 |
| SJT (behavioral tendency) | 0.26 |
| Conscientiousness (contextualized) | 0.25 |
| Interests | 0.24 |
| Emotional stability (contextualized) | 0.23 |
| Ability-based EI | 0.22 |
| Rationally keyed biodata | 0.22 |
| Extraversion (contextualized) | 0.21 |
| Conscientiousness (overall) | 0.19 |
| Unstructured interviews | 0.19 |
| Agreeableness (contextualized) | 0.19 |
| Openness (contextualized) | 0.12 |
| Extraversion (overall) | 0.10 |
| Agreeableness (overall) | 0.10 |
| Emotional stability (overall) | 0.09 |
| Job experience (years) | 0.07 |
| Openness (overall) | 0.05 |

## Combining multiple methods into one battery validity (R)

When the customer selects more than one assessment method, the methods are combined into a
single multiple-R estimate. This is NOT a linear sum: predictors overlap, so adding a second
method adds less than its standalone validity.

### Assumption

All chosen methods share a uniform intercorrelation of **0.2** with each other. This is a
reasonable default for distinct assessment methods measuring related but non-identical
constructs.

### Formula

For predictors with validity vector r (each predictor's correlation with job performance)
and intercorrelation matrix Rxx (1.0 on the diagonal, 0.2 on every off-diagonal element),
the combined multiple R is:

```
R = sqrt( r^T  ·  Rxx^-1  ·  r )
```

This is the standard multiple-correlation formula from regression. The matrix inversion
handles the overlap automatically.

### Cap

The combined R is capped at **0.60**. No assessment strategy in this model can claim a
validity above 0.60, even with many stacked methods. This keeps the estimate defensible.

### Single method

If the customer chooses exactly one method, R is simply that method's validity from the
table above (no combination, no cap needed since all single values are below 0.60).

### Worked examples (uniform intercorrelation 0.2)

- Structured interviews (0.42) alone: R = 0.42
- Structured interviews (0.42) + cognitive ability (0.31): R ≈ 0.48
- Cognitive (0.31) + SJT knowledge (0.26) + conscientiousness contextualized (0.25):
  R ≈ 0.40
- Structured interview (0.42) + cognitive (0.31) + conscientiousness (0.25)
  + integrity (0.31): R ≈ 0.53
- Six strong methods (structured interview, job knowledge, biodata, work sample, cognitive,
  integrity): R = 0.60 (hits the cap)

Notice how each added method contributes progressively less, and a large battery is held at
the 0.60 ceiling. This non-linear behavior is the whole point of the 0.2 shared-correlation
adjustment and the cap.

## Naming an assessment type (aliases and categories)

Users rarely speak in Sackett's exact labels. The engine resolves common plain terms to a
canonical method (case-insensitive) through `METHOD_ALIASES`, so "a cognitive test", "an
aptitude test", and "Cognitive ability tests" all resolve to the same coefficient. Anything
that does not resolve raises a clear error rather than being silently dropped, so a method is
never quietly miscounted. When a user names a category, map it during intake and state the
mapping you used.

Two mappings are deliberate judgment calls (easy to change in one place):

- **A generic "personality assessment"** anchors to contextualized conscientiousness (0.25),
  the single most robust personality predictor of job performance, as a conservative
  representative value. If the user has a specific, locally validated figure for their
  personality instrument (for example a Deeper Signals Core Drivers validation study), use that
  instead and note it in the methodology.
- **A generic "values" or "motivation" assessment** anchors to Interests (0.24), the closest
  construct in the Sackett table (vocational fit), treated as a proxy. Substitute a validated
  value where one exists.

## Do not stack facets of a single instrument

The 0.2 intercorrelation and the multiple-R combination are appropriate for **distinct methods
measuring distinct constructs** (for example a personality assessment plus a cognitive test plus
a structured interview). They are NOT appropriate for stacking several facets of one instrument
(for example five traits from the same personality questionnaire), because facets of one
instrument share far more variance than 0.2, so combining them this way overstates validity.
Model one personality assessment as a single method (one representative coefficient), then
combine it with other distinct methods.



The `scripts/roi_engine.py` module implements this combination with numpy. It reads the
method names the user chose, looks up each validity from the table above, builds the
intercorrelation matrix, computes the multiple R, and caps it at 0.60. It does this twice,
once for the current process and once for the proposed battery. The added validity
(R_proposed minus R_current, floored at zero) is what drives the performance value.
