# Evidence rules and the register

The architecture is a two-stage pipeline: extraction builds a structured evidence register,
and drafting writes the document from the register only. Any statement that cannot be
anchored to a register entry gets cut. This is what makes "every claim is traceable"
mechanical rather than aspirational, and it is what makes two runs on the same inputs
produce the same statuses.

## Evidence typology

Every evidence item gets a tier and an epistemic status.

| Tier | Description | Examples |
|------|-------------|----------|
| 1 | Other-rated observed behavior | 360 ratings and verbatims, reference interviews, case study or simulation performance, prior assessment center observations |
| 2 | Structured self-report | Psychometric instruments (personality, values, cognitive style; any vendor) |
| 3 | Self-reported behavioral accounts | Interview descriptions of specific past situations and actions |
| 4 | Self-characterization | Interview claims about own style, strengths, preferences |
| 5 | Record (tier code "R" in prose, stored as 5) | Biodata: education, tenure, role history. Epistemic status "fact" only if a source confirms it; otherwise "self_report" and reported "as stated" |

Epistemic statuses: `fact` (verified record), `observation` (tier 1 content),
`self_report` (tiers 2 to 5 unverified, with tier noted), `inference` (a conclusion drawn
by connecting evidence items), `hypothesis` (a possible interpretation the available
evidence cannot confirm; always flagged for probing and routed to Open Questions).

Weighting rules that follow from the typology:

- A theme supported by two or more independent sources, at least one of them tier 1, is
  the strongest class of finding and may be stated with confidence, still attributed.
- Psychometric findings uncorroborated by behavioral evidence are dispositional
  self-report ("the personality data suggest a preference for..."), never observed
  behavior.
- Interview self-characterization alone never supports a Consistent Strength or a
  Development Area.
- Cognitive ability results, where provided, are reported factually with their published
  interpretation, in lay terms in the body, with instrument names and scales confined to
  the methodology section.
- Tiers are working machinery, not reader vocabulary: they live in the register, the
  status computation, and the chat coverage playback, and never appear as tier codes in
  the document. The document's tables speak in lay terms (independent sources, observed
  behavior).

## Register schema

Write the register to the working directory as `register.json`:

```json
{
  "candidate": "Full name as it should appear",
  "role": "Role title",
  "organisation": "Organization name",
  "run_date": "8 Jul 2026",
  "audience": "search committee",
  "criteria": [
    { "id": "C1", "name": "Strategic Vision", "definition": "One-line definition",
      "origin": "user-provided" }
  ],
  "coding_frame": [
    { "criterion_id": "C1",
      "indicators": [
        { "id": "C1-E1", "polarity": "effective",
          "text": "Articulates a multi-year direction that others can accurately repeat",
          "origin": "derived-confirmed" },
        { "id": "C1-I1", "polarity": "ineffective",
          "text": "Presents plans that stay at the level of vision statements",
          "origin": "derived-confirmed" }
      ] }
  ],
  "context": { "summary": "Short mandate paragraph from Gate 3. Always populated." },
  "sources": [
    { "id": "S1", "name": "360 feedback report", "type": "360",
      "provider": "Vendor name", "date": "May 2026", "tier": 1,
      "targets": ["C1", "C3"],
      "extraction_notes": "Anything odd about how this source was read" }
  ],
  "evidence": [
    { "id": "E1", "source_id": "S1",
      "summary": "Paraphrase of the evidence item in the skill's words",
      "verbatim_anchor": "optional short quote from the user's own material",
      "epistemic_status": "observation", "tier": 1,
      "context_note": "optional situational note",
      "coding": [
        { "criterion_id": "C1", "indicator_ids": ["C1-E1"], "valence": "positive" }
      ] }
  ],
  "divergences": [
    { "criterion_id": "C1", "evidence_ids": ["E1", "E7"],
      "pattern": "self_vs_other",
      "hypothesis": "One possible reading, phrased as a hypothesis to probe" }
  ]
}
```

Field rules:

- **Coding replaces free valence judgment.** Each evidence item carries one coding entry
  per criterion it bears on. For a real criterion, `indicator_ids` name the matched
  indicators from that criterion's frame; `valence` is computed from their polarities
  (only effective: positive; only ineffective: negative; both: mixed). The validator
  recomputes every valence and fails on mismatch, so a hand-edited valence cannot survive
  the build. An item can carry different valences for different criteria (decisive for
  one criterion, corrosive for another); status computation consumes the per-criterion
  valence.
- **No indicator matched.** A coding entry may carry `indicator_ids: []` with a required
  `coding_note` and valence "neutral" (counts for coverage, not direction). If the same
  uncoded theme appears in two or more items, propose a frame addition to the user; on
  confirmation the indicator joins the frame with origin
  "added-during-extraction-confirmed" and the items are recoded. The frame never grows
  silently.
- **Targeted-source coding.** Where a source carries `targets`, its evidence codes first
  and primarily against those criteria. A coding entry against a non-targeted criterion
  must name the matched indicator and carry a one-line `justification`; general relevance
  is not enough, and the validator warns when the justification is missing. Untargeted
  sources (most references, biodata) code freely. Rationale: published evidence shows
  that rating everything against everything collapses construct discrimination, while
  targeted rating preserves it at almost no cost to convergence.
- **Pseudo-criteria.** Coding entries against MOTIVATION, DERAILER, or ONBOARDING carry
  no `indicator_ids`; their valence is extractor-assigned. They get no status and no
  criterion section; they feed the motivation, development, and onboarding sections. They
  are exempt from targeting.
- `divergences.pattern` is one of `self_vs_other`, `context_bound`, `temporal_change`,
  `instrument_disagreement`. Interpret divergence, never just flag it, and always as a
  labeled hypothesis.
- Do not put anything in the register that violates the protected-characteristics rule
  below. The validator screens for it, but the screen is a backstop, not the rule.

## Extraction procedure

Process sources one at a time. For long transcripts, work in passes: read a chunk, extract
evidence items, continue. Extraction from psychometric PDFs that are image-heavy or
chart-based is error-prone; read them with the pdf-reading approach and treat every number
with suspicion until verified.

**One-item-at-a-time coding discipline.** Code one evidence item at a time against the
frame: restate the candidate indicators for the criteria the item bears on, decide the
match, record the entry, move on. Never sweep a source and assign codings
impressionistically in bulk; published evidence shows batch evaluation degrades quality
and overweights construct-irrelevant features. One-candidate-per-run already provides
applicant-level independence; this extends the discipline to item level.

**Construct-irrelevance guard.** Code the behavior the item evidences, never the
eloquence, length, or confidence of the source text, except where an indicator itself
names such behavior. A long, fluent, assertive answer with no indicator behavior codes to
nothing; a short plain answer with clear indicator behavior codes to its indicator. Names
and identity details play no role in coding.

**Psychometric verification checkpoint (mandatory).** After extracting any psychometric
source, play back to the user: instrument name, scale names, score format (sten,
percentile, T-score, band), and the scores read. Ask the user to confirm or correct before
the data enters the register. Misreading a scale (a sten read as a percentile) is the
highest-severity failure mode for this skill's audience. If extraction fails or stays
ambiguous, ask the user to paste the key scores rather than guessing.

**Verification pass (mandatory).** After extraction, run a second structured pass that
re-derives every coding from the frame text and the item summary, in item order, without
reference to the first pass's rationale: for each item, read only the summary and the
frame, state the matching indicator ids, compare to the recorded entry. List every
disagreement with a proposed resolution, present them at the coverage checkpoint, and
resolve them with the user before statuses are computed. The register stores the
reconciled coding only. This is the in-context approximation of the published ensembling
result; true independent-call ensembles belong to API deployments (see
`validation_program.md`).

**Validation.** Run the validator, which checks structure and the frame, recomputes every
coding valence from indicator polarity (failing on mismatch), screens for
protected-characteristic content, computes coverage, and assigns statuses mechanically:

```bash
python3 scripts/validate_register.py register.json register_computed.json
```

**Coverage checkpoint (mandatory).** The validator prints a criterion-by-source coverage
summary with the computed statuses, uncoded-item counts, and any cross-target codings
lacking justification. Play it back to the user before drafting, together with the
verification-pass disagreements and their resolutions: flag criteria with no tier 1
evidence or no evidence at all, and let the user add sources or accept the gaps. Draft
only after they confirm.

**Status corrections.** Statuses are computed, not chosen. If a computed status looks wrong
during drafting, the fix is a register correction (an item's valence, tier, or criterion
mapping was wrong), then a re-run of the validator. Never hand-edit a status in the
narrative; the builder cross-checks narrative statuses against computed ones and fails the
build on any mismatch.

## Protected characteristics

Age signals, health and disability, family circumstances and pregnancy, race, ethnicity
and national origin, religion, sexual orientation and gender identity, union membership,
and political views never enter the register or the document. This holds even when a
source volunteers the information and even when it appears decision-relevant to the
source. If removing such material leaves a criterion without usable evidence, mark the
criterion Insufficient Evidence with a neutral note. The rule is stated openly in the
methodology section as a design feature. The screening term list lives in
`data/protected_characteristics.json`; hits are review flags to resolve individually, and
obvious false positives (for example "race to market") may be kept after confirming.

## The register as deliverable

The register is delivered on request, not by default. Mention once, in the closing message
at delivery, that the structured evidence register behind the document is available if the
user wants the full audit trail.
