# Evidence handling

This file governs how the review treats evidence. These rules are non-negotiable and
protect the credibility, neutrality, and constructiveness of every report.

## Framing: a constructive, neutral evidence review

The report is a constructive, evidence-based review that helps a practitioner understand
and strengthen their assessment strategy. It reviews an assessment, or a battery, against
recognized professional standards, reports what the evidence shows, and states plainly
where information was not found in the materials reviewed.

It never judges whether a tool is suitable, never rates, ranks, or endorses, and never
disparages a publisher. There is no verdict and no pass or fail. The goal is to complete
the picture and support the reader's own evaluation, not to cast doubt.

## The cardinal rule: never fabricate evidence

The review reports only what is present in the materials supplied (a technical manual,
the Deeper Signals evidence file for Deeper Signals tools) or what is surfaced by an
explicit, permitted web search. It never invents a study, a coefficient, a sample size,
or a citation.

- If a value is not in the materials and not found in permitted search, the status is
  "Not found in materials reviewed", never a guess.
- Never estimate a reliability or validity figure "based on typical instruments of this
  type." A plausible number is still a fabricated number.
- Never attribute a finding to a source you did not read. If search returns a claim, cite
  the specific source; if it does not, say the information was not located.
- It is always acceptable, and usually better, to record "Not found in materials reviewed"
  than to fill a gap.

## Absence is reported neutrally

"Not found in materials reviewed" means the information was not located in what was
reviewed. It is not a statement that the evidence does not exist, and it is not a
deficiency of the tool. Keep this framing everywhere: in the status cell, in any
narrative, and in the "Where additional information would strengthen the picture" section.

Two states must be kept strictly distinct in every finding:

1. Evidence reported but limited. The materials report something, and some aspects are
   present while others are not, or the reported detail is partial. Use "Partially
   documented" and describe, neutrally and with sourcing, what is present and what was
   not found.
2. Not found in materials reviewed. Nothing in the materials reviewed speaks to the area.
   Use "Not found in materials reviewed". Do not imply this is a problem with the tool,
   and do not imply it is acceptable either. It is simply what was available.

Never merge these two. A reader must always be able to tell whether partial evidence was
reported or whether nothing was found.

## Differential evidence availability

Evidence depth varies across instruments. A Deeper Signals tool documented in the full
Deeper Signals evidence file will naturally yield a fuller review than a competitor known
only from a public footprint. The "Scope and confidence" section must state this
explicitly so the review does not implicitly favor better-documented tools. A longer list
of "Documented" statuses for one instrument and more "Not found in materials reviewed"
for another reflects what was available to review, not the relative quality of the tools.

## Neutral status taxonomy

Each assessed area receives exactly one status. These describe the evidence that was
documented, not the quality of the tool. The content builder maps each to a calm,
non-semantic color (no red or green).

| Status | When to use |
|--------|-------------|
| Well documented | Comprehensive evidence is reported in the materials reviewed, consistent with what the standard describes |
| Documented | Relevant evidence is reported |
| Partially documented | Some evidence is reported; some aspects were not found in the materials reviewed |
| Not found in materials reviewed | The information was not located in what was reviewed (neutral) |
| Not applicable | The area does not apply to this method or use (state why) |

Builder keys: well_documented, documented, partially_documented, not_found, not_applicable.

Rules for using the taxonomy:

- The status describes the evidence. Substance, nuance, and any considerations live in the
  narrative and must be sourced and neutral. Where a model is debated in the literature,
  state that neutrally with citations rather than labeling the tool.
- Where a recognized benchmark exists (for example a commonly cited reliability
  threshold), the narrative may state the benchmark and the reported value side by side
  and let the reader compare. It must not declare a pass or fail.
- Keep "Partially documented" and "Not found in materials reviewed" strictly distinct.

## Sourcing and citation

- Deeper Signals tools: evaluate only from the supplied Deeper Signals evidence file
  (references/deeper_signals_factsheet.md). That file is authoritative for Deeper Signals
  instruments and must never be used to make claims about any other vendor's assessment.
- Manual-derived findings (any vendor): attribute to the manual by section where possible
  ("the technical manual reports alpha of 0.81 for the Composure scale").
- Search-derived findings: cite the specific source. Prefer peer-reviewed literature, the
  publisher's own technical documentation, and recognized reviews (for example Buros Mental
  Measurements Yearbook) over blogs or marketing.
- Consensus statements: when stating the state of the literature on a model, cite a
  representative source and frame it neutrally.
- Record every source in "Sources reviewed", marked as publisher document, peer-reviewed
  literature, or recognized review, so the reader can weight them.

## Publisher-agnostic and non-disparaging

- Apply the identical framework and the identical bar to every assessment. Brand
  reputation, market share, and popularity carry no weight.
- Never disparage a publisher, a competitor, or a named product. State findings about
  evidence, not judgments about companies. "No DIF analysis was located in the materials
  reviewed" is correct. "This vendor cuts corners on fairness" is not.
- Where a model is debated in the literature, report that neutrally with a citation and
  avoid pejorative characterizations.
- Never use the Deeper Signals evidence file to make claims, favorable or unfavorable,
  about another vendor's tool.

## Questions and next steps

Every review ends with forward-looking, constructive sections that turn what was not found
into action.

- Questions to ask the publisher. For each "Partially documented" or "Not found in
  materials reviewed" area, generate a specific, answerable, non-accusatory question the
  practitioner can put to the publisher, grouped by instrument.
- Suggested next steps. Practical actions split into immediate and short term (for example
  request specific evidence, confirm a job analysis, plan a local validation study, set up
  ongoing fairness monitoring). Next steps must never include fabricating evidence or
  overstating what a tool can do, and must never amount to a recommendation to buy or not
  buy a product.
