---
name: survey-logic-and-flow-review
description: >
  Reviews and tests the routing, skip logic, piping, randomisation and terminate
  points in a survey before it is fielded. Use when someone says "check the
  routing", "test the survey logic", "review the skip patterns", "the survey is
  ready for testing", "who sees this question", "the bases do not add up", "the
  piped text is wrong", or when a fielded study produced a base nobody can explain.
category: 02 Instrument Design
ref: 02.05
tier: 1
inherits: [K2, K3, K4, K5]
---

# Survey Logic and Flow Review

## 1. One-line description
Traces every path a respondent can take through an instrument, tests them as a respondent rather than reading them as an author, and establishes that every question has a defined base, every base has a defensible denominator, and no path produces data that cannot be analysed.

## 2. What this skill is used for

**The research problem it solves.** Wording defects are recoverable and routing defects are not. A leading question produces data that is biased in a known direction and can be caveated. A routing fault produces data that is missing, mis-based or attributed to the wrong people, and there is nothing to caveat because there is nothing there. If a filter excludes a group that should have been asked, those responses do not exist and cannot be reconstructed at any price. If a question is reachable by respondents who should never have seen it, their answers are mixed into a base that is no longer what it says it is, and the resulting percentage is wrong in a way that looks entirely normal on a chart. The compounding problem is that routing determines the denominator for every downstream percentage, so a base defined loosely at design time becomes a table nobody can defend at analysis, and the argument about what the number means happens in front of the client. These faults survive every other review because a questionnaire is written and read as a linear document, while a respondent experiences it as one path out of hundreds. Reading it as an author will never find them. Only walking the paths does.

**Where it sits in the research lifecycle.** After the instrument is drafted and the wording audit is done, and after it has been programmed if it is going to be, and before soft launch. It is the last checkpoint at which an irrecoverable error is still free to fix.

**Typical use cases.**
- Testing a programmed instrument before soft launch, against a documented path matrix rather than by clicking through it twice.
- Reviewing routing at specification stage, before programming, when changes cost minutes rather than days.
- Checking that quotas, screen-outs and routing interact the way the sample plan assumes.
- Verifying piped text renders correctly on every path, including the paths where the source question was skipped.
- Establishing the base for every question in a form that can be printed on a table, before the analysis plan depends on it.
- Diagnosing a live or completed study where a base is smaller than expected, a question has more responses than its filter permits, or completion times vary in a way nobody can account for.

**Who uses it.** Research executives and managers doing the pre-field check; research directors signing off; data and scripting teams working from a routing specification; client-side insight managers who will have to defend the bases in a table; anyone who has had to explain to a client why a chart says n=41.

## 3. When to use it

- An instrument has been drafted or programmed and is about to be tested or fielded.
- The questionnaire contains more than a couple of filters, or any nested filter (a filter applied to a group already filtered).
- Quotas exist, and respondents can be screened out, quota-terminated or routed differently depending on how they answer.
- Text, brands, dates or previous answers are piped into later questions.
- Any list, block or module is randomised, rotated or split between cells.
- The study will report subgroup bases, and someone will ask what the denominator is.
- A tracker wave is being changed, and a routing change would silently alter the base of a trended question.
- A fielded study produced a base, a completion time or a response count that nobody can explain.

## 4. When NOT to use it

- **The instrument's content is not settled.** Testing routing on a questionnaire that is still gaining and losing questions wastes the test and produces a matrix that is out of date before it is read. Fix the content first with **02.01 Survey Questionnaire Design**, and audit the wording with **02.04 Question Bias Detection**, then route.
- **The problem is wording, not flow.** This skill does not audit for leading, loaded or double-barrelled questions, unbalanced scales or missing answer options. It will happily pass a perfectly routed, thoroughly biased instrument. **02.04** owns that, and both passes are required.
- **The problem is who is in the sample rather than which questions they see.** Quota frame construction, screening criteria, incidence, fraud controls and replacement policy belong to **02.06 Screener and Quota Design**. This skill audits how the quota and screening structure interacts with routing and what it does to the bases; it does not design the quota frame.
- **The instrument has no branching.** A linear questionnaire that every respondent completes in full needs a base statement and a length check, not a path matrix. Say so rather than manufacturing a matrix for a single path.
- **Fieldwork is complete and the sample is spent.** The output then is not a defect list but a base-integrity assessment for the analysis and the report: which questions have a defensible denominator, which do not, and which findings must not be reported. Say this plainly rather than delivering a pre-field review after the fact.
- **The routing depends on a sample plan that has not been agreed.** Where quota targets, cell sizes or the minimum reportable base are still open, the base consequence audit cannot be completed. **01.06 Sampling Strategy** first, or run the review and mark the base adequacy checks as unresolved.
- **The instrument is a qualitative guide.** Guides have flexible routing by design, and imposing path testing on one misunderstands the instrument. **02.02 Discussion Guide Design**.
- **The request is to sign off a study you cannot test.** Where the programmed instrument is not available and only a document was supplied, a specification review is possible and a logic test is not. State which was done, per **K4 §6.4**, and do not let a document review be recorded as testing.

## 5. Required inputs

**Required. Without these the skill cannot run.**
- **The full instrument including every routing instruction, base instruction, quota reference, randomisation instruction, piping reference and terminate point.** Question text alone is not reviewable for logic. Where the routing lives only in the programmer's head or the scripting tool, ask for it to be written out; that request frequently finds the first defects on its own.
- **The sample and quota plan**, including quota cells, targets, whether quotas are interlocking, and the minimum base the study has agreed is reportable. Without the last of these, routing can be checked for correctness but not for consequence, which is half the value.
- **Access to the programmed instrument, or an explicit statement that only the document was reviewed.** These are different deliverables and must not be confused.

**Optional, and what each one adds.**
- **The analysis plan (01.07).** Turns the base audit from a technical exercise into a check against the actual tables: every planned cut has a base, and every base can be traced to a route.
- **Incidence estimates.** Let the review predict which routes will be starved of respondents and which quota cells will fill first, both of which change the fieldwork plan rather than the instrument.
- **Previous wave routing.** Identifies where a routing change would silently rebase a trended question, which is the most expensive undetected error in tracker work.
- **Device and mode profile.** Determines whether a long path is a length problem, and whether piping and randomisation behave the same way on every device.
- **The data map or expected export structure.** Lets the review confirm that what the instrument collects is what the analysis will find in the file, including how terminates, quota-fulls and partials are coded.
- **Fieldwork window and cost model.** Converts the length-by-path finding into a cost consequence, which is the form in which it gets acted on.

## 6. Questions to ask before starting

1. **What is the minimum base this study will report on?** Everything in the base consequence audit depends on it, and it is a decision the study has usually not made explicitly. Default: flag any route yielding under 100 as directional-only and any under 30 as unreportable as a percentage, per **K4 §7**, and state that these are the defaults applied.
2. **Are the quotas interlocking or independent, and at what point in the instrument are they counted?** Determines whether a respondent can pass the screener and be terminated later, and whether late-filling cells get a systematically different sample. Default: assume independent quotas counted at the end of the screener, and flag the assumption as one to verify with the field team.
3. **Which questions are trended, and has any routing into them changed?** A rebased trend is a wrong number presented as a comparison. Default: assume every repeated question is trended and check its base against the previous wave.
4. **What happens to a screen-out, a quota-full and a quality terminate: are they recorded, and separately?** Determines whether incidence can be measured at all and whether the sample can be audited. Default: assume they are recorded together, which is the common failure, and flag the need to separate them.
5. **What is randomised, and is the order recorded in the data?** Unrecorded randomisation means position effects cannot be checked or corrected. Default: assume rotation is applied and not recorded, and flag it.
6. **Can any respondent reach the end without answering anything substantive?** A path made entirely of "prefer not to say", "don't know" and "none of these" is a complete in the field report and nothing in the data. Default: assume it is possible unless the routing proves otherwise, and test for it explicitly.

## 7. Step-by-step methodology

**Step 1. Build the logic inventory.** Before tracing anything, extract every logical element into one table, because routing defects hide in the gaps between documents. One row per question, with: question number, base condition stated as a formal expression of prior answers, entry routes (which questions can lead here), exit routes (where each answer sends the respondent), quota references, randomisation or rotation instructions with pinned items, piped elements with their source question, terminate conditions, and whether the question is mandatory or skippable. A correct result is an inventory in which every routing instruction in the document appears exactly once, and every question has an explicit base condition. Questions whose base is written as prose ("asked of relevant respondents", "those who use the service") are marked as undefined; that phrase is not a base, and every one of them is a defect until it is expressed as a condition on specific answers to specific questions.

**Step 2. Restate every base as a formal condition and as an analysis denominator.** For each question write two things: the machine-checkable condition (Q4 = 1 or 2, and Q7 ≠ 5), and the sentence that will sit under the chart ("those who bought in the category in the past three months and did not use the retailer's website"). These must describe the same set. Where they do not, the disagreement is the defect, and it is usually the analyst's sentence that is right about what was wanted and the condition that is right about what was built. Then check the base is one a reader can hold in their head. Nested filters compound: a question asked of "those who considered switching, among those who have a contract, among those aware of the brand" has three multiplied incidences, produces a base nobody predicted, and will be reported as a percentage of something. Name the denominator now, in writing, because at analysis the choice of denominator will be made under time pressure by whoever builds the table.

**Step 3. Map the paths and build the path matrix.** Enumerate every branch point, then construct the set of paths that must be tested. Full enumeration is usually impossible (ten binary branches is over a thousand paths) and unnecessary. Cover, deliberately and in a documented matrix: every branch taken in both directions at least once; every combination of branches that jointly determine a base; every quota cell; every terminate point; the longest path; the shortest completing path; and every path a subgroup the analysis depends on will take. Add the adversarial cases, which is where most defects are found: the respondent who selects "none of these" everywhere it is offered, the one who selects "don't know" everywhere, the one who answers every question at the boundary value (the lowest qualifying age, the earliest qualifying date, exactly the threshold quantity), the one who selects every option in every multi-select, and the one who selects the single option that nothing was designed around. A correct path matrix names each test case, its persona in terms of the answers that define it, the questions it should see, the questions it must not see, the expected terminate or completion, and the expected length.

**Step 4. Detect the seven structural faults.** Walk the map looking for each, one fault class at a time.

| Fault | What it is | Why it is expensive |
|---|---|---|
| **Dead end** | A path that reaches a question with no valid answer available, or an exit with nowhere to go | Break-off, and the respondent is lost along with everything they had already given |
| **Orphan** | A question no path can reach | Silent. Nothing in the data reveals it; an objective is simply unmeasured, and it is found at analysis |
| **Over-reach** | A question reachable by respondents who should never see it | The base is contaminated by people the filter was supposed to exclude, and the percentage is wrong while looking normal |
| **Null completion** | A path that reaches the end without answering anything substantive | Counted as a complete, paid for, and empty |
| **Loop** | A route that can return a respondent to a question already answered | Duplicate records, inflated counts, and unpredictable data structure |
| **Unrecorded terminate** | A terminate that ends the interview without writing why | Incidence cannot be measured, screen-outs cannot be distinguished from quota-fulls, and the sample cannot be audited |
| **Contradictory condition** | Two routing rules that cannot both be satisfied, or that overlap so a respondent qualifies for two exits | Behaviour depends on the scripting tool's rule precedence, which is exactly the kind of dependency nobody documents |

**Step 5. Audit the base consequences, question by question.** This is the step that separates a logic check from a research review. For every filtered question, estimate the base it will actually deliver, using the study's incidence assumptions, and compare it against the minimum reportable base. Then examine four consequences. **Denominator drift:** does the same construct get reported on different bases in different places, so that two numbers in the same deck are not comparable. **Rebasing risk:** will anyone be tempted to report this as a percentage of total when it was asked of a subset, which is the most common single error in survey reporting and is created here, at design time, by an ambiguous base. **Subgroup collapse:** does the base survive the banner cuts the analysis plan requires, or does a route with 120 respondents become four cells of 30. **Trend integrity:** does the base of any repeated question match the previous wave exactly. Record every question whose base will be below the reportable minimum as a defect now, while it can still be fixed by widening a filter or moving a question, per **K5 §2.7** presenting the trade-off rather than deciding it.

**Step 6. Audit the quota and terminate structure against the routing.** Quotas and routing interact in ways that are invisible in either document alone. Check: **where each quota is counted**, and whether a respondent can pass the screener, invest five minutes and then be terminated, which is both an ethical and a data-quality problem. **What happens when a cell fills**, and whether the remaining fieldwork then draws a systematically different sample on every other variable, which it does, and which needs to be recorded for the analysis rather than discovered in it. **Whether screen-out, quota-full, quality terminate and drop-out are separately coded**, because incidence is measurable only if they are, and incidence is what the next study's feasibility and cost depend on. **Whether any terminate sits after a question that has already collected reportable data**, in which case that data exists for a partial sample and someone will eventually analyse it without knowing. And **whether quota membership itself depends on a routed question**, which creates the circular case where the answer that puts a respondent in a cell is only asked of people already in it.

**Step 7. Audit randomisation and its analysis consequences.** For every randomised or rotated element: confirm what rotates and what is pinned (non-substantive options such as "other" and "none of these" never rotate, and neither do items with a natural order); confirm whether the order presented to each respondent is written to the data file, because without it position effects cannot be detected or controlled; confirm that randomised blocks do not change the context of later questions differently for different respondents, which converts a controlled rotation into an uncontrolled context effect; and confirm that split-cell designs assign respondents in a way that keeps the cells comparable, and that cell membership is recorded. Where a randomised block sits above an unaided question, check that the rotation cannot leak the answer. State plainly what randomisation buys: it converts a fixed, directional bias into random noise the analysis can live with, and it does that only if the assignment is genuinely random and the record is kept.

**Step 8. Trace every piped element to its source.** Piping fails quietly and in front of the respondent. For every pipe, check: the source question is always answered on every path that reaches the pipe (the commonest defect is a pipe whose source sits behind a filter the piped question does not share); what renders when the source is blank, refused or "don't know"; whether the piped text is grammatical in every case, including plurals, articles, capitalisation, verb agreement and any gendered form; whether text piped from an "other, please specify" field can contain anything unusable, since that is free text a respondent wrote; whether piping from a multi-select handles one, several and many selections; and whether a pipe carries a brand, product or attribute name above the point where it should first be introduced, which is an exposure defect as well as a logic one. Test each pipe with a blank source, a maximum-length source, and a source containing an unexpected character.

**Step 9. Estimate length under every path, not on average.** The average length is the least useful number in the review, because nobody experiences it. Estimate the longest path, the shortest completing path, and the length for each subgroup the study depends on, using a stated timing convention labelled as a planning heuristic rather than a measurement. Two findings recur and both matter: the longest path frequently belongs to the most engaged, most category-involved respondents, who are exactly the ones the study most needs and who are being asked to do the most work; and a path that is far shorter than the rest often indicates a filter excluding people who should have been asked something. Report length by path, with the expected share of respondents on each.

**Step 10. Execute the test cases as a respondent, and record the evidence.** Walk each case in the path matrix through the live instrument, answering as that persona, recording what was actually seen against what should have been seen. Do not read the script and reason about it; the whole method rests on the difference between reading and doing, and a reviewer who reasons about the script will reproduce the author's assumptions exactly. Then check the data side: export the test records and confirm that the variables exist, that the base counts match the specification, that piped text and randomisation order were captured, that terminate reasons are separately coded, and that "not asked" is distinguishable from "asked and not answered", which is a distinction the analysis will need and which is silently lost by many exports. A correct result is a completed matrix with a pass or fail against every expected-seen and expected-not-seen condition, and a data file that reconciles.

**Step 11. Log every defect with its consequence, fix, retest, and control the change.** Each defect gets: where it is, what a respondent experiences, what the data will contain, whether it is recoverable at analysis, and the specific fix. Then, after any change, re-run every test case that touches the changed path, not just the one that failed. Fixes to routing break other routes at a high rate, and a partial retest is the mechanism by which most fielded logic errors reach the field. Freeze the instrument at sign-off, and treat any subsequent change, including a wording change that moves a question, as re-triggering the affected test cases. Record who signed off and against which version, per **K5 §2.8**.

## 8. Analytical framework

The review is built on one chain, applied to every question and readable in both directions:

    Respondent path → Questions seen → Base condition → Denominator → Reported figure

**Forward** is the test: a respondent with this profile takes this path, sees these questions, therefore belongs in these bases, which produce these denominators, which appear under these tables. The path matrix is the forward pass made explicit, one row per tested case.

**Backward** is the base audit, and it is the one that catches the expensive faults: this figure will be reported as a percentage of this denominator, which is this base, which is created by this routing condition, which is reached by respondents on these paths. The chain breaks most often at the fourth link, where the base condition as programmed does not describe the same set of people as the base sentence the analyst intends to print, and both parties believe they are describing the base.

Two rules govern the whole framework. **Every question has exactly one base, expressed as a condition on prior answers, not as a description.** And **every percentage has exactly one denominator, decided at design time and written down**, because a denominator chosen at analysis time is chosen by whoever is building the table at the time, under deadline, and per **K2 §7** that is a traceability break as well as an analytical one.

## 9. Output format

**1. Review scope.** What was reviewed (specification, programmed instrument, or both), which version, on which devices, and what was not reviewed.

**2. Verdict.** Ready to field, ready with the critical defects fixed, or not ready, with defect counts by severity.

**3. Logic inventory.**

| Q no. | Base condition (formal) | Base sentence (for tables) | Entry routes | Exit routes | Quota refs | Randomisation | Piped elements | Terminate conditions |
|---|---|---|---|---|---|---|---|---|

**4. Path matrix and test results.**

| Case | Persona (defining answers) | Questions expected | Questions that must not appear | Expected outcome | Expected length | Result | Notes |
|---|---|---|---|---|---|---|---|

**5. Defect log.**

| Ref | Location | Fault class | What the respondent experiences | What the data will contain | Recoverable? | Severity | Fix | Retest cases triggered |
|---|---|---|---|---|---|---|---|---|

**6. Base consequence table.** Every filtered question with its expected base, the minimum reportable base, whether it passes, and the banner cuts it will and will not support.

**7. Length by path**, with the expected share of respondents on each and the timing convention stated as a planning heuristic.

**8. Quota, terminate and randomisation findings**, including how each terminate type is coded and whether randomisation order is captured.

**9. Sign-off record.** Version tested, date, who tested, what changed since, and which cases were re-run.

**10. Open items and review points**, per **K5 §3**.

**When the inputs are thin**, the format is not filled in anyway. A base that cannot be expressed as a condition is recorded as `UNDEFINED` rather than guessed. An expected base that cannot be estimated because no incidence assumption exists is recorded as `[not available]` with what would be needed, per **K4 §2.1**. A specification review is never reported as a logic test.

## 10. Quality checks

Run before the instrument is signed off. These sit on top of **K4 §8**.

1. Every question has a base expressed as a formal condition on prior answers, and no base is written only as prose.
2. Every base condition and its table sentence describe the same set of respondents.
3. Every branch has been taken in both directions in at least one test case.
4. Every terminate point has been reached in a test case and its reason is separately recorded in the data.
5. No question is unreachable, and every question's entry routes are documented.
6. No respondent can reach a question they should not see, tested from the exclusion side as well as the inclusion side.
7. No path completes without a substantive answer, tested with the all-"don't know" and all-"none of these" personas.
8. Every filtered question's expected base has been estimated and compared against the minimum reportable base.
9. Every question's base survives the banner cuts the analysis plan requires, or the shortfall is logged.
10. Every trended question's base matches the previous wave exactly, or the change is logged as a rebasing.
11. Every pipe has been tested with a blank, a maximum-length and an unusual source value, on every path that reaches it.
12. Every randomised element has its pinned items confirmed and its presented order written to the data.
13. Quota counting points are documented, and the consequence of each cell filling is stated.
14. Length has been estimated for the longest and shortest paths, not only the average.
15. Every fix has been retested along with every other case touching the changed path, and the sign-off names the version tested.

## 11. Common failure modes

| Failure | How to recognise it | How to prevent it |
|---|---|---|
| **Reading instead of testing** | The review was done by reading the document and reasoning about the logic | Walk the paths as a respondent. A reader reproduces the author's assumptions exactly |
| **The happy path only** | Two or three sensible personas tested, all of whom qualify and answer normally | The adversarial cases in Step 3: all "none of these", all "don't know", every boundary value, every option selected |
| **Prose bases** | "Asked of relevant respondents", "those who are aware" | Every base restated as a condition on specific answers, in Step 2 |
| **Testing inclusion but not exclusion** | Every test confirms the right people saw the question; none confirms the wrong people did not | The path matrix has a must-not-appear column, and it is checked |
| **Base discovered at analysis** | The chart says n=41 and nobody predicted it | Step 5, run against incidence assumptions before fielding |
| **The rebased trend** | A repeated question's filter was widened and the trend now compares two different populations | Step 5's trend integrity check, run against the previous wave's routing, not its wording |
| **Piping tested with normal values only** | The pipe works for a respondent who answered everything | Test with blank, refused, maximum length and free-text sources |
| **Unrecorded randomisation** | Rotation was applied; the order is not in the data file | Confirm capture at Step 7 and again in the data-side check at Step 10 |
| **Terminates lumped together** | One completion status covers screen-outs, quota-fulls and drop-outs | Separate codes, confirmed in the export, or incidence is unmeasurable |
| **Partial retest** | One fix, one retest, and a new fault three questions downstream | Any change re-triggers every case touching that path |
| **Silent change after sign-off** | The instrument was corrected for a typo and a question moved | Version freeze, change control, and the sign-off names a version |
| **AI: plausible routing prose** | Generated routing instructions that read correctly and cannot be executed ("route relevant respondents to the appropriate section") | Require formal conditions. A routing instruction that cannot be written as a condition is not a routing instruction |
| **AI: unverifiable test claims** | A report that the logic was tested, produced without access to the programmed instrument | **K4 §6.4**. State whether a specification or a live instrument was reviewed; never let one be recorded as the other |
| **AI: invented incidence** | Expected bases calculated from incidence figures nobody supplied | **K4 §2.1**. Mark as `[not available]` and name what is needed |
| **AI: completeness by assertion** | A claim that all paths were traced, with no matrix | The matrix is the evidence. A path review with no documented case list has not happened |

## 12. AI guardrails

Skill-specific only. **K4** applies in full and is not repeated here.

1. **Never report that logic was tested when only a document was reviewed.** These are different deliverables with different assurance levels, and confusing them puts an untested instrument into field with a sign-off attached (**K4 §6.4**).
2. **Never produce a path matrix without listing the cases.** A statement that all paths were checked, unaccompanied by the case list and the pass or fail against each, is an assertion, not a review.
3. **Never invent incidence, expected bases or completion times.** Where the numbers to calculate an expected base are not supplied, write `[not available]` and name what would be required (**K4 §2.1, §2.4**).
4. **Never resolve an ambiguous routing instruction by choosing the sensible interpretation.** An ambiguous instruction is the defect. Record both readings, state what differs, and return it (**K4 §6.1, §6.2**).
5. **Never change routing to fix a base.** Widening a filter to reach a reportable base changes who is answering the question and what the answer means. Present the trade-off and leave it to the researcher (**K5 §2.7**).
6. **Never sign off an instrument as correct.** Report the cases tested, the results, and what was not tested. Untested paths exist in almost every review and their existence is part of the finding.
7. **Never treat a changed base on a trended question as a technical detail.** It is a break in comparability and it is reported to the researcher as one, with the affected series named.
8. **Never assume a scripting tool's default behaviour.** Rule precedence, blank piping, quota counting points and the handling of unanswered questions vary, and assuming one is a guess about a system you cannot inspect. Test it or state it as unverified.

## 13. Best-practice principles

1. **Test as a respondent, review as an analyst, and never read as an author.** The author's mental model is the thing being checked, so using it to do the checking guarantees the result.
2. **Routing errors are unrecoverable and wording errors are not.** This is why logic review deserves more time than it usually gets and gets less than wording review, which is more visible and less consequential.
3. **The routing is the denominator.** Every percentage in the eventual report was decided here, by a filter, months before anyone built a table.
4. **A base written in prose is not a base.** "Relevant respondents" is a placeholder that survives to analysis and then becomes an argument.
5. **Test what must not appear, as deliberately as what must.** Over-reach is invisible in the data and produces a contaminated base that looks completely normal.
6. **The adversarial respondent finds the defects.** The person who selects "none of these" every time, or answers "don't know" throughout, or sits exactly on a boundary, is worth ten sensible test personas.
7. **Nested filters multiply, and nobody's intuition survives three of them.** Calculate the expected base rather than estimating it, and do it before fielding rather than after.
8. **Randomisation only helps if the order is recorded.** Unrecorded rotation converts a measurable position effect into unattributable noise.
9. **Piping breaks in front of the respondent.** It is the only defect class the participant sees directly, and it costs credibility as well as data.
10. **Length is a property of a path, not of an instrument.** The average tells you nothing about the experience of the respondents you most need.
11. **A fix is a new defect until it is retested.** Retest the whole affected path, not the question that failed.
12. **Terminates are data.** Screen-outs, quota-fulls and quality removals separately coded are how incidence is measured, how the next study is costed, and how a sample can be defended.

## 14. Worked example

**INPUT**

A fictional home improvement retailer, Thornhill Home and Garden, is fielding a category study of 900 respondents ahead of a range decision. The instrument has 32 questions, quotas on region and on shopper type (recent purchaser, browser, non-shopper), a screener terminating non-shoppers of the category, filters on which retailer was last used, and text piping the last-used retailer name into six later questions. The analysis plan requires reporting by shopper type and by whether Thornhill was the retailer used.

**PROCESS**

*Step 1, inventory.* Thirty-two rows built. Four questions had bases written as prose: "asked of relevant respondents" appeared twice, "those familiar with the category" once, and one question had no base instruction at all, which turned out to mean the author intended it for everyone and the programmer had filtered it. All four marked `UNDEFINED` and returned for a formal condition.

*Step 2, bases.* Q19, on satisfaction with the last purchase, had the formal condition "Q7 = 1" (bought in the past three months) but the analyst's table sentence read "recent purchasers who bought in store". Those are different sets: Q7 did not distinguish channel. Caught here, this is a five-minute fix. Caught at analysis it is an argument in front of a client about what a chart means.

*Step 3, path matrix.* Fourteen branch points. Nineteen test cases constructed: each branch both ways, each quota cell, each terminate, longest path, shortest completing path, plus five adversarial personas.

*Step 4, structural faults.* Two found. A dead end at Q11: respondents who selected "none of these" at Q10 were routed to Q11, which asked which of the retailers named at Q10 they used most, and offered no valid answer. An over-reach at Q24: the filter was written as "Q7 = 1 or Q8 = 1", where the intent was "and", so browsers who had not bought were being asked about their purchase experience. The over-reach was rated critical, because it does not break anything visible: those respondents would have answered something, and the resulting base would have been larger than expected and quietly wrong.

*Step 5, base consequences, and the judgement call.* The expected base for Q26, asked of recent purchasers who used a competitor and were aware of Thornhill's range, calculated to approximately 70 on the study's own incidence assumptions, against a minimum reportable base of 100. The analysis plan required this question split by region, which would have produced cells in the twenties. The team's first instinct was to widen the filter by removing the awareness condition. This was recorded as a trade-off rather than applied: removing the condition would have included respondents answering about a range they had never seen, which does not produce a larger base for the same question, it produces a different question with a larger base. Two honest options were presented, boost the sample of that subgroup or report the question on total without the regional split, and marked **RESEARCHER DECISION REQUIRED** per **K5 §2.7**, because it trades cost against granularity.

*Step 6, quotas.* The shopper-type quota was counted at Q9, four minutes into the interview, so quota-full respondents were being terminated after answering nine questions, including two that would be analysed. Flagged on two grounds: those nine questions exist for a partial sample that someone will eventually analyse without knowing, and terminating engaged respondents four minutes in is an ethical and incentive problem as well as a data one. Recommended moving the quota count to the end of the screener. Separately, screen-outs and quota-fulls shared a single completion code, which would have made category incidence unmeasurable and left the next study to be costed on a guess.

*Step 8, piping.* The last-used retailer name piped into six questions from Q12. Two defects. Q12 permitted "other, please specify", so free text a respondent typed would render mid-sentence in six later questions, including anything unusable. And one path reached Q22 without passing Q12 at all, so the pipe would render blank, producing "How satisfied were you with your visit to ?".

*Step 9, length.* The longest path ran roughly 40% longer than the average and belonged to recent purchasers who had used more than one retailer, which is the highest-value subgroup in the study.

*Steps 10 and 11.* All nineteen cases walked in the live instrument. Six defects logged, fixed, and every case touching a changed path re-run, which caught a seventh: the fix to the Q11 dead end had routed "none of these" respondents past Q13, a question they should have seen.

**OUTPUT**

A logic inventory of 32 rows with four bases returned as undefined; a path matrix of 19 documented cases with pass or fail against expected-seen and expected-not-seen; a defect log of seven entries (two critical, three major, two minor) each with the respondent experience, the data consequence and the fix; a base consequence table flagging two questions below the reportable minimum; length by path; quota and terminate findings covering the counting point and the shared completion code; a sign-off record naming the version tested and the cases re-run; and one **K5** decision point on boosting sample versus dropping a regional split.

## 15. Advanced usage

**Tracker waves.** The highest-risk change in tracker work is not a wording change, which everyone notices, but a routing change, which nobody does. Before any wave, diff the routing against the previous wave question by question and treat every difference as a potential rebasing. A filter widened by one option, a quota counted at a different point, a question moved to a different position in a randomised block: each changes the base or the context of a trended measure while the question text stays identical, and the resulting movement will be reported as a change in the market. Where a routing change is genuinely necessary, it is a parallel-run decision, not a scripting decision.

**Complex quota structures.** Interlocking quotas multiply cells fast, and a fully interlocked frame with three variables can leave small cells that fill late or never. The routing consequence is that the last portion of fieldwork draws from a progressively narrower pool, so the final respondents differ systematically from the first on everything not being quota-controlled. Review this as a design property rather than a fieldwork problem: identify which cells will fill last, estimate what the residual pool looks like, and record it for the analysis. Where a cell cannot realistically be filled, that is a finding for **02.06** and **01.06**, not a reason to relax a filter mid-field.

**Split-cell and experimental designs.** Where an instrument carries a designed split (two wordings, two orders, two concepts), the logic review acquires an extra obligation: confirm that assignment is genuinely random rather than sequential, that cell membership is written to the data, that the cells are balanced on quota variables, and that nothing downstream of the split treats the cells as one population. Confirm too that the split does not interact with routing, which happens whenever a respondent's cell determines which questions they can reach and the analysis then compares them.

**Reviewing after fielding.** Where a study is complete and a base looks wrong, the data is the evidence. Cross-tabulate each filtered question against its filter to find respondents who answered a question they should not have reached, and count respondents who should have reached a question and have no record. Compare completion times by path against the estimates. Inspect the distribution of randomisation positions for imbalance. The output is a base-integrity statement naming which findings are safe, which need rebasing, and which cannot be reported, plus a routing specification for the next wave.

## 16. Skill chain

**Recommended previous skills**
- **02.01 Survey Questionnaire Design.** Hands over the instrument with routing specified at design level, and the base and analysis-output columns of the mapping grid.
- **02.04 Question Bias Detection.** Should run first, because wording changes move questions and moving questions changes routing.
- **02.06 Screener and Quota Design.** Hands over the quota frame, the screening criteria and the terminate structure this review tests against the routing.

**Recommended next skills**
- **03.04 Fieldwork Monitoring and Response Quality.** Takes the expected bases, path lengths and terminate codes and watches them in field, where the first live data confirms or contradicts every estimate made here.
- **01.07 Analysis Plan Development**, revisited, where the base consequence audit changes what can be reported.

**Runs well alongside**
- **04.05 Weighting and Base Management**, downstream, which inherits the base definitions this review fixes.
- **01.06 Sampling Strategy**, wherever a route cannot deliver a reportable base and the answer is sample rather than routing.
- **02.07 Scale and Measurement Selection**, where a randomised or split-cell scale test is carried inside the instrument.

---
A Yazi Supplied Skill and resource.