---
name: sampling-strategy
description: >
  Designs who is studied and how many. Use for "what sample size do we need", "how
  many interviews is enough", "is this sample representative", "we need results by
  region and by segment", "what can we claim from a non-probability sample", "how
  many people do we need to detect that difference", "who is missing from this
  sample". Covers population and frame definition, coverage error, probability and
  non-probability selection and what each licenses, stratification, quotas and
  boosts, precision-driven and comparison-driven sizing, subgroup base
  requirements, weighting, incidence and feasibility, and non-response.
category: 01 Research Strategy and Design
ref: "01.06"
tier: 1
inherits: [K2, K3, K4, K5]
---

# Sampling Strategy

## 1. One-line description
Defines the population, the frame that will actually be used and the gap between them, selects a sampling mechanism and states precisely what it licenses you to claim, and sizes the study from the subgroups that must be reported rather than from the total.

## 2. What this skill is used for

**The research problem it solves.** Sampling failures are the least visible and least recoverable errors in research. A study can be well designed, well written and well analysed, and still be about the wrong people, because the population was never defined, the frame was accepted as given, or the size was set by the budget and then reported as though it had been derived. Two failures dominate. The first is the claim mismatch: a non-probability sample reported with the language of population estimation, complete with a margin of error that is arithmetically calculable and substantively meaningless. The second, and the more common in practice, is the sizing failure where the total works and the subgroups do not: a study of 800 that everyone is happy with until the analysis needs six sectors by three size bands, and every cell is unreportable. This skill defines the population first, measures the frame gap, states the licensed claim in one sentence, and derives the total from the cells rather than the cells from the total.

**Where it sits in the research lifecycle.** After method selection (01.04) and alongside plan development (01.05), because sample structure drives recruitment lead time. Its subgroup requirements are an input to the analysis plan (01.07) and its base thresholds constrain every claim the report will make.

**Typical use cases.**
- A brief specifies a sample size and the analysis it wants cannot be supported by it.
- A study must report on subgroups and the base per cell has never been worked out.
- The word "representative" is being used and needs to be either earned or replaced.
- A hard-to-reach population makes the ideal design infeasible and the alternative must be chosen and defended.
- A qualitative study needs a defensible number for a proposal without pretending saturation can be forecast.
- Weighting is proposed and its consequences for base sizes and testing have not been examined.
- An achieved sample differs from the designed one and someone must say what can still be claimed.

**Who uses it.** Quantitative research managers and directors, client-side insight leads reviewing designs, evaluation and policy analysts, UX researchers sizing mixed-method work, and postgraduate researchers writing a sampling section. It assumes no statistical background but does not simplify the parts where simplification would mislead.

## 3. When to use it

- A method is agreed and you need to determine who is studied and how many.
- Someone has asked for "a representative sample" and it is not clear of what.
- The analysis plan names subgroups and nobody has checked the bases they will require.
- Incidence is low, the population is specialised, or access is restricted.
- You need to decide between more sample and more depth on a fixed budget.
- A quota or boost design is proposed and its analysis consequences need stating.
- The study will be weighted and the effective base needs to be understood before, not after.
- Fieldwork has closed short and you must state what the achieved sample supports.

## 4. When NOT to use it

- **Before the method is selected.** Sample design follows the method, and a sampling plan written for an unchosen design will be rebuilt. Use **01.04 Research Method Selection** first. This skill also does not choose between qualitative and quantitative; it sizes whichever was chosen.
- **Before the population is a research question's population.** If the study's questions are still a topic, the population definition will be written to fit an assumed sample source, which is exactly backwards. Route to **01.02 Business Problem to Research Question**, which sets the population and unit of analysis, and return here to make it operational.
- **To select a supplier, panel, list source or fieldwork partner.** This skill has no view on sources and must not develop one. Source characteristics enter only as methodological properties: coverage, selection mechanism, and what is knowable about non-response.
- **To rescue a claim the sample cannot support.** Where a non-probability sample has already been fielded and someone wants a margin of error, the answer is that none is available, not a smaller one. Weighting a non-probability sample to population totals corrects composition on the weighting variables and does not convert it into a probability sample (K4 §2.5).
- **For statistical testing of the achieved data.** Sizing decides what could be detected; testing decides what was. Go to **05.02 Statistical Testing**, and for power calculation on an experimental design, to the experimental design skill referenced by 01.04 (**06.04**).
- **When the population cannot be reached at all by any honest route.** The output is that the study cannot be run as specified, with the population definition preserved so that the gap is visible. A weaker sample presented as an approximation of the right one is worse than no study, because the report will read as though it were about the intended population.
- **When the design is qualitative and the request is for a number that will be defended as a sample size.** Qualitative adequacy is governed by scope and saturation, not by power, and a fixed number presented as statistically derived is a false claim. This skill gives a range with a stated review point, and says why.
- **When the sample frame contains personal data whose use for research is not established.** Feasibility is not permission. Route to **13.05 Research Ethics and Consent Design** before any frame is used.

## 5. Required inputs

**Required.**
- **The population of interest**, as the research question defines it: inclusion rules, exclusion rules, the unit of analysis, the time reference and the geography. If it is not available, stop and derive it from the question rather than from the available list.
- **Every subgroup that will be reported or analysed separately**, from the analysis plan or the objectives. This is the input that determines the size of the study, and it is the one most often supplied late.
- **The frame or access route that will actually be used**, described in methodological terms: how membership arises, who could be in it, who could not.
- **The claim that must be supportable at the end.** "38% of the customer base, plus or minus 3 points" and "six of twelve participants described the same barrier" are different claims requiring different designs.

**Optional, and what each one adds.**
- **Known or estimated incidence of the target audience**, with its source. Converts feasibility from a guess into an estimate and frequently rules out a design immediately. Where absent, it is stated as unverified, never invented.
- **Prior variance or prior proportions on the key measures.** Sharpen precision calculations considerably; without them, sizing must assume the most conservative case and say so.
- **The smallest difference that would change a decision.** The single most valuable input to comparison-driven sizing, and a business judgement rather than a statistical one (K5 §2.1).
- **Population totals for weighting**, with their source and date. Determine whether weighting is possible at all and what its efficiency cost will be.
- **Historical response or cooperation rates for this audience and route.** Turn contact requirements from arithmetic into planning, and drive recruitment lead time in 01.05.
- **The design of any previous wave.** Where comparability matters, the sample specification is a constraint, not a choice.

## 6. Questions to ask before starting

1. **Who must the findings be about, and how is membership decided?** Why it matters: everything else is downstream, and an ambiguous population produces an unarguable study. Default if unanswered: adopt the narrowest defensible definition, state it, and flag that widening later requires new fieldwork.
2. **Which subgroups will be reported separately, and does a decision turn on each?** Why it matters: subgroup requirements set the total, and subgroups that exist only because a variable exists inflate the study for nothing. Default: assume total sample only, and state explicitly that no subgroup reading is supported.
3. **What claim must the findings support, and to whom?** Why it matters: it decides whether a probability design is necessary or whether description of the achieved sample suffices. Default: assume a sceptical audience, design to the higher standard, and say you assumed it.
4. **What is the incidence of this audience, and how do we know?** Why it matters: it determines feasibility, contact volumes and cost, and it is the assumption that most often breaks a fieldwork plan. Default: mark it unverified, make the design conditional on a feasibility check, and schedule the check first (01.05).
5. **What is the smallest difference that would change what you do?** Why it matters: comparison-driven sizing is impossible without it, and detectable-difference is not the same as decision-relevant difference. Default: do not invent one. Report the sizing as conditional and name this as the missing input.
6. **Will the data be weighted, on what, to what source?** Why it matters: weighting decisions change effective base sizes and therefore every claim's reliability. Default: assume unweighted, and state that any later weighting will reduce effective bases and require the reported bases to be revised.
7. **Who will be hardest to reach, and what do we do when that cell will not fill?** Why it matters: the decision made calmly now is better than the one made in week three of fieldwork. Default: define the fallback in advance (extend, relax, substitute, or report short with the base shown) and record which was chosen.

## 7. Step-by-step methodology

**Step 1. Define the target population operationally.**
Write inclusion rules, exclusion rules, the unit of analysis, the time reference and the geographic boundary. The test of a good definition is not elegance but reproducibility: could two people, given the same individual and the same rules, classify them identically? "Small business customers" fails. "Organisations with between one and forty-nine employees, holding an active account at any point in the twelve months to the fieldwork start date, with the respondent being the person who makes or influences purchasing decisions for that account" passes. Note the unit explicitly, because the analysis unit and the response unit are frequently different: the unit may be the organisation while the respondent is a person, and every finding then describes organisations through one informant, which is a limitation to state now rather than discover in the report.

**Step 2. Specify the frame and measure the gap.**
The frame is whatever will actually be used to reach people: a customer list, a membership register, an administrative database, an access route to an online population, a set of locations, a referral chain. Describe how membership in the frame arises, then write four things. **Undercoverage**: who is in the population but cannot be in the frame. **Overcoverage**: who is in the frame but not in the population, and how they will be screened out. **Duplication**: whether an individual or an organisation can appear more than once, and with what effect on selection probability. **Differential**: whether the undercovered group differs on the outcomes being measured, and in which direction. That fourth item is the one that matters. Coverage error is not a problem because someone is missing; it is a problem because the missing group differs. A correct result is a written coverage statement naming who is absent and the likely direction of the resulting bias, which travels into the report's limitations and into every claim about the total (K3 §6).

**Step 3. Choose the selection mechanism and write the licensed claim in one sentence.**
Probability designs give every population member a known, non-zero chance of selection: simple random, systematic, stratified, cluster, multi-stage, and selection with probability proportional to size. They license population estimation, quantified sampling error and confidence intervals, provided non-response is modest or modelled. Non-probability designs (quota, purposive, convenience, referral chains, self-selection into an available source) do not. They license description of the achieved sample, comparison between groups within it with the selection mechanism stated, and hypothesis generation. They do not license a margin of error, and no sample size makes them do so. Write the claim in one sentence and put it at the top of the sampling section: for example, *these findings describe the 812 people who responded from this source, whose composition matches national totals on age, sex and region after weighting; they are not a probability sample of the national population and no margin of error applies*. This sentence is the most useful single output of the skill, because it is what stops the report claiming more than the design allows (K4 §3.3).

**Step 4. Design the structure: stratification, quotas and boosts, with their consequences.**
These three are frequently confused and their analysis consequences differ. **Stratification** happens before selection: the frame is divided and a sample drawn within each stratum. It guarantees the presence of groups, reduces variance for the total where strata differ on the measure, and requires the design to be reflected in variance estimation, which means intervals and tests must account for it rather than assuming simple random sampling. **Quotas** operate during fieldwork on observable characteristics. They control the composition of the achieved sample, which is useful and limited: a quota fills the cell with whoever in that cell was willing and available, so it constrains composition without removing non-response bias, and its main risk is hiding that bias inside a total that looks correct. **Boosts** deliberately over-sample a subgroup so it can be analysed in its own right. They require weighting back for any total-level reporting, and the boosted cases often reach the study by a different route, which means they may differ systematically from the main sample in ways weighting does not fix. State that risk when a boost is used. A correct result names, for each structural element, why it is there and what it obliges the analysis to do.

**Step 5. Size the study, and say which logic applies.**
Two logics, and confusing them is the commonest sizing error after ignoring subgroups.

*Precision-driven*, where the study exists to estimate something. The inputs are: the required half-width of the interval on the key estimate, the expected value of that estimate (proportions near 50% need the largest sample), the confidence level, a finite population correction where the population is small relative to the sample, and the design effect from clustering, stratification and weighting. The relationships worth knowing without arithmetic: precision improves with the square root of sample size, so halving an interval requires roughly four times the sample; the gain from adding sample flattens sharply beyond a few thousand; and the design effect divides the sample into a smaller effective base, so a weighted study of 1,000 may carry the precision of 700.

*Comparison-driven*, where the study exists to detect a difference. The inputs are: the smallest difference that would change a decision (not the smallest detectable one, which is a different and much less useful quantity), the expected split between the groups being compared, the variability of the measure, the significance threshold, the desired power, and the number of comparisons planned, because multiplicity raises the effective bar (01.07). The relationship worth knowing: required sample rises steeply as the difference of interest shrinks, so halving the difference you want to detect roughly quadruples the sample needed, and an unequal split between groups is driven by the smaller group.

Do not produce a number without the inputs. Where they are missing, state which inputs are required and what each would change. A confidently stated sample size derived from nothing is a fabrication with a plausible face (K4 §2.1).

**Step 6. Size the subgroups, and derive the total from the cells.**
This is the real sizing task and the step that prevents the commonest failure in the field. List every cut that will appear in the report or in the analysis plan, including crossed cuts (region by segment, not just region and segment separately, because crossed cuts are what people ask for at the debrief). For each cell, establish the minimum base required by what will be claimed about it, applying the thresholds in K3 §3.2: below 100, directional reading only with the base shown; below 30, no percentages at all. Where the cell will be compared with another, the base required is set by the difference to be detected, not by a reporting minimum. Now build the total upward from the cells rather than downward from a headline number. The result is usually uncomfortable, and it should be presented as it comes out. Four honest resolutions exist when the derived total is unaffordable, and one of them must be chosen explicitly: report fewer cuts; boost the cuts that matter and weight back; combine categories in advance, deciding now rather than in the analysis how sectors or age bands merge; or accept that a given cut will be described qualitatively with counts and no percentages. Naming which was chosen is what prevents the analysis discovering the constraint later. This decision has a business component and carries a `RESEARCHER DECISION REQUIRED` marker per K5 §2.1.

**Step 7. Size qualitative work by scope and saturation, honestly.**
Qualitative sample size is not a power calculation and any formula presented as producing one should be treated as false precision. It is governed by two things. **Scope**: the number of distinct groups the question spans, built as a variation matrix (the characteristics across which experience plausibly differs, crossed only where the crossing matters), with enough sessions per cell to see whether a pattern repeats within it. **Saturation**: whether new sessions are still producing new material, which is judged during fieldwork against the code frame and cannot be forecast. The practical output is therefore a range with a stated review point: a planned number, a minimum below which the scope is not covered, and a defined moment (commonly after the first two-thirds) at which the team assesses whether to stop, extend or re-target. Say plainly in any proposal that the number is a planning estimate. Two prohibitions follow: never convert qualitative counts into percentages, and never present saturation as achieved without saying what was assessed and when (07.01, 07.03).

**Step 8. Decide weighting before fieldwork, not after.**
Three decisions, all made in advance. Whether the study will be weighted at all. On which variables, chosen because they relate both to response propensity and to the measures of interest, since weighting on a variable unrelated to the outcome costs efficiency and buys nothing. And to what source, named and dated, because a weighting target from an outdated or differently defined population introduces its own error. Then state the consequences: weighting reduces the effective base, and every base reported and every test run must use the effective base rather than the count of respondents. Weighting corrects known composition. It does not correct unknown selection, and it cannot convert a non-probability sample into a probability one. Where extreme weights would be required, cap them and report the capping (K4 §4.4).

**Step 9. Test incidence and feasibility, and pre-decide the shortfall response.**
Convert incidence into a contact requirement: how many approaches produce one qualifying, consenting, completing case in the hardest cell. Where incidence is unknown, say so and make the design conditional on a feasibility check placed first in the plan (01.05). Identify the two or three cells that will be hardest to fill, and decide now what happens if they cannot be: extend fieldwork (costs elapsed time and shifts the sample toward late responders), relax the quota (changes the population the cell describes), substitute an adjacent group (changes what the cell can be called), or report short with the base shown and the limitation attached. Record which is chosen. The value of this step is entirely in its timing: the same decision made under pressure in week three is reliably worse.

**Step 10. Document non-response and who is systematically missing.**
Response propensity is not random, and it usually correlates with the very things studies are about: engagement, satisfaction, availability, digital confidence, literacy, and interest in the topic. Three actions. Record the designed structure against the achieved structure, cell by cell, so the shortfalls are visible rather than absorbed into a total. For each shortfall, state who is under-represented and the likely direction of the effect on the key measures. And where a response rate is genuinely unknowable (as it is for most self-selected sources), say so rather than reporting a completion rate as though it were a response rate, which is a different quantity measuring a different thing (K4 §7). A correct result is a section a reader can use to discount the findings appropriately, which is the point.

## 8. Analytical framework

    Population        (who the findings must be about)
      → Frame         (who can actually be reached) ... gap = coverage error
        → Selection   (probability or not) ......... determines the licensed claim
          → Structure (strata, quotas, boosts) ..... determines analysis obligations
            → Size    (precision-driven or comparison-driven, derived from cells)
              → Achieved sample .................... what actually came back
                → Licensed claim .................. what may be said, and about whom

Read downward when designing. The single most important transition is population to frame, because that gap cannot be repaired by any later step and is invisible in the data.

Read upward when reporting. Given the achieved sample, what may be said? Walk back through the chain: the achieved structure against the design, the size against the claims, the selection mechanism against the language used, the frame gap against the population named in the headline. Most over-claiming in research reports is a failure to run this upward read, and it can be run in fifteen minutes.

## 9. Output format

A **Sampling Strategy**, in this order.

**1. Population definition.** Inclusion, exclusion, unit of analysis, time reference, geography. Plus the response unit where it differs from the analysis unit.

**2. Frame and coverage statement.**

| Frame used | How membership arises | Undercovered | Overcovered | Duplication | Likely direction of bias |

**3. The licensed claim.** One sentence, stating what these findings will describe and what they will not.

**4. Selection design.** Mechanism, and for probability designs the selection procedure and probabilities; for non-probability designs, the recruitment mechanism and the self-selection risk.

**5. Structure.**

| Element (stratum / quota / boost) | Variable and levels | Why it is here | What it obliges the analysis to do |

**6. Sizing.**

| Reporting cell | What will be claimed about it | Minimum base required | Basis (reporting threshold / difference to detect) | Planned base |

Followed by the derived total, the affordable total, and the explicit resolution where they differ.

**7. Sizing inputs and assumptions.** Every input used, labelled supplied, historical or assumed. Missing inputs are named as missing, with what each would change.

**8. Qualitative scope**, where applicable: the variation matrix, the planned range, the minimum, and the saturation review point.

**9. Weighting.** Whether, on what, to what source and date, expected efficiency loss, capping rule, and the instruction that effective bases are used in all reporting and testing.

**10. Feasibility and shortfall plan.** Incidence with its source or an unverified flag, contact requirements, hardest cells, and the pre-decided response to a shortfall in each.

**11. Non-response plan.** What will be recorded, and how under-representation will be reported.

**When the evidence is thin.** Say what is unknown rather than supplying it. Incidence with no source is `unverified`, not an estimate. A sample size with no inputs is not produced; the inputs required are listed instead. A qualitative number is a range with a review point, never a point estimate defended as sufficient. Where the design cannot support the analysis requested, that statement is the output (K4 §1).

## 10. Quality checks

Run before the design is committed. These sit on top of K4 §8.

1. Could two people apply the population definition to the same individual and agree?
2. Is the analysis unit distinguished from the response unit where they differ?
3. Does the coverage statement name who is missing and the direction of the likely bias, not merely that a gap exists?
4. Is the licensed claim written as one sentence, and does it match the language the report will use?
5. Is the word "representative" either removed or qualified by the frame it is representative of?
6. Does every quota, stratum and boost carry the analysis obligation it creates?
7. Was the total derived from the reporting cells rather than the cells checked against a total?
8. Does every crossed cut that will be requested at the debrief appear in the cell table?
9. Does any planned cell fall below the K3 §3.2 thresholds, and is the consequence stated rather than discovered?
10. Are all sizing inputs labelled supplied, historical or assumed, with none invented?
11. Is the smallest decision-relevant difference supplied by a person, or is the sizing marked conditional?
12. Are effective bases, not respondent counts, used wherever weighting applies?
13. Is incidence either sourced or flagged unverified, and is the design conditional where it is unverified?
14. Has the response to each hard cell's shortfall been decided in advance and recorded?
15. Is any margin of error attached to a non-probability sample anywhere in the document?

## 11. Common failure modes

| Failure | How to recognise it | How to prevent it |
|---|---|---|
| **The total that works and subgroups that do not** | A comfortable headline base and unreportable cells at analysis | Step 6: derive the total from the cells, including crossed cuts |
| **Frame accepted as population** | The report says "customers" and the frame was one channel's list | Step 2's four-part coverage statement, written before fieldwork |
| **Margin of error on a non-probability sample** | An interval quoted on an opt-in or quota sample | Step 3's licensed claim sentence, placed at the top of the section |
| **Quota mistaken for representativeness** | Composition matches population totals, so the sample is called representative | Quotas control composition, not selection. State it in the same sentence as the quota design |
| **Weighting as a repair** | A skewed non-probability sample weighted to national totals and then described as national | Weighting corrects known composition only. Say what remains uncorrected |
| **Qualitative size as a statistic** | A defended number of interviews with an implied sufficiency claim | Step 7: a range, a minimum, and a saturation review point, stated as a planning estimate |
| **Saturation asserted** | "Saturation was reached" with no account of what was assessed | Say what was assessed, when, and against what frame (07.01) |
| **Detectable rather than decision-relevant difference** | Sizing derived from what the budget could detect | Step 5: the input is the difference that would change an action |
| **Shortfall improvised** | Quotas relaxed mid-fieldwork with no record | Step 9: decide and document the shortfall response in advance |
| **Completion rate reported as response rate** | A percentage of starters presented as a response rate | They measure different things. Report what is knowable and say what is not |
| **AI: invented incidence or response rates** | Confident percentages for hard-to-reach audiences that nobody supplied | K4 §2.1. Unverified is a valid and required value |
| **AI: sample size from nothing** | A round number produced without variance, difference or precision inputs | Name the inputs required. Do not produce the number |
| **AI: symmetric cell design** | Equal bases across cells regardless of what each cell must support | Bases follow claims, and claims differ by cell |
| **AI: representativeness language** | "Nationally representative" attached to any composition-matched sample | Never used without naming the frame and the coverage gap (K3 §6) |

## 12. AI guardrails

Skill-specific. K4 applies in full and is not repeated here.

1. **Never state an incidence, response rate, cooperation rate or feasible sample as fact unless supplied.** Mark it unverified and make the design conditional on a check.
2. **Never produce a sample size without its inputs.** If variance, expected proportion, decision-relevant difference or required precision are missing, list them as required inputs rather than assuming them into a number.
3. **Never attach a margin of error, confidence interval or significance test interpretation to a non-probability sample as though it estimated population error.** Where such statistics are computed for internal comparison, say explicitly what they do not mean.
4. **Never describe a sample as representative without naming the frame it represents and the coverage gap it carries** (K3 §6).
5. **Never treat weighting as a correction for selection.** State what weighting fixes and what it leaves in place, in the same passage.
6. **Never present a qualitative number as sufficient in advance.** Adequacy is assessed during fieldwork, and a proposal number is a planning estimate labelled as one.
7. **Never derive subgroup bases from a total that was set elsewhere.** The direction of derivation is cells to total, and reversing it is the mechanism of the field's most common sizing failure.
8. **Never define a population to fit an available source.** Define who the findings must be about, then test feasibility against it openly, and report the gap where it does not close.
9. **Never let a shortfall be absorbed silently.** Achieved against designed structure is reported cell by cell, with the direction of the resulting bias stated.
10. **Never carry a sample design forward when the analysis plan adds a subgroup.** A new reported cut is a new base requirement, and it must be resolved before fieldwork rather than at analysis.

**Human review points** (see K5 §2 for the classes):
- **Researcher decision required** at Step 6 where the derived total exceeds what is affordable, and one of the four resolutions must be chosen. Class 2.7 with a Class 2.1 component, since which cuts matter is organisational knowledge.
- **Researcher decision required** on the smallest decision-relevant difference, which is a business judgement and not a statistical one. Class 2.1.
- **Researcher sign-off required** on the licensed-claim sentence, because it fixes what the study will be permitted to say about whom, and it will be quoted. Class 2.3.
- **Researcher review recommended** where a boost or a substituted group changes what a cell can be called.

## 13. Best-practice principles

1. **Define the population before you look at what is available.** The order is the whole discipline. Reversing it produces a study about whoever was reachable, and the redefinition never appears in the report.
2. **The frame gap is the error you cannot fix later.** Size, weighting and analysis all operate inside the frame. Nobody outside it is recoverable by any technique.
3. **Sample size follows the smallest thing you must report on.** No one has ever been disappointed by a total-sample base. They are routinely disappointed by a subgroup of 40.
4. **Ask for the crossed cuts now.** People request region at the design stage and region by segment at the debrief. Getting the crossed cuts on the table early is the cheapest sizing insight available.
5. **Say in one sentence what the sample licenses.** If that sentence is uncomfortable to write, the design is over-claiming, and the discomfort is the finding.
6. **Quotas control who is in the sample, not why they answered.** Composition matching is worth having and is not representativeness. Treating the two as equivalent is the most widespread misunderstanding in commercial sampling.
7. **Precision is bought with the square root, so buy only what you will use.** A tighter interval on a number that will be reported as "about a third" is expenditure on nothing.
8. **Decide the shortfall response before fieldwork.** Every option costs something, and choosing calmly in week zero produces a better choice than choosing under pressure in week three.
9. **Qualitative adequacy is a judgement made during fieldwork, not a number promised beforehand.** Say so in the proposal, give a range and a review point, and defend the scope rather than the count.
10. **Weighting is a composition correction with an efficiency cost.** It buys comparability on the weighting variables and costs effective base everywhere. Both belong in the design note.
11. **Non-response is a finding, not an administrative detail.** Who did not answer, and how they differ, frequently tells you more about the population than the achieved sample does.
12. **Write the limitation while you are designing, not while you are reporting.** The sampling limitations of a finished study are already fixed at the design stage; drafting them early is simply reading the design honestly.

## 14. Worked example

Fictional scenario, commercial business-to-business. All figures are illustrative and belong to the scenario.

    INPUT

    A payments provider briefs a study: "We need a nationally representative
    survey of 500 small businesses, with results by sector and by size band,
    so we can see where the demand for the new service is strongest."

**Process.**

*Step 1, population.* "Small businesses" was operationalised as organisations with one to forty-nine employees, trading for at least twelve months, where the respondent is the person who decides or influences payment provider selection. The analysis unit is the organisation, the response unit is one informant, and that distinction was written into the design because a single informant's account of an organisation's decision is weaker evidence than it looks.

*Step 2, frame.* The only practical route was a business-audience access source. Undercovered: sole traders without a registered business presence, and organisations whose payment decisions are made by an external bookkeeper. Overcovered: organisations above the size threshold, screened out. Differential: businesses that had recently changed provider were plausibly more likely to participate in a study about payments, which would bias any measure of switching intent upward. That last observation changed how the switching measure would be reported.

*Step 3, licensed claim.* Non-probability. The claim sentence was drafted immediately: *these findings describe 500 respondents recruited from a business audience source, quota-controlled on sector and size band; they are not a probability sample of the small business population and no margin of error applies.* The word "representative" was removed from the brief's framing and replaced.

*Step 6, the sizing failure and the judgement call.* The brief wanted results by sector and by size band. Six sectors by three size bands is eighteen cells; at 500 that averages 28 per cell, and the largest sectors would take most of them, leaving several cells in single figures. Crossed reporting was therefore impossible at any base. Working upward from the cells, reporting all eighteen at even a directional standard required well over 1,500 interviews, which was several times the available budget.

Four resolutions were put to the client. Report sector and size separately but never crossed, which fits 500 comfortably and answers a narrower question than the brief asked. Collapse six sectors into three defensible groups, decided in advance on the basis of payment behaviour rather than convenience, giving nine cells. Boost the two sectors the commercial decision actually turns on and weight back for total reporting, accepting that boosted cases arrive by a different route. Or report the crossed cuts qualitatively as counts with no percentages.

> **Researcher decision required.** The four resolutions differ in what the
> study can say, not in its cost. Which sectors matter commercially, and
> whether a crossed sector-by-size reading is genuinely needed for the
> investment decision, is organisational knowledge this analysis does not
> contain (K5 §2.1). What turns on it: whether the study answers the
> question as briefed or a narrower one, and whether that is acceptable.

The client chose collapsing to three sector groups plus a boost on the single sector under active consideration. Boosted cases were flagged in the data so that any total-level figure could be produced both weighted and unweighted, and the difference reported.

*Step 9, feasibility.* Incidence of the decision-maker role within the target size band was unverified. The design was made conditional on a screening test in the first week, with a pre-decided response: if incidence fell below the assumed level, the smallest size band would be reported at a directional standard with its base shown rather than the fieldwork being extended past the decision date.

    OUTPUT

    A Sampling Strategy with: an operational population definition distinguishing
    analysis and response units; a four-part coverage statement identifying a
    likely upward bias on switching intent; a one-sentence licensed claim
    replacing the word representative; a quota design with its analysis
    obligations stated; a cell table showing the derived total against the
    affordable total and the chosen resolution; a boost with its weighting and
    flagging rules; incidence marked unverified with a week-one check; and a
    pre-decided shortfall response for the hardest cell.

## 15. Advanced usage

**Designing for the analysis that will be asked for, not the one that was briefed.** Ask what cuts the audience will request when they see the results. Region by product, tenure by satisfaction, and any cut that maps to an internal team structure are near-certain requests. Sizing for the two most likely of these costs little at design and prevents the study's most predictable disappointment.

**Sequential and adaptive sampling.** Where incidence is genuinely unknown, a small first tranche that measures incidence and variance before committing the rest is usually cheaper than a full commitment plus a rescue. Set the decision rule for the second tranche before the first is fielded, so the adaptation is a plan rather than a reaction.

**Small populations.** Where the population is a few hundred (a member organisation, a professional body, a set of accounts), a census is often cheaper than a sample and removes sampling error entirely, leaving non-response as the only sampling-related concern. The finite population correction also matters here in a way it does not for large populations, and it means the required sample is smaller than intuition suggests.

**Hard-to-reach and low-incidence audiences.** Referral chains and other network-based recruitment reach populations no list contains, at the cost of a selection mechanism driven by the social structure of the population itself. What they license is description of a connected subset, not estimation. Where such a design is used, the recruitment chain is documented (seeds, waves, referral counts) so a reader can see the structure of what was reached.

**Sampling for comparison over time.** Where a study will be repeated, the sample specification is a constraint from wave one onward, and changing the source, the quota structure or the channel breaks comparability more thoroughly than changing question wording does. Any planned change needs a parallel-run bridge or an accepted series restart (05.05).

**Reading an achieved sample after the fact.** Where fieldwork has closed and the structure differs from the design, run the framework upward: achieved structure against designed, effective bases after weighting, cells that fell below threshold, and the direction of the shortfall's likely bias. The output is a revised statement of what may be claimed, produced before analysis rather than after the report is challenged.

## 16. Skill chain

**Recommended previous skills**
- **01.04 Research Method Selection.** Hands over the design and the sample structure it implies, including the reporting unit that drives scale.
- **01.02 Business Problem to Research Question.** Hands over the population definition and the subgroups the decision requires comparing.

**Recommended next skills**
- **01.07 Analysis Plan Development.** Receives the achievable bases and specifies the comparisons, tests and thresholds they can support. Where the analysis plan adds a subgroup, the sample design must be re-run.
- **Category 02 Instrument Design.** Receives the screening and quota requirements, which become the screener.

**Runs well alongside**
- **01.05 Research Plan Development**, which schedules recruitment against the hardest quota cell.
- **05.01 Descriptive Analysis** and **05.02 Statistical Testing**, which inherit the effective bases and the licensed claim.
- **13.05 Research Ethics and Consent Design**, wherever the frame contains personal data or the audience includes vulnerable participants.

---
A Yazi Supplied Skill and resource.