Sampling Strategies and Representative Data

Simple random sampling (SRS) is the design in which every unit in the population has an equal probability of being selected, and every possible sample of the specified size is equally likely. This equality of selection probability is what justifies standard formulas for means, proportions, and their standard errors. If the selection probabilities are not equal, those formulas give biased estimates unless appropriate weights are applied.

Implementing SRS requires a sampling frame: a complete enumeration of all units in the population. In laboratory settings, the frame might be a batch of tablets to be tested for drug content, a set of glass fragments from a crime scene, or a collection of soil samples from a suspect location. A random number generator assigns each unit a number, and units are selected in number order up to the required sample size. The key requirement is that every unit on the frame must have had a genuine chance of selection.

SRS is rarely optimal when the population has known substructure. If a drug seizure consists of tablets from three different presses with different fill distributions, a single SRS over all tablets may happen to draw mostly from one press. This produces correct point estimates on average, but with higher variance than a design that exploits the known structure. Stratified sampling addresses this.

Stratified sampling divides the population into strata before sampling, then draws an independent sample from each stratum. The within-stratum samples are combined using weights proportional to each stratum's share of the population to produce overall estimates. Because the stratification controls for between-stratum variation, the resulting estimates have lower variance than an SRS of the same total size, provided the strata genuinely differ from one another.

In proportional allocation, the sample from each stratum is sized in proportion to the stratum's population share. In optimal (Neyman) allocation, strata with higher variability receive larger samples, which is more efficient but requires prior knowledge of within-stratum variance. A forensic application: testing a large drug seizure where tablets from three batches are stored separately. Stratifying by batch and sampling proportionally ensures each batch is represented, and any batch-to-batch differences in purity can be estimated directly.

Stratified sampling is also the standard approach for constructing forensic reference databases that must represent multiple demographic groups. A database stratified by declared ethnicity and sampled from multiple geographic sources will produce frequency estimates that can be applied stratum-specifically, avoiding the error of treating a heterogeneous population as uniform. The CODIS STR frequency tables used in US courts, and the analogous databases used in UK and European casework, are stratified by population group for precisely this reason.

Cluster sampling is used when a complete individual-level sampling frame does not exist or is impractical to construct, but a list of naturally occurring groups does. A random sample of groups (clusters) is selected, and units within selected clusters are then examined. In single-stage cluster sampling, all units in selected clusters are examined. In two-stage cluster sampling, a random subsample within each selected cluster is drawn.

The chief disadvantage of cluster sampling is design effect: because units within a cluster tend to resemble one another more than units drawn from different clusters, the effective sample size is smaller than the nominal count of observations. Standard SRS formulas applied to cluster sample data underestimate variance and produce confidence intervals that are too narrow. Correct analysis requires accounting for the intraclass correlation within clusters.

Forensic examples of cluster-style sampling arise when examining documents from a seized archive, where folders are the clusters; when testing a large quantity of individually wrapped drug packages, where boxes are the clusters; or when studying populations across jurisdictions, where courts or police districts are the natural grouping. In each case the analyst must decide whether to sample and analyse at the cluster level, the unit level, or both, and must report which level was used when presenting estimates.

Sampling error arises from the randomness of selection: any particular sample will differ from the population by chance. This variation is predictable: its magnitude depends on the population's variability and the sample size, and it can be estimated from the data itself. Confidence intervals are the standard way to communicate sampling error; they capture the plausible range of population values consistent with the observed sample.

Sampling bias is fundamentally different. It arises when the selection process systematically over-represents or under-represents certain units. The classic example is a voluntary response survey: people who respond tend to hold stronger opinions than those who do not, so the results are biased toward extreme positions regardless of how many people respond. Increasing the sample size from a biased source makes the estimate more precise but not more accurate: it narrows the confidence interval around the wrong value.

In forensic science, bias enters at several points. Cases selected for study are not random: they are cases that were investigated, prosecuted, and tested. Exhibits selected for laboratory analysis are chosen by investigators, not randomly from all possible physical traces at a scene. Reference populations for frequency estimation may be drawn from convenience samples rather than systematic probability samples. Each filter introduces a potential discrepancy between the sample and the population to which inferences are meant to apply.

The distinction matters for evaluating forensic statistics in court. A defence expert who demonstrates that the reference database used to estimate a match probability was drawn from a non-representative sample has identified a structural problem that the prosecution cannot remedy by pointing to the database's size. The Judicial Committee of the Privy Council addressed this type of argument in R v Doheny and Adams [1997], noting that the value of a statistical comparison depends on the representativeness of the reference data, not merely its volume.

Forensic sampling operates under constraints that textbook survey sampling does not face. The most fundamental is that many forensic exhibits are finite and consumed by analysis. A 10-gram drug powder can yield at most a few analytical samples. A small bloodstain on a garment can be consumed entirely by DNA extraction. The analyst cannot take a larger sample if the first results are inconclusive; the sampling plan must be decided before any material is committed to analysis.

A second constraint is that the relevant population is often undefined. When characterising a drug seizure, the population of inference might be the specific batch, the supplier's production run, or all material from the same source. These are different populations, and the appropriate sampling strategy, and the interpretation of the results, differs for each. The analyst must state the population of inference before selecting a sampling design.

A third constraint is non-random case selection. The cases that reach a forensic laboratory are filtered by investigative decisions, resource allocation, and legal threshold requirements. This means that laboratory validation studies conducted on casework samples inherit the biases of that selection process. A study of measurement error based on operational casework cases may not generalise to the full range of material that could theoretically be submitted, because only a subset of possible inputs ever arrive.

Jurisdictions differ in their formal requirements. In England and Wales, the Forensic Science Regulator's Codes of Practice require accredited providers to document sampling protocols under ISO/IEC 17025. In the United States, OSAC (the Organization of Scientific Area Committees for Forensic Science) publishes sampling guidance for specific forensic disciplines. In India, the Bureau of Indian Standards provides general sampling standards, and forensic laboratories seeking NABL accreditation must document sampling procedures as part of their quality system. The common thread across all frameworks is that the sampling protocol must be decided and recorded before analysis, not reconstructed afterwards.

Forensic reference databases, whether DNA allele frequency tables, glass refractive index distributions, or soil composition ranges, are samples from some underlying population. Their value as inferential tools depends on whether that population matches the population of casework exhibits and potential contributors. A glass database compiled from European float glass manufacturers will not give accurate frequency estimates for glass from South Asian manufacturers if production processes differ.

Assessing representativeness requires knowing how the database was built. Key questions are: What was the sampling frame? Were units selected by a probability mechanism or by convenience? Were certain subgroups systematically excluded? How old is the database, and could the target population have changed since collection? For DNA databases, the additional question is whether the contributors gave informed consent under applicable law: the EU General Data Protection Regulation, India's Digital Personal Data Protection Act 2023, and similar instruments impose constraints on whose data may be retained and used for frequency estimation.

The practical consequence of a non-representative database is an inaccurate likelihood ratio or random match probability. If the database under-represents the genetic profile common in a defendant's ancestry group, the reported rarity of a DNA profile will be overstated, inflating the weight of the evidence against the defendant. Several appeal decisions in multiple jurisdictions have turned on this point, including cases reviewed by the UK Court of Appeal and the US National Academy of Sciences report Identifying the Culprit (2009), which called for systematic population sampling in database construction.