Sampling Bias
Definition
A systematic distortion in an estimate caused by a selection process that does not give all units a known probability of inclusion. Unlike sampling error, bias does not decrease as sample size increases and cannot be quantified without an independent reference.
- Cause
- Non-random or unequal selection probability
- Behaviour vs sample size
- Does not shrink as sample size grows
- Quantifiable?
- Not without an independent reference
- Contrast term
- Sampling error
Common questions
Why can't a larger sample fix sampling bias the way it fixes sampling error?+
Bias comes from a flawed selection process that systematically favours certain units, so collecting more data from that same flawed process just produces a larger, still-skewed dataset rather than one that converges on the truth.
How would an analyst even know sampling bias is present if it can't be calculated from the sample?+
Bias is typically detected by comparing the sample against an independent, trusted reference, such as known population characteristics or a differently collected dataset, since the biased sample alone cannot reveal its own systematic skew.
What is a practical example of sampling bias in forensic or investigative data collection?+
Surveying only cases that reached trial, and ignoring those resolved by plea or dismissed pretrial, would systematically exclude certain outcome types, producing conclusions that do not generalise to the full caseload.
Related terms
- Cluster Sampling
- A design in which the population is divided into naturally occurring groups called clusters, a random sample of clusters is selected, and...
- Sampling Error
- The difference between a sample statistic and the true population value, arising from the randomness of selection. It can be estimated from...
- Sampling Frame
- The list or description of all units from which the sample can be drawn. Any unit not on the frame cannot be...
- Simple Random Sampling
- A design in which every possible subset of a given size has an equal probability of being selected. It requires a complete...
- Stratified Sampling
- A sampling design that divides the population into homogeneous subgroups (strata) and samples each stratum separately. Allows the auditor to apply higher...