Confidence Intervals and What a Sample Can Support

The confidence level attached to an interval describes the procedure that generated it, not the particular interval in hand. Imagine repeating the same experiment 100 times, each time drawing a fresh sample of the same size and computing a 95% interval from each sample. On average, 95 of those 100 intervals will contain the true parameter and 5 will not. You do not know, for any given interval, whether it is one of the 95 or one of the 5.

This is the correct statement. Three statements that look similar but are wrong: (1) there is a 95% probability that the true value is in this interval; (2) 95% of the data falls in this interval; (3) if the experiment is repeated, the result will fall in this interval 95% of the time. Each of these confuses the parameter, the data distribution, or the interval procedure with the others.

Why does the distinction matter practically? Because in legal settings, the phrase a 95% probability that the true frequency lies between X and Y implies the court can treat X and Y as bounds on a probability. If the interval is 1-in-1000 to 1-in-500, a defender might argue the court should use the larger value (weaker evidence) because there is a 5% chance even that is too small. The correct framing instead reports the estimate and interval as a description of estimation precision, not as a probabilistic bracket on the true value.

The standard interval for a population mean uses the sample mean, the standard error, and a critical value from the t or z distribution. The formula is: interval equals sample mean plus or minus (critical value times standard error). The standard error equals the sample standard deviation divided by the square root of n. For large samples (n above about 30) the z critical value is used; for small samples the t critical value with n minus 1 degrees of freedom is used.

For the normal distribution at 95% confidence, the critical value is 1.96 (commonly rounded to 2 in approximate calculations). At 99% confidence it is 2.576. At 90% confidence it is 1.645. Choosing a higher confidence level produces a wider interval for the same data, because you need more room to achieve higher coverage probability.

For proportions, such as the frequency of a feature in a reference database, the standard interval uses the sample proportion p-hat. The standard error is the square root of p-hat times (1 minus p-hat) divided by n. This approximation works well when both n times p-hat and n times (1 minus p-hat) are greater than 10. For rare features where the expected count is small, exact binomial intervals such as the Clopper-Pearson interval are more appropriate and give wider, more conservative bounds.

Interval width equals twice the margin of error, which equals twice the critical value times the standard error. Because the standard error is the population standard deviation divided by the square root of n, doubling n reduces the standard error by a factor of 1.41 and reduces the interval width by the same factor. To halve the interval width, n must be quadrupled. This inverse square root relationship is why large forensic reference databases have disproportionate value: each additional individual narrows the interval, but the gains diminish as the database grows.

Population variance has the opposite effect: higher variance produces a wider standard error and a wider interval for the same n. In forensic soil analysis, for example, elemental concentrations may vary greatly between geographic regions. A database drawn from a single narrow region will show lower variance and tighter intervals than a database drawn from a whole country. Both can be valid, but they answer different questions: the regional database estimates frequency within a region; the national database estimates frequency across a country.

The practical implication for forensic scientists: when a reference database is small, the confidence interval around a rare feature frequency may be very wide. A DNA profile frequency estimated from 500 individuals in a specific population group could have a 95% interval spanning an order of magnitude. Presenting only the point estimate and omitting the interval creates an appearance of precision the data cannot support. Courts in multiple jurisdictions have begun requiring disclosure of the database size and the interval alongside the point estimate, as discussed in Role of Statistics in Evidence Evaluation.

A two-sided interval covers both tails of the sampling distribution and answers: what range of parameter values is consistent with this sample? A one-sided interval covers only one tail and answers: what is the highest (or lowest) plausible value of the parameter? Forensic applications often call for one-sided intervals because the inferential question is one-directional.

In DNA frequency estimation, the question relevant to defence is: what is the largest plausible value of the profile frequency? A large frequency means the evidence is weaker. The NRC II report in the United States and subsequent guidance from ENFSI and other bodies recommended reporting an upper confidence bound on the profile frequency, or using a conservative point estimate derived from the upper bound of the frequency of each allele, rather than a central estimate. This gives defendants the benefit of database uncertainty.

The corresponding one-sided 95% upper bound uses the critical value 1.645 rather than 1.96, because only one tail is being controlled. This makes the one-sided 95% upper bound narrower than the two-sided 95% upper limit for the same data, which is a source of confusion when comparing reported values across studies that use different conventions.

The intuitive interpretation people want for a confidence interval (there is a 95% probability the true value is in here) is actually the correct interpretation of a Bayesian credible interval. A 95% credible interval is derived from the posterior distribution of the parameter after combining the prior distribution with the likelihood from the data. It directly answers: given what I knew before and what the data show, where does 95% of the posterior probability for the parameter lie?

The price of the Bayesian interpretation is a prior. For a DNA profile frequency, the prior would encode beliefs about how common the profile is before seeing the database data. If the prior is flat (no prior knowledge), the credible interval and the confidence interval are often numerically similar. Where they diverge is when the prior is informative, either concentrating probability on small or large frequencies. In casework, choosing and justifying a prior is a substantive decision that affects the conclusion and must be disclosed and defended.

Misidentifying which framework is being used leads to specific errors. Treating a credible interval as a confidence interval understates the role of the prior. Treating a confidence interval as a credible interval makes an unwarranted probability statement about the parameter. Both errors appear in peer review and in court, and both undermine the credibility of statistical evidence.

The challenge in court is to communicate what the interval means without requiring the audience to understand sampling theory. Several standard phrasings have been developed through guidelines from bodies including the UK Forensic Science Regulator, the Scientific Working Group for DNA Analysis Methods (US), and the European Network of Forensic Science Institutes. A recommended form is: the estimated frequency is X, based on a database of N individuals; the 95% confidence interval for this estimate is [lower bound] to [upper bound]; this means the procedure used to compute this interval will bracket the true frequency in 95 out of 100 comparable studies.

Under the Bharatiya Sakshya Adhiniyam 2023 (which replaced the Indian Evidence Act 1872), expert opinions on scientific matters are admissible where the expert has relevant qualifications and the underlying method is disclosed. Disclosing a confidence interval and the database size is part of disclosing the method. Under the US Daubert standard, the error rate and known standards of operation of a technique are relevant to admissibility. An interval width that is not disclosed is an undisclosed error rate. UK Crown Court guidance similarly requires that statistical evidence in forensic reports be accompanied by the assumptions and limitations of the analysis.

Two forms of honest communication are useful in practice. First, state the result and its interval together: the estimated match probability is 1 in 4000, with a 95% confidence interval of 1 in 8000 to 1 in 2200. Second, when a narrow conclusion is needed (for example, to say the frequency is no higher than 1 in 1000), use the upper bound of a one-sided confidence interval explicitly rather than the point estimate. Both approaches prevent misuse of an estimate as if it were a known fact rather than a sample-based approximation. The connection between these principles and the broader practice of evaluative reporting is treated in Numbers in Forensic Conclusions.