Probabilistic interpretations of beyond reasonable doubt

Probabilistic interpretations of beyond reasonable doubt

Swedish legal writing sometimes glosses “beyond reasonable doubt” as 98 percent certainty. That number is not a statutory threshold, a holding that courts use a calibrated probability model, or an observed accuracy rate. Kvalitetssäkring av bevisprövningen i brottmål presents it as one attempted translation of a qualitative legal standard and immediately explains why it cannot be empirically verified.

What one in fifty would mean

Suppose a decision-maker could validly assign each convicted defendant a 0.98 probability of guilt. If those assessments were calibrated, then among a large class of cases assigned 0.98, about two percent would be factually innocent: an expected one person in fifty.

The precise correction is that one in fifty would be innocent, not that one identifiable case would be “not certain at all.” Every case in the class carries epistemic uncertainty; the underlying events are not 98 percent true. If every conviction were merely required to exceed 0.98, the expected error rate could be below two percent because some cases would be assessed at higher probabilities.

For case probabilities p_i, the expected innocent count is the sum of one minus p_i across the convicted cases.

This arithmetic is valid only when the probabilities themselves have a defensible meaning.

Why the percentage is pseudo-precise in practice

A defensible posterior probability requires at least a defined reference class, base-rate assumptions, likelihood estimates for the evidence, and calibration against independently known outcomes. A trial normally supplies none of these in measured form. The court evaluates a singular historical event from a selected and incomplete record.

The percentage therefore conflates distinct claims:

Claim What would be needed to support it
This defendant is 98 percent likely to be guilty A valid probabilistic model for this evidence and its alternatives
Cases assessed at 98 percent are correct 98 percent of the time Independent ground truth and calibration across comparable cases
At least 98 percent of convictions are factually correct A representative audit that can observe both detected and undetected errors
The legal standard should tolerate a two-percent residual risk A normative decision about the relative costs of false conviction and false acquittal

The first three are empirical claims. The fourth is a moral and legal policy choice. Quoting one number can make the policy choice look as if it had already been validated by the empirical claims.

The comparison with p less than 0.05

The conventional scientific cutoff p < 0.05 is a useful comparison precisely because it is not the same quantity. A p-value is calculated under a specified statistical model. Roughly, it asks how incompatible the observed or more extreme data would be with that model. It is not the probability that the null hypothesis is true and does not make a positive hypothesis “95 percent certain.”

When a test and decision rule are specified in advance and their assumptions hold, an alpha level of 0.05 can control a long-run Type I error rate. That is a real operating property, although it does not by itself control the proportion of published positive claims that are false. Researcher choices, multiple testing, selective publication, weak power, and violated assumptions can make actual scientific practice substantially less reliable than the nominal level suggests.

The ASA statement on p-values therefore says that a p-value does not measure whether a hypothesis is true and that conclusions should not depend only on crossing a numerical threshold. The association’s 2019 special-issue editorial went further in urging science to move beyond the ritual of p < 0.05.

The legal 98-percent gloss has less formal content. There is no specified null hypothesis, sampling process, likelihood model, precommitted calculation, or calibration set. The resemblance lies in the institutional ritual: a continuous and difficult uncertainty is converted into a binary authorization by a conventional-looking number. The difference is that the p-value at least has a defined mathematical meaning under stated assumptions; the courtroom percentage is usually assigned by no reproducible operation.

Systemic error defeats the intuitive model

The one-in-fifty intuition also imagines independent, random mistakes. Criminal-process failures are often correlated. A leading interview can shape later expert analysis; an early prosecution hypothesis can shape the disclosed file; a remand decision can anchor the eventual fact-finder; and appellate review can inherit the same record. Such mechanisms can create clusters of confidently wrong judgments rather than a stable background error rate.

This is why False accusations as an epistemic risk and Rättssäkerhet in Swedish criminal cases focus on error-generating mechanisms and auditability. Without calibration, “98 percent” is not reassurance. It is an unsupported numerical costume for the very judgment whose reliability is in question.

Sources

  1. 2009-lambertz-kvalitetssakring-bevisprovning.pdf
  2. 2006-jk-felaktigt-domda.pdf
  3. 2009-jk-rattssakerheten-i-brottmal.pdf
  4. 2016-asa-statement-p-values.pdf