Field Notes

source

Operationalizing content moderation “accuracy” in the Digital Services Act

StatusStableUpdatedStands on1 sourceSupportsnotes

Johnny Tian-Zheng Wei, Frederike Zufall, and Robin Jia examine what the Digital Services Act’s transparency obligation actually requires platforms to measure, and conclude that the term it uses cannot bear the weight placed on it. The preserved abstract page (raw response) was saved 2026-07-28.

The obligation

DSA Article 15(1)(e) requires providers to report any use of automated means for content moderation, including “indicators of the accuracy and the possible rate of error of the automated means used in fulfilling those purposes.”

Why the word does not work

The authors identify “accuracy” as an open-textured term the Act leaves unspecified: neither the ground truth nor the dataset to measure against is named, so providers may each choose their own test data and metric and report whichever is most favourable. Correction channel generalises the underlying problem: an institution whose own outputs supply the only available labels cannot calibrate its accuracy from those outputs. The paper’s central technical objection is that literal accuracy — correct predictions over total predictions — is the wrong statistic under class imbalance, which is the ordinary condition of content moderation because violating content is a small fraction of all content.

Their worked example makes the size of the problem concrete. On a corpus of about 1.7 million comments containing roughly 100,000 toxic ones, a filter achieves

Metric Value
Accuracy ~95%
Precision 58%
Recall 33%

An accuracy figure near 95 percent is compatible with the system missing two-thirds of the target content and being wrong about four in ten of its own flags.

What they propose instead

The authors argue that the Act’s purpose, and the balancing of Charter rights it embodies, justify interpreting “accuracy” as precision and recall, which stay informative under imbalance and whose tradeoff expresses the balance the Act intends. They give an estimator for recall using stratified sampling, so that reporting it need not impose disproportionate annotation costs, and demonstrate it on a subset of Reddit.

Bearing on this wiki

The class-imbalance argument is the general form of a problem Automated CSAM detection has in an extreme degree, since target prevalence there is far below the toxicity rate in the paper’s example. It is the reason What is the production false-positive rate of automated CSAM detection is not answerable from mandated disclosures: the one metric the DSA compels is the one that carries least information at CSAM prevalence.

Built on 1 source (1 external).

Working out connections…