Field Notes

source

How we detect, remove and report child sexual abuse material

StatusStableUpdatedStands on1 sourceSupportsnotes

Google’s official explainer, published 28 October 2022 two months after the NYT false-positive story, describes the company’s CSAM pipeline in its own terms. The preserved post (raw response) states the mechanism this wiki’s concept note adopts: hash matching against databases of known material (with hashes shared by NCMEC and the IWF, each independently re-verified by Google before use), plus machine-learning classifiers that flag previously unseen imagery, with specialist human review of every newly flagged item before it is reported or acted on. Google claims its systems are designed to recognize benign imagery such as a child in a bathtub and that the combination keeps its false-positive rate “incredibly low” — a figure it does not publish.

The post discloses first-half-2022 scale: over one million reports to NCMEC and approximately 270,000 account suspensions. It also commits, in response to the controversy, to more detailed suspension notices and to an appeals process that accepts context and documentation “from relevant independent professionals or law enforcement agencies” — the remediation gap documented in CSAM scanning externalizes error costs.

As the regulated party’s self-description it is primary evidence for how the pipeline is supposed to work, but its accuracy claims are unverifiable assertions; treat “incredibly low” as a claim, not a measurement, as tracked in What is the production false-positive rate of automated CSAM detection.

Built on 1 source (1 external).

Working out connections…