Field Notes

synthesis

Reliability of mobile forensic extraction as evidence

StatusWorkingUpdatedStands on9 sourcesSupportsnotes

A claim that evidence was “found on the suspect’s phone” by a tool such as Cellebrite’s UFED bundles three different propositions with three different failure modes: that the content is what it appears to be, that it was present on the device, and that the suspect put it there by a culpable act. How much trust a secret, non-replicable forensic method deserves is not one question but three, and the answers differ enough that courts and defenders should not let the tool’s brand collapse them into one.

The Swedish frame: admission without an adversarial gate

Swedish criminal procedure evaluates evidence freely (RB 35:1) and has no admissibility doctrine that could exclude an unreliable forensic method. The entire weight of the reliability question therefore lands on the court’s assessment of probative value, which can only be as good as the defence’s ability to probe the method. Two features of current practice hollow that probe out:

  • Vendor-enforced secrecy. Cellebrite trains its law-enforcement customers to keep extraction techniques “hush hush” and to classify them as law-enforcement sensitive, which is precisely the information an adversarial test would need: how the tool works, what it is known to get wrong, and what the examiner selected and excluded.1
  • Judicial non-engagement. In the EncroChat cases, the collection method was shielded by French defence secrecy, HD declined to review the reliability question, and Swedish courts admitted the material anyway — establishing that conviction-grade digital evidence can rest on a method no Swedish institution has examined.2 How do Swedish courts handle EncroChat reliability challenges tracks the open residue.

The result is a structural answer to how rättssäker a secret forensic method is in Sweden, in the procedural sense Rättssäkerhet carries in Swedish administrative and public debate: formally admissible and practically hard to challenge, which is weaker due process than a system with a reliability gate would provide, not stronger. Rättssäkerhet in Swedish criminal cases develops the framework; Anonyma vittnen is the adjacent problem of unexaminable testimony.

What validation exists, and what it certifies

Sweden does have a quality layer. NFC’s laboratories are accredited by Swedac under ISO/IEC 17025 for many forensic methods, with documented standard procedures, proficiency testing, and examiner competence requirements, and the prosecution service’s guidance (RåV 2022:31) describes how forensic examinations should be commissioned and documented.3 4 This is genuine assurance about the laboratory process: that a competent examiner followed a documented method.

It is not assurance about the tool’s claims. Accreditation validates that a method was followed, not that the method’s output is true. The proprietary internals of UFED — what its parsers do with a malformed database, what its exploits change on the device, what its report generator includes and omits — remain a trade secret no accreditation has opened. Tool-validation studies in the field test acquisition and parsing against prepared test data; they do not test what the tool does when the device being scanned is itself hostile. That gap is not theoretical.

The tool-integrity problem

In 2021, Moxie Marlinspike demonstrated that specially formatted files placed in any app on a scanned device could execute arbitrary code on the Cellebrite machine performing the extraction — modifying not only the current report but all previous and future reports, with no detectable checksum or timestamp changes.5 The same analysis found UFED bundling FFmpeg libraries from 2012 with over a hundred unpatched vulnerabilities. A forensic tool that parses untrusted input without mitigations, from a company that hoards vulnerabilities and has itself been breached repeatedly (Cellebrite § How hoarding causes harm), cannot be treated as a neutral witness. The defence question “could the extraction pipeline itself have been compromised or manipulated” has a documented affirmative answer.

The provenance problem: presence is not an act

Even a perfectly functioning tool only attests that data was present in storage. The inference from presence to a culpable act crosses terrain where documented failures cluster:

  • Malware-planted evidence. In the Bhima Koregaon case, Arsenal Consulting’s court-filed forensic reports showed that activist Rona Wilson’s laptop had been controlled by NetWire malware for 22 months, that the incriminating documents were delivered to a hidden folder by the malware, and that none of the ten key documents was ever opened on the machine — while the government’s own laboratory reported finding no malware at all.6 Two laboratories, opposite conclusions, on the most consequential question in the case. That is what the absence of a shared, examinable method looks like.
  • Automated placement. Messaging auto-downloads, thumbnail caches, sync artifacts, and backup restores place content on devices without any viewing, let alone intent.
  • Ambiguous artifacts. Under Mobile-device extraction and evidentiary selection, a timestamp can record synchronization, caching, or application behavior rather than a human act; precision is not validity.

Working the example: two very different CSAM findings

The distinction the question turns on is between self-verifying content and tool-attested presence.

Extraction finds CSAM depicting the suspect and the accuser, verified as neither AI-generated nor manipulated. Here the content carries its own proof independent of the pipeline: the depicted persons are identifiable and the material’s authenticity can be confirmed by other means. The tool’s role reduces to establishing that the material was on this device — a provenance claim that still requires an intact chain of custody and rules out planted placement, but whose evidential core does not depend on trusting Cellebrite’s software. This finding is strong no matter the extraction tool.

Extraction finds generic, publicly circulating CSAM. The finding’s entire weight rests on pipeline integrity and on the presence-to-act inference: that the files were really on the device (tool parsing, carving, and report fidelity), that they were not planted (malware, shared access, auto-download), and that the suspect knowingly acquired them (timestamps, viewer artifacts, surrounding activity). Each link is contestable, and the Marlinspike result means “the UFED itself was compromised or manipulated” is a legitimate defence hypothesis rather than a conspiracy theory. Absent independent re-examination, this finding should carry limited weight on its own.

The general rule: trust tracks what would have to be true for the evidence to exist if the suspect were innocent. For self-verifying content, almost nothing. For tool-attested presence of generic material, a working parser, an uncompromised tool, no third-party placement, and a culpable acquisition — each individually possible to defeat, and none examinable when the method is secret.

What would make an extraction examinable

The controls split by threat model, and conflating them is how secret tools acquire unearned trust.

Controls against human tampering — chain of custody, examiner testimony, accreditation — answer who held the device and whether documented procedure was followed. They detect swapped exhibits and sloppy handling. They cannot detect a silent software compromise: Marlinspike’s manipulation was undetectable to the operator, so an examiner testifying that the tool was not compromised is testifying to something they cannot know. These controls are necessary and radically insufficient.

Controls against software-level failure must verify without trusting the first tool:

  • Pre-exploitation imaging. The desktop hygiene of imaging before analysis mostly does not exist on mobile: on a locked modern phone, exploitation precedes any usable image, so the first tool to touch the device is trusted with everything it later reports. A second, independent extraction of the same live device does not cure this — it reads back whatever the first tool left behind, including anything it planted on-device.
  • Consistency-web analysis. On-device planting has to fake filesystem journals, application database internals, log cross-references, and backup manifests into one mutually consistent fabric — categorically harder than altering a report, and the genuine defence-expert task.
  • Device-external corroboration. Cloud-side account records, carrier logs, backups predating seizure, and other devices bypass the contaminated device entirely. For tool-attested claims, this is the strongest available check and the one a court should demand.
  • Reproducible tooling. Open methods and documented versions let an independent party rerun the analysis, which is exactly what trade secrecy and “hush hush” training foreclose.

One direction distinction matters for how scared to be of what: the Marlinspike result is device-to-tool — a prepared device corrupting the reports of whatever scans it. Tool-to-device planting requires operator or vendor malice, or upstream compromise of the tool fleet, and must additionally survive the consistency web. Both scenarios defeat testimony; only the second defeats a clean re-extraction, and only device-external corroboration addresses both.

The institutional condition the checks presuppose

Every control above is latent: it exists only if someone performs it, and performance is not the default. Formally the prosecution must prove its case; practically, an accredited-lab extraction report arrives with institutional authority, and the defence must affirmatively manufacture doubt against it — a de facto burden shift for every tool-attested claim. Scrutiny has to be initiated by the one participant with the least capacity to initiate it.

The activation conditions are worst precisely where the stakes are highest. In a CSAM-class case:

  • The evidence short-circuits analysis. An image classified as abuse material reads as the verdict itself; questions about how it was found register as quibbles about an established fact.
  • The defence task is stigmatized. Challenging the provenance of such material is reputationally costly for counsel and for any expert willing to take the work.
  • The expertise is asymmetric. The state has NFC and accredited laboratories; the defence must fund its own expert, and Swedish courts rarely appoint one. Most cases never reach an expert fight at all — counsel sees an unrebutted report and advises accordingly.
  • The guilt narrative predates the artifact. The confirmation-bias literature (Cognitive and human factors in digital forensics, Confirmation bias in criminal cases) shows ambiguity resolving toward guilt at every filter once a suspect is identified, which is what the invisible-filters problem in Mobile-device extraction and evidentiary selection looks like from the inside.

The documented exceptions prove the mechanism. The Bhima Koregaon planting was uncovered because an unusually resourced defence obtained clone images by court order and commissioned Arsenal Consulting — conditions almost never present. Courts that did engage did so after an external actor had already done the scrutiny. The presumption of innocence, as Rättssäkerhet in Swedish criminal cases puts it, is not only a courtroom instruction; institutions should not extract and selectively narrate more than they can reliably interpret. An extraction pipeline whose checks activate only for the well-resourced inverts that: it is most examinable in the white-collar cases that least resemble the caseload, and least examinable in the stigmatized cases where a wrong finding is most final.

The Swedish terminal case: contraband evidence seals itself

In Sweden the CSAM scenario goes one step further than scarce scrutiny: independent defence re-examination is not merely unaffordable but criminal. Swedish possession doctrine counts deleted-but-recoverable files as possession (RH 2007:21), so a raw forensic image whose unallocated space may contain recoverable material is itself contraband; the forfeiture act for child-pornography material directs that the storage media be destroyed (NJA 2014 s. 14); and the defence’s copy right in 23 kap. 21 a § RB may be withheld for the material’s “omfattning eller karaktär” — its extent or character. The consequence: an Arsenal Consulting-style re-analysis, in which a defence-hired expert receives the image and tests the extraction, cannot lawfully happen. The expert would commit the possession offence by receiving the evidence. The only lawful examiners of the artifact are the police, NFC, and the court — the state’s own chain.

This is the terminal form of the unexaminable-evidence structure: not a gap in practice that resources or doctrine could close, but a class of cases where the evidence’s legal status makes adversarial verification impossible by definition. What remains to the defence is argument over descriptions, hash lists, procedures, and testimony — everything except the artifact itself. Whether any Swedish case has carved out supervised or court-controlled defence-expert access to such images, and what courts do when the defence raises planting or tool-compromise hypotheses it is legally barred from testing, is the open residue (Are Swedish mobile-extraction filters disclosed at trial tracks the disclosure side).

Swedish NFC accreditation and RåV 2022:31 supply the first category at the process level. Whether routine Swedish practice also delivers the second — raw-image access for the defence, disclosed filters and versions, external corroboration — is the open question in Are Swedish mobile-extraction filters disclosed at trial, and the audit list in Mobile-device extraction and evidentiary selection is the checklist against which to measure it.

Until the defence can verify rather than trust, a secret forensic method is evidence about the tool’s output, not independent evidence about the world.

  1. Cellebrite asks cops to keep its phone hacking tech ‘hush hush’, TechCrunch, August 2023. ↩

  2. Högsta domstolen prövar inte Encrobevisning, SVT, 2021. ↩

  3. Metoder och standardförfaranden för forensiska undersökningar, Polisen / NFC. ↩

  4. Rättslig vägledning 2022:31, Forensiska undersökningar, Åklagarmyndigheten, 2022. ↩

  5. Exploiting vulnerabilities in Cellebrite UFED and Physical Analyzer from an app’s perspective, Signal Blog, April 2021. ↩

  6. Jailed activist Rona Wilson’s computer was compromised, finds digital forensics analysis, The Hindu, February 2021; Arsenal Consulting reports I and II, 2021. ↩

Built on 9 sources (1 archived here, 8 external).

Working out connections…