Chapter 3 — The Trace and What It Can Bear

In 2009 the US National Academy of Sciences published the report that broke the profession's nerve. Its central conclusion, in the committee's own framing: with the exception of nuclear DNA analysis, no forensic method had been rigorously shown to be capable of consistently and with a high degree of certainty demonstrating a connection between evidence and a specific individual or source. [ESTABLISHED] (NAP; Innocence Project, ten years on)

The report's sharpest observation was structural rather than technical: before it, forensic science had been tested and appraised by standards described in law rather than in the scientific method. Courts had been deciding what counted as science. [ESTABLISHED] (forensicstats.org)

Seven years later the President's Council of Advisors on Science and Technology went discipline by discipline and asked a narrower question: does this method have foundational validity — has anyone demonstrated, empirically, that practitioners using it produce accurate results at a known error rate?

This chapter is that scorecard, plus the strongest objections to it, plus the thing underneath both that turns out to matter more than any individual technique.

The scorecard

(PCAST report PDF)

Read the fingerprint row twice. It is the most misunderstood line in forensic science. Fingerprints passed. They are valid. And in the FBI/Noblis "black box" study — 169 examiners, 3,628 non-mated comparisons — examiners produced six false positives: a rate of about 1 in 604, with a 95% upper confidence bound near 1 in 306. [ESTABLISHED] (NIJ; original study PDF)

Both facts are true simultaneously. A discipline can be genuinely scientific and still be wrong once in every few hundred identifications, and if you run a national database against millions of records, "once in six hundred" is not a rounding error — it is a queue of innocent people. Missed identifications, incidentally, run far higher than false ones; the discipline is deliberately tilted away from false incrimination.

What died

Hair microscopy. The FBI, the Innocence Project and the National Association of Criminal Defense Lawyers jointly reviewed FBI microscopic hair comparison testimony given before 2000. In the 268 cases where examiner testimony was used to inculpate a defendant at trial, examiners made scientifically invalid statements — claims exceeding what the science could support — in 257 cases: 96%. Thirty-five of those defendants had received a death sentence. Erroneous hair statements were found in 33 of the 35. [ESTABLISHED] (FBI press release)

That is not a discipline with a quality problem. That is a discipline that was never doing what it said it was doing, sustained for decades by the fact that nobody had checked.

Bite marks. Examiners cannot reliably agree that an injury is a human bite mark, let alone match it to a source. Both underlying assumptions — that human dentition is unique in a forensically useful way, and that skin reliably records it — are unproven. At least 26 people have been wrongfully convicted, arrested or charged on bite-mark evidence. [ESTABLISHED] (NIST IR 8352sup3; Innocence Project)

Arson indicators. Cameron Todd Willingham was convicted in 1992 and executed in 2004 for a house fire that killed his three children. The roughly twenty "arson indicators" used against him — pour patterns, crazed glass, and the rest — have since been shown, one by one, not to distinguish arson from accidental fire. Guidance contradicting the trial theory had already been published before his trial. [ESTABLISHED] on the fire science; [CONTESTED] on the legal characterisation, since the Texas Forensic Science Commission reviewed the case without issuing a declaration of innocence. (Marshall Project; DPIC)

Bloodstain pattern analysis. The NAS found its conclusions "more subjective than scientific" and the associated uncertainties "enormous." A 2021 black-box study of 75 practising analysts found conclusions were often erroneous, and — the detail that belongs in this book — misclassification rates moved depending on whether the analyst had been given biasing context. [ESTABLISHED] (ScienceDirect; NIJ)

The objection, stated properly

The reform position is not unopposed and it would be dishonest to present it that way.

The US Department of Justice formally disputed PCAST's framing. Its argument: PCAST treated pattern-comparison as metrology — measurement — when examiners in fact compare features visually. DOJ agreed that methods must be scientifically valid and reliable; it did not agree with PCAST's specific findings. [CONTESTED] (DOJ statement)

The firearms examiners' professional body, AFTE, contested the "single study" characterisation as an undercount of their field's research base. And the subsequent research partly vindicates them: later work reported false-positive rates of 0.656% for bullets and 0.933% for cartridge cases, with false negatives at 2.87% and 1.87%. [CONTESTED] (AFTE response; Ames II line of research)

Practitioners also argue PCAST relied on studies that did not permit verification and understated real-world reliability. [CONTESTED] (TDCAA)

My reading, offered as a judgement rather than a finding: the practitioners are right that the field has been validated further since 2016, and the reformers are right that the burden was always on the discipline to produce those numbers before testifying, not after being embarrassed into it.

Even the gold standard has a soft centre

DNA passed both reports. It is still the strongest forensic evidence that exists. And it has two weaknesses that matter enormously in practice.

Mixtures. When a sample contains three or more contributors, or very little material, analysts increasingly turn to probabilistic genotyping software. PCAST called the validity studies supporting it scarce, and results depend on inputs the analyst sets — including the assumed number of contributors. [CONTESTED] (Federal Judicial Center; Harvard JOLT)

How much does that matter? In one documented case, the same DNA evidence was run through two leading systems. STRmix returned a likelihood ratio of 24 in favour of the hypothesis that the person was not a contributor. TrueAllele returned between 1.2 million and 16.7 million in favour of inclusion, depending on reference population. The divergence traced to modelling parameters, thresholds, and an ad hoc procedure at some loci. [ESTABLISHED] (Thompson 2023, J. Forensic Sci.; arXiv commentary)

Same sample. Opposite conclusions, seven orders of magnitude apart. Remember this when Part II describes software that scores the similarity between cases: a number produced by a model is a statement about the model as much as about the world.

Transfer. DNA moves. Secondary and even tertiary transfer — person to object to person — is documented, unknown contributors turn up in most samples in some persistence studies, and the research on how often the last person to touch something is actually the major contributor is explicitly inadequate. [CONTESTED] (ScienceDirect)

Your DNA being somewhere is not evidence you were there. It is evidence your DNA was there.

The thing underneath all of it

Now the finding that reorganises the chapter.

Itiel Dror took latent-print examiners and re-showed them pairs of prints they themselves had judged, confidently, about five years earlier. This time the prints came with biasing context — for instance, that the suspect had confessed. Two-thirds of the examiners changed their decision. The prints had not changed. [ESTABLISHED] (Dror, Human Factors in Forensic Science)

The canonical case is Brandon Mayfield. After the 2004 Madrid train bombings, which killed 191 people, the FBI matched a latent print from a bag of detonators to a lawyer in Portland, Oregon. He was arrested. The Spanish National Police then independently identified the print as belonging to an Algerian national, Ouhnane Daoud. The US government paid Mayfield two million dollars.

The Justice Department's Inspector General identified the mechanisms. Confirmation bias: verifying examiners, told a match had already been made, downplayed differences they had themselves noticed. Verification bias: the "independent" verifier knew the answer before looking. [ESTABLISHED] (NPR)

Note carefully what failed. Not the fingerprint method — that is foundationally valid. Not the examiners' competence. The information architecture. They were told the answer before they were asked the question.

The fix, proposed by Dror with Thompson, Meissner, Kornfield, Krane, Saks and Risinger in 2015, is Linear Sequential Unmasking: the examiner documents a conclusion about the trace before being exposed to the reference sample or anything about the suspect, and a taxonomy classifies contextual information by how biasing it is, constraining what may change after unmasking. [ESTABLISHED] as a published protocol; its field-wide adoption I could not measure. [ABSENT] (J. Forensic Sci. 2015; LSU-Expanded, 2022)

The arithmetic of the whole system

The Innocence Project's fixed dataset of 375 DNA exonerations (1989–2020) breaks down as: eyewitness misidentification 69%, misapplication of forensic science 43%, false confessions 29%, informants 17%. [ESTABLISHED] (Innocence Project)

The broader National Registry of Exonerations counter stood at 3,855 exonerations and more than 36,111 years lost when checked on 29 August 2026 — a live figure that will have moved by the time you read this. [ESTABLISHED] (exonerationregistry.org)

The percentages exceed 100 because cases carry multiple causes. That is the real lesson: wrongful convictions are rarely one bad technique. They are a bad identification, confirmed by a bias-exposed examiner, sealed by a confession extracted from someone who should never have been interviewed that way. The last of those is not a forensic failure at all, and it has its own chapter.

And note the counterweight, because it belongs in the same paragraph. The first criminal conviction ever secured by DNA — Colin Pitchfork, 22 January 1988 — came out of an investigation that first cleared a man named Richard Buckland, who had falsely confessed. The technology's founding case is an exoneration and a conviction in the same file. [ESTABLISHED] (ITV News)

The Move — Blind yourself before you look. Before examining any evidence, write down your conclusion about the material in isolation, dated, before you learn who the suspect is or what anyone else believes. If you already know, say so in writing and record that your assessment is context-exposed. This is Linear Sequential Unmasking reduced to something one person can do alone, and it is the only technique in this chapter with direct experimental support for reducing error: Dror's two-thirds reversal is what happens when it is absent.

What would change my mind: if a large field study showed examiners working under sequential unmasking produced the same error rates as those working with full context, the protocol would be theatre. The evidence for it is currently experimental and analogue rather than operational, and I could not find a field-scale outcome evaluation. [ABSENT]