Litigating Forensic Fingerprints Brendan Max Chief Forensic Science
Litigating Forensic Fingerprints Brendan Max Chief, Forensic Science Division Cook County Public Defender Office
Typical Fingerprint Claims • Biology- Fingerprints are unique. • Method- My method of analyzing fingerprints is scientifically valid and objective. • Certainty- When I use this method, there is no real chance that I get it wrong. • Opinion- All that said, the latent print in this case matches the defendant and no one else on Earth. No t Tr ue!
Typical Fingerprint Claims • Biology- Fingerprints are unique.
Uniqueness • To claim 100% match, uniqueness was a required proposition. • Examiners had to claim that no two people could have the same friction ridge patterns on their fingertips. • The snowflake argument- nature does not repeat itself.
Uniqueness Ridge detail on all 70 billion fingers is unique. (assuming 7 billion world population)
Uniqueness Ridge detail on 1. 8 trillion fingers is unique. (108 billion people have lived on Earth)
Uniqueness Ridge detail will never repeat on all fingers to come 100, 000, 000, 000 (crude way to say all who have lived or will live in the future)
Uniqueness “SUCH A PROPOSITION IS HIGHLY DUBIOUS, AND RELIES ON CLAIMS AND OBSERVATIONS THAT HAVE NEITHER BEEN RECORDED NOR COMPILED IN A SYSTEMATIC MANNER. ”
Uniqueness “THE CONCEPT OF ‘UNIQUENESS” HAS MORE THE QUALITIES OF A CULTURAL MEME THAN A SCIENTIFIC FACT. ”
Uniqueness “A truly random sample of a large number of human beings may indicate that none of them have the same mother; but we know that to conclude that not one person on the Earth shares the same mother defies common sense. ”
Uniqueness
Typical Fingerprint Claims • Biology- Fingerprints are unique. No t Tr ue!
Typical Fingerprint Claims • Biology- Fingerprints are unique. • Method- My method of analyzing fingerprints is scientifically valid and objective.
Fingerprint Method ACE-V • Analysis • Comparison • Evaluation • Verification
Fingerprint Method ACE-V • Analysis = I look at the print • Comparison = I look at the print • Evaluation = I look at the print • Verification = My coworker does too
Fingerprint Method “ACE-V PROVIDES A BROADLY STATED FRAMEWORK FOR CONDUCTING FRICTION RIDGE ANALYSES. HOWEVER, THIS FRAMEWORK IS NOT SPECIFIC ENOUGH TO QUALIFY AS A VALIDATED METHOD FOR THIS TYPE OF ANALYSIS. ACE-V DOES NOT GUARD AGAINST BIAS; IS TOO BROAD TO ENSURE REPEATABILITY AND TRANSPARENCY; AND DOES NOT GUARANTEE THAT TWO ANALYSTS FOLLOWING IT WILL OBTAIN THE SAME RESULTS. FOR THOSE REASONS, MERELY FOLLOWING THE STEPS OF ACE-V DOES NOT IMPLY THAT ONE IS PROCEEDING IN A SCIENTIFIC MANNER OR PRODUCING RELIABLE RESULTS. ”
Fingerprint Method “ALTHOUGH ACE-V IS A SYSTEMATIC PROCESS, MEANING THAT THE EXAMINATION PROCEEDS IN AN ORDERLY AND LOGICAL FASHION, THIS DOES NOT, BY ITSELF, DEMONSTRATE THAT THE RESULTS ARE ACCURATE AND REPRODUCIBLE. ”
“. . . orderly and logical fashion. ”
“. . . orderly and logical fashion. ” • New “scientific” method for obtaining food from Mc. Donalds. • “ACE-V”
“. . . orderly and logical fashion. ” • Assess the menu items • Communicate choices to the employee • Exchange currency for food items • Verify that I have received correct items
“. . . orderly and logical fashion. ” • Am I now a scientist every time I walk into Mc. Donalds?
Fingerprint Method • Well, ACE-V may not technically be “validated, ” but it is objective.
Fingerprint Method
Neumann et al. , “Improving the Understanding and the Reliability of the Concept of ‘Sufficiency’ in Friction Ridge Examination, ” Plos. One (2013).
Neumann et al. , “Improving the Understanding and the Reliability of the Concept of ‘Sufficiency’ in Friction Ridge Examination, ” Plos. One (2013).
Fingerprint Method FBI White Box Study
Ulery et al. , “Measuring What Latent Fingerprint Examiners Consider Sufficient Information for Individualized Determinations, ” Plose. One (2014).
21 features (association) 11 features (inconclusive) 5 features (exclusion)
Fingerprint Method • Well, a lot of scientific processes have subjective elements. In the hands of well-trained examiners, this method is very reliable.
Fingerprint Method • Most common ways for scientists to determine if a method is scientifically reliable is to assess reproducibility and repeatability.
Fingerprint Method • Reproducibility: Examiner 1: “Association” Examiner 2: “Association”
Fingerprint Method • Repeatability: Day 1: “Association” Day 20: “Association”
Fingerprint Method FBI Black Box Study
Fingerprint Method
Fingerprint Method
Fingerprint Method
Fingerprint Method • In addition to these problems with the ACE-V method, some examiners use a version of the ACE-V method that is even more flawed.
Fingerprint Method
Fingerprint Method Ridge Ending? Bifurcation?
Fingerprint Method
Is there a problem with this method?
Fingerprint Method The problem is cognitive bias: • Exposure to unnecessary information prior to making any judgment calls, which can influence the judgment calls. • Whether we realize it or not, context affects interpretation.
Fingerprint Method
Fingerprint Method “Let me know if you can find the Dalmatian dog we are looking for in your forensic data? ”
Fingerprint Method
Fingerprint Method Four of five examiners changed their opinion.
Fingerprint Method
Biasing information affected 16% of opinions.
Fingerprint Method
51
52
53
54
Fingerprint Method The OIG determined that feature decisions “were adjusted or influenced during the comparison phase by reasoning ‘backward’ from features that are visible in the Mayfield exemplars. ”
Fingerprint Method “As a matter of scientific validity, examiners must be required to ‘complete and document their analysis of a latent fingerprint before looking at any known fingerprint’ and must separately document any data relied upon during comparison or evaluation that differs from the information relied upon during analysis. The FBI adopted these rules following the Madrid train bombing case misidentification; they need to be universally adopted by all laboratories. ” 56
Typical Fingerprint Claims • Biology- Fingerprints are unique. • Method- My method of analyzing fingerprints is scientifically valid and objective. No t Tr ue!
Typical Fingerprint Claims • Biology- Fingerprints are unique. • Method- My method of analyzing fingerprints is scientifically valid and objective. • Certainty- When I use this method, there is no real chance that I get it wrong.
Certainty Claims Two valid error rate studies: • Ulery/FBI Black Box • Miami Dade 59
Certainty Claims • Published in 2011 • “The accuracy of decisions made by latent print examiners has not been ascertained in a largescale study, despite over one hundred years of the forensic use of fingerprints. ” 60
Certainty Claims • Ulery FBI Black Box. • Error rate- 1 false positive in 604 cases. • Upper bound- 1 in 306 cases. 61
Certainty Claims • Miami Dade Study. • Error rate- 1 false positive in 24 cases. • Upper bound- 1 in 18 cases. 62
Certainty Claims And there are three reasons to think that error rates are even higher in actual criminal cases than the Ulery and Miami Dade studies suggest: • Hawthorne Effect • Inconclusive rates • Representativeness
Certainty Claims Hawthorne Effect: • The alteration of behavior by the subjects of a study due to their awareness of being observed.
Certainty Claims • Hawthorne Effect • “Ideally, a study would be conducted in which participants were not aware that they were being tested. ” 65
Certainty Claims Inconclusive rates: • In error rate studies, examiners can simply take a pass if the comparison is too difficult. • Each time an examiner takes a pass on a difficult comparison, this represents a potential false positive averted. • If examiners are taking passes in error rate studies at a much higher rate than in casework, the overall false positive rates from the studies are wrong.
Certainty Claims • False positive rate of 1 in 604 with upper bound of 1 in 306 67
Certainty Claims 68
Certainty Claims If your local fingerprint lab is not taking a pass on 20% of suitable latent comparisons, then the error rate of 1 in 604 is too low. 69
Certainty Claims 70
Certainty Claims 71
Certainty Claims Representativeness: • Must consider how difficult the latent print is in criminal case v. how difficult the latent prints were in error rate studies. • If the latent in a case is a difficult one, the overall error rates from published studies will underestimate the chance of the error because they were based on a combination of easy and more difficult latent print examinations. 72
In Ulery/FBI study, only 6 false positives in 4083 decisions, but. . . 73
1 false positive in 30 decisions for this difficult print set. 74
1 false positive in 26 decisions for this difficult print set. No reason to believe in low FP rates in difficult cases! In difficult case, a FP rate of 1 in 30 is more accurate than 6 in 4083. 75
Certainty Claims • In cases involving AFIS, errors are even more likely than in cases with an identified suspect.
Certainty Claims • The job of AFIS is to search millions of prints to find several/many that look very similar to the latent print. • Close non-matches. • It is like a really well constructed lineup.
Certainty Claims A fingerprint case with an identified suspect is akin to a lineup where the suspect stands out: “light skinned, braids, light eyes”
“light skinned, braids, light eyes”
“light skinned, braids, light eyes”
Certainty Claims An AFIS case is akin to a lineup where the suspects all look very similar: “light skinned, short dark hair, brown eyes”
“light skinned, short dark hair, brown eyes”
AFIS cases are more challenging- there will be more errors.
Certainty Claims “Recommendation 3. 6: When comparing latent prints to exemplars generated through AFIS searches, examiners must recognize the possibility and dangers of incidental similarity. Adjustments such as a higher decision threshold, stricter tolerances for differences in appearance, and explicit feature weighting need to be considered. Modifying quality assurance practices for this scenario also should be considered. ”
Typical Fingerprint Claims • Biology- Fingerprints are unique. • Method- My method of analyzing fingerprints is scientifically valid and objective. • Certainty- When I use this method, there is no real chance that I get it wrong. No t Tr ue!
Typical Fingerprint Claims • Biology- Fingerprints are unique. • Method- My method of analyzing fingerprints is scientifically valid and objective. • Certainty- When I use this method, there is no real chance that I get it wrong. • Opinion- All that said, the latent print in this case matches the defendant and no one else on Earth.
Match Opinions • This print came from the defendant to the exclusion of all others. • This print is identified as having come from the defendant. • This print is individualized to the defendant. • The defendant is the source of this print. Absolute Source Attribution
Match Opinions “AS INDICATED ABOVE, A FINGEPRINT IDENTIFICATION WAS TRADITIONALLY CONSIDERED AN ‘INDIVIDUALIZATION, ’ MEANING THAT THE LATENT PRINT WAS CONSIDERED IDENTIFIED TO ONE FINGER OF A SPECIFIC INDIVIDUAL AS OPPOSED TO EVERY OTHER POTENTIAL SOURCE IN THE UNIVERSE. HOWEVER, THE RECENT ATTENTION FOCUSED ON THIS ISSUE REVEALS THAT THIS DEFINITION NEEDLESSLY CLAIMS TOO MUCH, IS NOT ADEQUATELY ESTABLISHED BY FUNDAMENTAL RESEARCH, AND IS IMPOSSIBLE TO VALIDATE SOLELY ON THE BASIS OF EXPERIENCE. ”
Match Opinions “As a result, several well respected and authoritative scientific committees and organizations have recommended forensic science laboratories not report or testify, directly or by implication, to a source attribution to the exclusion of all others in the world or to assert 100% certainty and state conclusions in absolute terms when dealing with population issues. ”
Match Opinions “The latent print on Exhibit X and the record fingerprints bearing the name XXXX have corresponding ridge detail. The likelihood of observing this amount of correspondence when two impressions are made by different sources is considered extremely low. ”
Match Opinions • Prosecutor response: “This print is identified has having come from the defendant to the exclusion of all others. ”
Match Opinions • Prosecutor response: “This print is identified has having come from the defendant to the exclusion of all others. ”
Match Opinions “This blue car belongs to me to the exclusion of all others. ” “This blue car belongs to me. ”
Match Opinions
Match Opinions
Match Opinions
Typical Fingerprint Claims • Biology- Fingerprints are unique. • Method- My method of analyzing fingerprints is scientifically valid and objective. • Certainty- When I use this method, there is no real chance that I get it wrong. • Opinion- All that said, the latent print in this case matches the defendant and no one else on Earth. No t Tr ue!
Fingerprints in 2017 • Sounds kind of objective and reliable, but the research tells a different story. • There is a lot of change and reform going on within the discipline, and some labs have gotten the message while others have not. • There is a lot of room for advocacy/litigation.
Fingerprint Litigation Strategies • Conduct pre-trial interviews • Litigate admissibility motions • Challenge at trial
9 10 11 8 7 4 6 5 2 1 3
Fingerprint Litigation Strategies BM then asked JC whether he was familiar with the Ulery studies on reproducibility and repeatability. JC said that he could not testify to these studies, and also said that “I am not an expert in reproducibility studies. ” BM then asked JC whether the process of analyzing a print and coming to an opinion is a subjective process, meaning that two different examiners can come to different opinions while looking at the same print evidence. JC said, “I don’t know what words you are trying to put in my mouth. ”
Fingerprint Litigation Strategies BM asked JC whether he knew of the published error rate studies in the field, and JC said that he could only tell me whethere was an error in this case. BM asked JC what he would say about published error rate studies if asked at trial. JC said that he could not testify to error rates in the field, could only testify to whethere was an error in this case, and stated that “I am not an expert in error rates. ”
Fingerprint Litigation Strategies BM asked: Can you describe any ways in which the chances of an erroneous fingerprint association are increased when you use AFIS, than in the situation where you don’t use AFIS and just compare the latent to known suspects? CS stated no. BM asked: Have you been trained in any way that using AFIS to associate a print to a suspect creates new types of potential error in a case? CS stated that she had never received any such training.
Fingerprint Litigation Strategies “This new research goes to the heart of testimony by fingerprint examiners in two ways - it has limited what examiners should say about a fingerprint association and has reformed best-practices to minimize error. Any examiner not familiar with this research and reform is not engaging in best-practices, and will overstate the probative value of a fingerprint association at trial. ”
Fingerprint Litigation Strategies “Officer XXXXX appears to be unaware of major developments in his own profession, and has not received the training necessary to qualify as a fingerprint expert in 2016. Based on a pre-trial interview with Officer XXXXX, it is clear that he is unaware of the most basic and fundamental reforms in his field and the important research that has driven these reforms. Because of this, Officer XXXXX fails to follow current best practices in the field, and would misstate the probative value of the fingerprint evidence in this case were he permitted to testify as an expert in forensic fingerprint comparison. ”
Fingerprint Litigation Strategies
Fingerprint Litigation Strategies
Fingerprint Litigation Strategies
Fingerprint Litigation Strategies Illinois Rule Evidence 703: “. . . If of the type reasonably relied upon by experts in the particular field in forming opinions or inferences upon the subject, the facts or data need not be admissible in evidence. ”
Fingerprint Litigation Strategies Illinois Rule Evidence 703: • Does not matter if the examiner has read the article • Does not matter if the examiner agrees with the article • Only matters that the article is the type of source that experts in the field typically rely on
Fingerprint Litigation Strategies
Fingerprint Litigation Strategies • Some publications are admissible as exceptions to hearsay through the government documents exception. • State v. Harper, 821 N. W. 2 d 412 (Wis. 2012). • Wisconsin court held that the NAS report was admissible under the government document exception due to the fact that the National Academy of Sciences is federally chartered by the U. S. Congress to draft reports on scientific topics.
Fingerprint Litigation Strategies Case outcomes have been great for our clients: • Often, the state makes amazing offers rather than respond to our motions. • When they don’t, we win at trial.
Fingerprint Litigation Strategies Case outcomes have been great for our clients: • March res. burg jury trial • March res. burg bench trial- Not guilty
Fingerprint Litigation Strategies “Mr. Daniels, as a latent print examiner, is woefully unknowledgeable about anything other than one DOJ report. . . he seems oblivious and doesn’t seem concerned about educating himself as to what’s going on in his own field. . . he certainly doesn’t strike me as the world’s greatest latent examiner. ” “The Chicago Police Department procedures where they have no audits, no verification of their procedures, no participation- they don’t follow the FBI rules- casts a doubt on their whole departmental procedures. I find that to be woefully inadequate adding to the problems for the State proving their case beyond a reasonable doubt. ”
Fingerprint Litigation Strategies “I’m also questioning the value of testimony of Ms. Seavers when she testifies that her training entailed a bachelor’s degree in architecture and design with no background in biology, science, or mathematics. . . there is no certification process certifying her as a latent print examiner, and that during the period between 2008 and 2015. . . she had little, if any , follow-up training. . . I’m also concerned about the fact that the CPD Lab has no accreditation, no auditing system, no quality review, no error check process, no written professional development bench marks, that there’s no standard operating procedures or guideline. ”
THANKS Brendan Max brendanpmax@yahoo. com
- Slides: 118