Firearms and Tool Mark Identification Responding to Recent

Firearms and Tool Mark Identification Responding to Recent Criticisms - Ronald Nichols, ATF FSL - SF

Joan Griffin and David La. Magna n Daubert Challenges to Forensic Evidence: Ballistics Next on the Firing Line – The Champion, September/October 2002, 20 -23; 5862.

Michael Saks n Implications of the Daubert Test for Forensic Identification Science – Shepard’s Expert and Scientific Evidence, 1(3), Winter 1994, 427 -432. n The Coming Paradigm Shift in Forensic Identification Science – Science, August 5, 2005, 892 -895 with Koehler

Adina Schwartz n A Challenge to the Admissibility of Firearms and Toolmark Identifications: Amicus Brief Prepared on Behalf of the Defendant in United States v. Kain, Crim. No. 03 -573 -1 (E. D. PA. 2004) – The Journal of Philosophy, Science & Law, Volume 4, December 2004.

Adina Schwartz n A Systemic Challenge to the Reliability and Admissibility of Firearms and Toolmark Identification – The Columbia Science and Technology Law Review, Volume VI, 2005.

Lisa Steele n “All We Want You to Confirm is What You Already Know” A Daubert Challenge to Firearms Identification – Criminal Law Bulletin.

Schwartz Columbia STLR Article n Addresses Concerns Raised by Others in Comprehensive Format

Common Themes Scientific Reliability n Lack of Representative Databases n Lack of Statistical Treatment n Lack of Adequate Proficiency Testing n No Defined Error Rates n Current Computerized Technology Does Not Answer the Questions n

Scientific Reliability Individual Characteristics are Comprised of Non-Unique Marks n Subclass Characteristics May be Confused with Individual Characteristics n Individual Marks on a Particular Tool Change Over Time n

Representative Databases n Like DNA n Issue Broached by Dr. S. Bunch, “Consecutive Matching Striation Criteria: A General Critique. ” Journal of Forensic Science, Vol. 45, No. 5, September 2000, 955 -962.

Statistical Treatment n Like DNA n Issue Addressed by Dr. S. Bunch, “Consecutive Matching Striation Criteria: A General Critique. ” Journal of Forensic Science, Vol. 45, No. 5, September 2000, 955 -962.

Proficiency Testing n Not Representative n Inadequate and Ineffective Administration of Tests to Examiners n Examiner Bias

Error Rates n Like DNA n Claims of Infallibility and Absolute Identifications

Computerized Technology n Does Not Address Needs of Representative Databases and Statistics

Scientific Foundation Established Discipline firmly rooted in application of the Scientific Method n Repeated testing of hypotheses related to identification has been performed n Has resulted in standard statement settings guidelines for defining identification criteria n

AFTE Theory of Identification* IS a standard defining identification criteria n IS established as a theory based on principles of the scientific method n IS a statement of the relevant scientific community n NOT referenced by any recent criticisms n * Theory of Identification, Range of Striae Comparison Reports and Modified Glossary Definitions – an AFTE Criteria for Identification Committee Report. AFTE Journal, 24(2), April 1992, 336 -340.

AFTE Theory of Identification Addresses issues of common origin n Defines observational objectives n Defines sufficiency of agreement to establish common origin, i. e. , identification criteria n Defines meaning of identification n Defines role of subjectivity n

Common Origin n “The theory of identification as it pertains to the comparison of tool marks enables opinions of common origin to be made when the unique surface contours of two tool marks are in ‘sufficient agreement’. ”

Observational Objectives n “Specifically, the relative height or depth, width, curvature and spatial relationship of the individual peaks, ridges and furrows within one set of surface contours are defined and compared to the corresponding features in the second set of surface contours. ”

Identification Criteria n “Agreement is significant when it exceeds the best agreement demonstrated between tool marks known to have been produced by different tools and is consistent with the agreement demonstrated by tool marks known to have been produced by the same tool. ”

What an Identification Means n “The statement that ‘sufficient agreement’ exists between two tool marks means that the agreement is of a quantity and quality that the likelihood another tool could have made the mark is so remote as to be considered a practical impossibility. ”

Subjectivity n “Currently the interpretation of individualization/identification is subjective in nature, founded on scientific principles and based on the examiner’s training and experience. ”

Scientific Reliability Issues of Contention – Part 1: Individual Characteristics are Comprised of Non-Unique Marks

Individual Marks Comprised of Non-Unique Marks n The contention – “As a result of the overlapping individual characteristics of toolmarks made by different tools, examiners who assume that a certain amount of resemblance proves that the same tool produced both test and evidence toolmarks may be wrong…”* *Schwartz, Columbia STLR, p. 6.

Individual Marks Comprised of Non-Unique Marks The issue – Identification Criteria n Addressed by n – Theory of Identification – Methodology – Extensive scientific studies – Individual training, experience and expertise

Theory of Identification n Identification criteria has two parts – exceeds the best agreement demonstrated between tool marks known to have been produced by different tools – and is consistent with the agreement demonstrated by tool marks known to have been produced by the same tool

Methodology n Comparative examinations are an efficient means by which examiners can observe and easily document through photomicrography similarities and differences between patterns of tool marks

Scientific Studies* Controlled studies pursued according to the tenants of the scientific method n Have included an assortment of tools and tool manufacturing processes n Have included tools known to have been manufactured consecutively n Have all supported appropriateness of the Theory of Identification n *Two review articles include, Nichols, R. “Firearm and Toolmark Identification Criteria: A Review of the Literature, ” JFS 42(3), 466 -474 and “Firearm and Toolmark Identification Criteria: A Review of the Literature – Part 2. ” JFS 48(2), 318 -327.

Individual Training and Experience Vital to examiner’s ability to properly interpret observed correspondence n Gained through hours of microscopic comparisons directed at gaining familiarity of correspondence to be expected in known non-match situations of striated and impressed tool marks n

Contention – Small Tool Marks n Schwartz - See e. g. , John E. Murdock…(stating that a “considerable amount of agreement” among striated toolmarks made by different tools is especially likely to be found “if the width of the mark being compared is quite small [say, two millimeters or less]”)…* *Schwartz, at 6 -7, n. 13.

Contention – Small Tool Marks n n Murdock never said or implied this* 2 mm mark came from Butcher & Pugh+ and had nothing to do with expected high likelihood of similarity, but rather was simply defined as the size of a mark likely to have enough individual detail to permit a determination of common origin *Murdock, J. E. “Some Suggested Court Questions to Test Criteria for Identification Qualifications. ” AFTE Journal, 24(1), 69 -75. +Butcher, S. and Pugh, D. “A Study of Marks Made by Bolt Cutters. ” JFSS, 15(2), 120.

Contention – High Percentage of Matching Striations in KNM n Screwdrivers – Schwartz – up to 25% – Reference is Burd and Kirk* in which the 25% is from a single KNM comparison and in which they say the number of matching striations (irrespective of position) as a whole is a poor indicator because tool marks can have high line densities that would logically result in more coincident matching of striations *Burd, D. and Kirk, P. , “Tool Marks: Factors Involved in Their Comparison and Use as Evidence, ” Journal of Police Science, 32(6), 465.

Contention – High Percentage of Matching Striations in KNM n Bolt Cutters – Schwartz – 28% – Reference is Butcher and Pugh* in which the 28% is the highest of a total of 880 KNM comparisons in which only three approached the level of 28% *Butcher, S. and Pugh, D. “A Study of Marks Made by Bolt Cutters. ” JFSS, 15(2).

Contention – High Percentage of Matching Striations in KNM n Bullets – Schwartz – 15 -20% – Reference is Biasotti* who said, “…even under such ideal conditions the average percent match for bullets from the same gun is low and the percent match for bullets from different guns is high, which should illustrate the limited value of percent matching lines without regard to consecutiveness. ” *Biasotti, A. , “A Statistical Study of the Individual Characteristics of Fired Bullets. ” JFS, 4(1).

Scientific Reliability Issues of Contention – Part 2: Subclass Characteristics May be Confused with Individual Characteristics

Subclass Characteristics n Defined as “Discernible surface features of an object which are more restrictive than CLASS CHARACTERISTICS in that they are: (1) Produced incidental to manufacture; (2) Are significant in that they relate to a smaller group source (a subset of the class to which they belong); (3) Can arise from a source which changes over time. Examples would include: bunter marks, extrusion marks on pipe, etc. Caution should be exercised in distinguishing subclass characteristics from INDIVIDUAL CHARACTERISTICS”* *“Theory of Identification, Range of Striae Comparison Reports and Modified Glossary Definitions – an AFTE Criteria for Identification Committee Report. ” AFTE Journal, 24(2), 336 -340.

Subclass Characteristics n The contention – “A tool may also be wrongly identified as the source of a toolmark it did not produce if an examiner confuses subclass characteristics shared by more than one tool with individual characteristics unique to one and only one tool. ”* *Schwartz, p. 8.

Subclass Characteristics The issue – potential for confusion with individual characteristics n Addressed by n – Studies of manufactured tools – Knowledge of manufacturing processes – Knowledge of tool use and how it relates to orientation of any potential subclass characteristics

Newly Manufactured Tools n The contention – “Despite their knowledge of this variation, firearms and toolmark examiners have not formulated any generalizations or statistics about which types of tools can be expected to produce toolmarks with subclass or individual characteristics when they are newly manufactured. ”* *Schwartz, p. 9.

Newly Manufactured Tools n Nineteen different references address the issue of subclass characteristics n These references allow examiners to move from knowledge of manufacturing processes to generalizations regarding subclass potential to specific case application

Subclass References Observed on consecutively made, broach-cut rifle barrels from Cooey* n Observed on consecutively made, broach-cut rifle barrels from Winchester (after absorbing Cooey)+ n Observed on groove impressions of a number of F. I. E. revolvers# n *Churchman, J. The Reproduction of Characteristics in Signatures of Cooey Rifles. R. C. M. P. Gazette, 11(5), 133 -140. +Skolrood, R. “Comparison of Bullets Fired From Consecutively Rifled Cooey. 22 Calibre Barrels. ” Canadian Society of Forensic Science Journal, 8(2), 1975, 49 -52. #Lomoro, V. “Class Characteristics of 32 SWL, F. I. E. Titanic Revolvers. ” AFTE Journal, 6(2), 18 -21.

Subclass References Consecutively button-rifled barrels did not demonstrate subclass carry-over from one barrel to the next* n Potential for subclass characteristics (rare in any event) directly related to manufacturing process+ n *Murdock, J. “A General Discussion of Gun Barrel Individuality and an Empirical Assessment of the Individuality of Consecutively Button Rifled. 22 Caliber Rifle Barrels. ” AFTE Journal, 13(3), 84 -111. +Biasotti, A. “Rifling Methods – A Review and Assessment of the Individual Characteristics Produced. ” AFTE Journal, 13(3), 34 -61

Subclass References n n While limited subclass existed for three barrels manufactured from one button rifled blank, they did not preclude correct identification of bullets to the barrel from which they were fired* Previously examined broach-cut barrels examined afresh – subclass transferred from only some barrels to only some groove impressions on lead bullets, copper jacketed bullets did not display any subclass characteristics from the barrels+ *Matty, W. “A Comparison of Three Individual Barrels Produced From One Button Rifled Barrel Blank. ” AFTE Journal, 17(3), 64 -69. +Tulleners, F. and Hamiel, J. “Sub Class Characteristics of Sequentially Rifled. 38 Special S&W Revolver Barrels. ” AFTE Journal, 31(2), 117.

Subclass References n Lathe turned firing pins produced remarkably similar concentric circles causing examiners to look for other markings within the firing pin* *Matty, W. and Johnson, T. “A Comparison of Manufacturing Marks on Smith & Wesson Firing Pins. ” AFTE Journal, 16(3), 51 -56.

Subclass References n Cut, stamped, and milled breech faces can all result in subclass causing examiners to look at other features within the breech face markings* *Lardizabal, P. “Cartridge Case Study of the Heckler and Koch USP. ” AFTE Journal, 27(1), 49 -51; Thompson, E. “False Breech Face Id's. ” AFTE Journal, 28(2), 95 -96; Matty, W. “Lorcin L 9 MM and L 380 Pistol Breechface Tool Mark Patterns. ” AFTE Journal, 31(2), 134137; Lopez, L. and Grew, S. “Consecutively Machined Ruger Bolt Faces. ” AFTE Journal, 32(1), 19 -24.

Subclass References n CNC (Computer Numerical Controlled) Machining – Consecutively manufactured Ruger bolt faces demonstrated subclass yet significant individual features to permit identification* – Consecutively manufactured. 22 caliber barrels with subclass on the breech ends causing carryover of characteristics in anvil marks+ *Coffman, B. “Computer Numerical Control (CNC) Production Tooling and Repeatable Characteristics on Ten Remington Model 870 Production Run Breech Bolts. ” AFTE Journal, 35(1), 49 -54. +Nies, R. “Anvil Marks of the Ruger MKII Target Pistol – An Example of Subclass Characteristics. ” AFTE Journal, 35(1), 75 -78.

Subclass References n Ten Consecutively Manufactured Extractors* – Persistent subclass on two surfaces – Despite subclass on tools, test marks did not show subclass *Nichols, R. “Firearm and Tool Mark Identification: The Scientific Reliability and Validity of the AFTE Theory of Identification Discussed Within the Framework of a Study of Ten Consecutively Manufactured Extractors. ” AFTE Journal, 36(1), 67 -88.

Subclass References n Other Tools – Screwdrivers – could exist unless ground after stamping or die process* – Pliers – though not present from broach cut process, would have been irrelevant because direction of use perpendicular to orientation of tool marks+ *Burd, D. and Kirk, P. , “Tool Marks: Factors Involved in Their Comparison and Use as Evidence, ” Journal of Police Science, 32(6), 1942. +Cassidy, F. “Examination of Tool Marks from Sequentially Manufactured Tongue and Groove Pliers. ” JFS, 25(4), 796 -809.

Subclass References n Persistence across generations – defects in molds can result in similar marks on many items* and a defect in a master mold could be reproduced in many molds which in turn…+ *Kreiser, J. “Identification of Cast Bullets and Their Molds. ” AFTE Journal, 17(3), 88 -90. +Miller, J. “An Introduction to the Forensic Examination of Tool Marks. ” AFTE Journal, 33(3), 233 -248.

Rate of Change for Subclass Characteristics n The contention – “Nor have they developed statistics or generalizations about the rate(s) at which subclass characteristics on toolmarks produced by various types of tools can be expected to be replaced and/or joined by individual characteristics. ”* *Schwartz, p. 9

Rate of Change for Subclass Characteristics n Contention is – True, and – Irrelevant n Can assess potential for subclass n Then assess potential for transference n If potential exists in both cases, examiner should be conservative and use other features for purposes of identification

Distinguishing Between Subclass and Individual Characteristics n The contention – “Firearms and toolmark examiners have also failed to develop any rules for distinguishing between subclass and individual characteristics. ”* *Schwartz, p. 9.

Distinguishing Between Subclass and Individual Characteristics n Contention is NOT true – Previously cited 19 studies – Specifically, “That the occurrence of subclass characteristics in rifled firearm barrels is a rare event that can be easily determined by direct inspection of the rifling or a barrel cast…”* *Tulleners, F. and Hamiel, J. “Sub Class Characteristics of Sequentially Rifled. 38 Special S&W Revolver Barrels. ” AFTE Journal, 31(2), 121.

Training and Experience Vital to Subclass Determinations Schwartz contends that “examiners can rely on their personal familiarity”* n NOT true n – 19 studies – Regardless, such training and experience is vital for an examiner to assess and evaluate potential for subclass *Schwartz, p. 9.

Scientific Reliability Issues of Contention – Part 3: Changing Surface of Tools Over Time

Tool Surface Changes n The contention is that because the surface of a tool changes over time, this presents a “barrier in the way of firearms and toolmark identification’s goal of individualization. ”* *Schwartz, p. 11.

Tool Surface Changes n NOT detrimental – It is through use that tools acquire more individual characteristics vital to individualization – Tool surfaces do not change so rapidly so as to preclude examiner’s ability to compare and render conclusions of identification

Studies Concerning Tool Surface Changes 501 bullets from a 5. 56 NATO rifle – first and last compared and identified* n Lead build-up could affect reproducibility such that best results obtained with bullets fired in close sequence+ n *Hamby, J. “Identification of Projectiles. ” AFTE Journal, 6(5/6), 22. +Biasotti, A. “Bullet Bearing Surface Composition and Rifling (Bore) Conditions as Variables in the Reproduction of Individual Characteristics on Fired Bullets. ” AFTE Journal, 13(2), 94 -102.

Studies Concerning Tool Surface Changes 501 test fired bullets and cartridge cases from Raven, 25 ACP caliber pistol – identified first and last bullets and cartridge cases* n Lead build-up studied in 900 test firings – bullets fired 25 -50 shots apart still identifiable+ n *Shem, R. and Striupaitis, P. “Comparison of 501 Consecutively Fired Bullets and Cartridge Cases From a. 25 Caliber Raven Pistol. ” AFTE Journal, 15(3), 109 -112. +Kirby, S. “Comparison of 900 Consecutively Fired Bullets and Cartridge Cases From a. 455 Caliber S&W Revolver. ” AFTE Journal, 15(3), 113 -126.

Studies Concerning Tool Surface Changes 5000 test fired. 45 ACP caliber bullets and cartridge cases – identified first and last bullets and cartridge cases* n 7100 test fired 5. 56 mm caliber cartridge cases – ejectors from #9 and #7060 identified+ n *Ogihara, Y. , et al. “Comparison of 5000 Consecutively Fired Bullets and Cartridge Cases From a. 45 Caliber M 1911 A 1 Pistol. ” AFTE Journal, 15(3), July 1983, 127 -140. +Schecter, B. , et al. “Extended Firing of a Galil Assault Rifle. ” AFTE Journal, 24(1), 37 -45.

Studies Concerning Tool Surface Changes 4000 test fired steel jacketed bullets – identified first and last bullets* n 25 cuts by bolt cutters in lead – reproducible through all 25; steel shackle cuts showed differences attributed to metal of shackle+ n *Dolleing, B. “Comparison of 4000 Consecutively Fired, Steel-Jacketed Bullets. ” Presented at the 53 rd Annual AAFS Seminar, Seattle, WA, February 2001. +Hall, J. “Consecutive Cuts by Bolt Cutters and Their Effect on Identification. ” AFTE Journal, 24(3), 260 -272.

Tool Surface Changes n Tool surfaces will change – More individual characteristics – Not rapid enough to de-value identification discipline – Not significant enough to prevent ability to make identifications – Worse case scenario – cannot achieve an association

Scientific Basis for Firearms and Tool Mark Identification Has Been Validated

Not Validated? n Criticisms focus on DNA analogy – Statistics to deal with uncertainty – A more objective way – Establishing representative databases

Uncertainty n The contention – Absolute individualization is not possible – Need statistics to deal with uncertainty n The claim – The goal of the discipline is to individualize* – Individualization is not possible therefore, not validated *Schwartz, p. 2.

Uncertainty n Criticisms fail to cite AFTE Theory of Identification – Never claims absoluteness – “likelihood…is so remote as to be considered a practical impossibility”* *AFTE Glossary. “Theory of Identification as it Relates to Toolmarks. ” AFTE Journal, 30(1), Winter 1998, 86.

Meaning of Identity n What do we mean when we make claims of identity? – Absolute – Practical – Not really sure n How do we support our claim? – Infallible – Training and experience?

Defining Identity n Paul Kirk* – Identity means practical or determinable identity only – Cannot ever establish absolute identity because cannot test all objects *Kirk, P. Crime Investigation. New York: Interscience Publishers, 1953, 16.

Defining Identity n David Stoney* – “proving uniqueness” is a “ridiculous notion” – “Even without theoretical models and statistics, we can, and do, make absolute identifications. We can apply scientific, critical judgment, expert and informed, to make the subjective determination of identity (or less absolutely, of ‘very *Stoney, D. rare’). ” “What Made Us Think We Could Individualize Using Statistics? ” JFSS. 31(2), April/June, 1991.

Defining Identity n Important to – Concede not absolute – Intelligently discuss issue of uncertainty n Class characteristics alone significantly narrow range of possibilities n Interpretation of observed correspondence is subjective n Some statistical models have been developed

Objectivity n Criticisms confused with regard to issue of objectivity/subjectivity and how it relates to claimed identifications (absolute)

Objectivity n Objectivity and Subjectivity – Observations made by trained examiners are objective – Interpretation of objective observations is subjective

Objectivity and CMS is not inherently more objective than non-CMS, pattern matching n CMS is simply a means of describing the best known non-match n The CMS criteria has been tested through 6, 000+ known non-matches and has not been violated, BUT… n

Objectivity and CMS Every object not tested n Therefore, even with CMS there is some level of uncertainty with regard to what an identification means n “… likelihood…is so remote as to be considered a practical impossibility”

Representative Databases Criticisms focus on discrete categories of tools n BUT, individuality is not necessarily dependent on the tool itself, but by the manufacturing process of the tool surface n

Representative Databases Many types of tools, only so many ways in which they can be manufactured n Examining propensity for creation of class, subclass and individual characteristics based on manufacturing method n

Representative Databases n DNA Analogy Inappropriate – DNA deals with subclass characteristics only; statistics are a means for calculating frequency of a combination of subclass characteristics – DNA is genetically based, therefore, discrete populations are needed to account for expected differences in frequencies

Representative Databases n DNA Analogy Inappropriate – Firearms identification based on individual characteristics, not subclass – Manufacturing methods are limited and propensity for class, subclass and individual characteristics can be inferred from the manufacturing method

Validation n Science of firearms and tool marks identification has been validated – multitude of studies have shown it is possible to distinguish between tool marks produced by tools, even those consecutively manufactured

Validation Criticisms focus on validation of the identification criteria n Issue pales when it is understood what is meant by identity n

Role of Statistics n Defining the remoteness of the likelihood “How remote is that practical impossibility? ”

Role of Statistics n Schwartz claims that “Firearms and toolmark examiners do not even attempt to answer this question. ”* *Schwartz, p. 13.

Role of Statistics Biasotti, 1959 in examining application of CMS to identification criteria discussed probabilities* n Brackett, 1970, idealized striated marks to developed probabilistic models for CMS+ n *Biasotti, A. , “A Statistical Study of the Individual Characteristics of Fired Bullets. ” JFS, 4(1), Jan. 1959. +Brackett, J. “A Study of Idealized Striated Marks and Their Comparison Using Models. ” JFSS, 10(1), Jan. 1970, 27 -56.

Role of Statistics n n n Blackwell and Framan, 1980 – using Brackett’s formulae similar results to Biasotti* Deinet, 1981 – probabilities of random occurences of matches using striated tool marks+ Uchiyama, 1988 – using large tolerances for striation correspondence, similar results to Biasotti# *Blackwell, R. and Framan, E. , “Automated Firearms Identification Systems AFIDS: Phase I. ” AFTE Journal, 12(4), Oct. 1980, 11 -37. +Deinet, W. Studies of Models of Striated Marks Generated by Random Processes. JFSS, 26(1), Jan. 1981, 35 -50. #Uchiyama, T. , “A Criterion for Land Mark Identification. ” AFTE Journal, 20(3), July 1988, 236251.

Role of Statistics n n n Miller and Neel, 2004 – statistical significance of various CMS runs in 1, 000 known non-match comparisons* Stone, 2003 – mathematical model for impressed tool marks+ Collins, 2005 – empirical study testing Stone’s mathematical model# *Miller, J. and Neel, M. “Criteria for Identification of Toolmarks Part III Supporting the Conclusion. ” AFTE Journal, 36(1), 7 -38. +Stone, R. “How Unique are Impressed Toolmarks? ” AFTE Journal, 35(4), 376 -383. #Collins, E. “How ‘Unique’ are Impressed Toolmarks? An Empirical Study of 20 Worn Hammer Faces. ” Presented at the CAC Semi-Annual Seminar, May 2005, Oakland, CA.

Role of Statistics n Bayesian – Arbitrary nature of likelihood ratio scale does not define remoteness of practical impossibility n Probabilistic – Adding or multiplying – determination of randomness; is it determinable?

Role of Proficiency Testing and Error Rates in Firearms and Tool Mark Identification

Proficiency Tests and Error Rates n Proficiency Testing – Provide a general indicator of error rate – Statistics not wholly capable of answering question of identification criteria validation because of non-quantitative aspects – Therefore,

Proficiency Testing and Error Rates …proficiency tests can offer to the court a reliable practical indicator of how often the profession, using accepted procedures, practices and controls, makes a false identification. * *Grzybowski, R. ; Miller, J. ; Moran, B. ; Murdock, J. ; Nichols, R. ; Thompson, R. “Firearm/Toolmark Identification: Passing the Reliability Test Under Federal and State Evidentiary Standards. ” AFTE Journal, 35(2), 216.

Proficiency Testing and Error Rates n Firearms – CTS: 1978 -1997 – 0. 9% – CTS: 1998 -2002 – 1. 0% n Tool Marks – CTS: 1978 -1997 – 1. 0% – CTS: 1998 -2002 – 1. 5%

Proficiency Testing and Error Rates n Does not mean that every firearms and tool mark identification is right only a certain percentage of the time BUT, n A certain percentage of reported results included a false identification

Validation Studies and Error Rates Brundage, 1992 – ten consecutively manufactured Ruger barrels; 30 laboratories – 15 unknowns identified to correct barrel in each instance* n Hamby, 2003 – further study on Brundage barrels, 294 respondents with no false identifications+ n *Brundage, D. “The Identification of Consecutively Rifled Gun Barrels. ” Presented at the 25 th Annual AFTE Training Seminar, Indianapolis, IN, June 1994. +Hamby, J. “An Update on the Identification of Bullets Fired From Consecutively Rifled 9 mm Ruger Pistol Barrels. ” Presented at the 34 th Annual AFTE Training Seminar, Philadelphia, PA, May 2003.

Validation Studies and Error Rates Thompson and Wyant, 2004 – ten consecutively manufactured knives, 103 respondents with 10 unknowns each; 8 false identifications for 0. 77%* n Murphy and Bunch, 2003 – 360 cartridge case comparisons from Glock pistols, 0 false identifications+ n *Thompson, E. and Wyant, R. “KIP (Knife Identification Project). ” Presented at the 34 th Annual AFTE Training Seminar, Philadelphia, PA, May 2003. +Bunch, S. and Murphy, D. “A Comprehensive Validity Study for the Forensic Examination of Cartridge Cases. ” AFTE Journal, 35(2), 201 -203.

Validation Studies and Error Rates n Miller and Neel, 2004 – 1000 applications of CMS criteria to 2 dimensional tool marks; no known non -match comparison violated criteria* Miller, J. and Neel, M. “Criteria for Identification of Toolmarks Part III Supporting the Conclusion. ” AFTE Journal, 36(1), 7 -38.

Validation Studies and Error Rates “A better estimation of error rate in casework would be most rigorously achieved by the reexamination of several thousand cases where each case was examined by a panel of experts to achieve consensus. In the absence of a massive increase in funding, this is unlikely to happen. ” * *Gutkowski, S. , “Error Rates in the Identification Sciences, ” The Forensic Bulletin, Summer 2005, 28.

Proficiency Testing and Error Rates n Proficiency Testing – Assessment of n Laboratory practice n Quality assurance n Quality control – Involve tools from a variety of manufacturing methods

Proficiency Testing and Error Rates Proficiency testing and validation studies are not perfect but offer a good picture on how often the profession makes a mistake.

Current Computerized Technology is Misunderstood

Computerized Technology n The contention – “…computerization has not eliminated the risks of misidentifications and missed identifications by firearms as well as toolmark examiners. ” * *Schwartz, p. 28.

Computerized Technology n Current technology, e. g. , IBIS was never intended to address the issues raised by critics n IBIS is an INVESTIGATIVE tool

Judiciary Understands Critical Issues

Judiciary Understands n The charge - “No court, including the two recent courts that have excluded particular identification testimony, has recognized the systemic scientific problems with the field. ” * *Schwartz, p. 32.

Judiciary Understands n The alternative explanation - such issues have been examined by the court and they have concluded that they do not have the significance attributed to them by Schwartz.

Sexton v. State - 2002 Court Recognizes Potential for Subclass

Sexton v. State - 2002 n Foundation of discipline is sound n Application in this particular case was not n Court approached situation correctly

Sexton v. State - 2002 n Identification of magazine marks on cartridge cases – Poor foundation for testimony – Scant references – No application from general to magazines – When asked, expert could not explain manufacturing process for magazines

People v. Hawkins 1995 Court Recognizes Importance of Training and Experience

People v. Hawkins - 1995 n Court faced with non-CMS articulation of identification criteria offered by the prosecution and Biasotti’s CMS work offered by the defense

People v. Hawkins - 1995 n Court correctly – Deemed that Biasotti’s work was not at odds with testimony – Recognized value of training and experience and how Biasotti’s work fit in that scheme

Commonwealth v. Ellis 1974 Court Properly Contends with Differing Identification Criteria and Rate of Change of Tool Surface

Commonwealth v. Ellis 1974 n Case circumstances – No firearm recovered – Bullet from victim compared with bullets from a tree connected to suspect – Two experts n Agreed as to two bullets from tree n Disagreed as to third – One identification – One inconclusive

Commonwealth v. Ellis 1974 n Identification criteria – Differing results not typical but considering condition of evidence, not surprising – Interpretations of observed correspondence is subjective – With comparisons close to known nonmatch condition, differences are expected

Commonwealth v. Ellis 1974 n Identification criteria – Court did adequately assess the limitations of the discipline and appropriately assigned the task of weight to the jury

Commonwealth v. Ellis 1974 n Rate of Change of Tool Surface – Court felt adequately addressed by experts – Court felt it did not present a systemic problem

State v. Fasick – 1928 and State v. Clark -1930 Court is Capable of Assessing Sufficiency of Science

State v. Fasick - 1928 Comparison of tool marks on branches to a knife n Court appeared scientifically inclined understanding that different angles and testing procedures would result in differences in tool marks n Court understood reasonable set of expectations regarding proper testing n

State v. Fasick - 1928 Court recognized that reasonable expectations were not met by expert n Court correctly excluded evidence n

State v. Clark - 1930 n Circumstances similar to Fasick – Same state – Same type of tool – Marks produced on branches n Holding of court completely different than Fasick

State v. Clark - 1930 n Court did not ignore Fasick - “The facts in State v. Fasick…distinguish that case from the case at the bar. In the Fasick Case there was only one mark on the two pictures admitted in evidence which compared one with the other. In the case at the bar there are more than fifty marks appearing on the pictures of the cut surfaces of the fir boughs which can be identified as appearing on the cut surfaces of the

State v. Clark - 1930 n Schwartz agrees with Saks that the distinction was superficial* n Clark case was reported in literature and differences are far from superficial+ *Schwartz, p. 36, n. 163. +May, L. S. “The Identification of Knives, Tools and Instruments A Positive Science. ” American Journal of Police Science, 1930, 246 -259.

State v. Clark - 1930 n Examiner – Designed test cutting instrument n Reproducible n Could vary numerous angles – Identified runs of CMS on photographs – Considered statistics Differences far more than superficial.

Ramirez v. State – 1989, 1995, 2001 Court Questions Infallibility of Firearms and Tool Mark Identification

Ramirez v. State – 1989, 1995, 2001 Court referred to Hart’s method as novel based on claims of infallibility n Court concerned about lack of concrete items to consider as objective criteria n Singly applied to this case n Evidence excluded n

Ramirez v. State – 1989, 1995, 2001 Court erred in that Hart use traditional non-quantitative comparative technique – not novel n Infallibility, if claimed, not related to method but rather issues of error rate n Error rate poorly articulated and nonresponsive n

Ramirez v. State – 1989, 1995, 2001 Court did appear to recognize critical issues and areas of concern n Court felt responses did not adequately address these issues n – Does not mean they could not be – All these issues addressed in presentation – To disqualify evidence discipline wide is not warranted

Conclusions Firearms and tool mark identification is rooted in firm scientific foundations, critically studied according to the precepts of the scientific method culminating in the AFTE Theory of Identification.

Conclusions n If accounted for, concerns of – Differing identification criteria – Potential for subclass characteristics – Changes in tool surface over time Do NOT invalidate identification discipline as a science

Conclusions The firearms and tool mark identification discipline has been validated in a manner appropriate for evidence of the kind to be expected in firearms and tool mark examinations

Conclusions Proficiency tests and error rates have been studied and can provide the court and community with a useful guide as to the frequency with which misidentifications are reported in the community using appropriate methodologies and controls.

Conclusions In those instances in which the discipline and interpretation of results has been well-articulated, the courts have recognized and have an appreciation for the intricacies of firearms and tool mark identification.

Conclusions Critics have called into questions very critical issues such as the potential for subclass characteristics and identification criteria that, if not adequately addressed by an individual examiner, could lead to an incorrect interpretation of the observations made in a particular case. Questioning, as they do, is valuable in probing the sufficiency of knowledge and application on the part of the individual examiner in a particular case. Extending this discipline wide, however, is in definite error.