20072008 SWGDRUG Updates Uncertainty Document Christian C Matchett

2007/2008 SWGDRUG Updates Uncertainty Document Christian C. Matchett, F-ABC Georgia Bureau of Investigation presented at 2008 CLIC Technical Training Seminar

2007/2008 SWGDRUG Updates Uncertainty Document sponsored by the Drug Enforcement Administration Office of Forensic Sciences and the National Institute of Standards and Technology

Why Address Uncertainty? n n Forensic community asking for guidance Accrediting bodies establishing measures of assessing conformity with ISO Customer requirements Jurisdictional requirements n n Transparency (nothing to hide) Potential Exculpatory Information

Core Committee Actions n Jan 2008 – Draft Uncertainty document approved and released for public comment n n n Posted on the website since February 2008 July 2008 – Comments were addressed and final document adopted on July 22, 2008 The document should be posted on the website by the end of September 2008

SWGDRUG Approach n n Tailor the recommendations to drug analysis and answer specific uncertainty questions Ensure dialog on topic is not dictated to us by those outside of the seized drug community Offer guidance and direction to laboratories and accrediting bodies Aware that there is a wealth of information that already exists on Uncertainty n No intentions of repeating existing information

Purpose of Document n Provide General Guidance n n Raise Awareness and Put In Context n n Uncertainty is not doubt, it provides assurance that results and conclusions are fit for purpose Determine Laboratory Responsibility n n Principles and themes conveyed, not step-by-step instructions Consider customer requirements and address uncertainty through training, procedures and documentation State Benefits n n Enhanced confidence through increased understanding of results Provides Mechanism to express reliability of results

Application of Uncertainty n n Qualitative Quantitative n n Purity Weights

Qualitative Analysis n n Individual methods have limitations and, consequently, uncertainty Understanding limitations allows analysts to build an appropriate analytical scheme to correctly ID drugs or chemicals n n It is expected that an appropriate analytical scheme will result in, effectively, no uncertainty in reported identifications Relevant limitations should be documented and may need to be in report

Qualitative Examples n Use Part III B Methods of Analysis/Drug Identification n n IR and microcrystalline test positive for cocaine – effectively NO uncertainty Limitations n n Marquis test positive for methamphetamine – could be methamphetamine or other amphetamines GC/MS test positive for ephedrine – could be ephedrine or pseudoephedrine

Quantitative Analysis n n Uncertainty is defined as an estimate attached to a test result which characterizes the range of values within which the true value is asserted to lie Precise calculations of measurement uncertainty is not always required

Seized Drug Numerical Results n Primary numerical values reported in the analysis of seized drugs are n n Weight and Purity Where a value is critical, an appropriate measurement uncertainty determination shall be applied n n Weight close to a statutory threshold Purity of drug close to level which affects sentencing

Weight n Uncertainty of a reported value is dependant on the weighing process. Factors include: n n n n Single item versus multiple items (# of weighing operations) Tare function as separate weighing operation Extrapolation of population weight from limited sampling of multiple items Aggregate weighings Incomplete recovery of material from packaging Balance selection (e. g. , readability, capacity) Balance operation (e. g. , sample placement, environmental conditions)

Purity n Sources of uncertainty for purity determination n n Sampling plan (e. g. , handling of multiple exhibits) n Sample homogeneity Analytical method n n n Sample preparation (e. g. , size, matrix effects, solubility) Analytical technique Reference material (e. g. , purity of standard) Equipment and instrumentation performance (e. g. , glassware, pipetters, balances, chromatographs) Concentration of analyte Environmental conditions

Purity Approaches n Analytical Error n n Sampling Error n n n Address both systematic and random error through method validation and quality assurance The sample and sampling procedure are often the greatest contributors to measurement uncertainty Where appropriate, confidence levels (e. g. , 95%, 99. 7%) shall be selected based on considerations relevant to the analytical context Record uncertainty information in validation documents and/or case records

Uncertainty Budget n n All sources of error are separately identified and tabulated Assign values to each error source using n n n Empirical data n Validation process, Historical performance data, Control chart data, proficiency tests Published data Combination of empirical and published data Can exclude insignificant sources Calculate combined and expanded uncertainty using significant values for procedure

Non-Budget Approach n Example 1: Use of data from replicate analyses from a validated method with an appropriate sampling plan n n Sources of uncertainty that are separately assessed in the budget method are collectively assessed by experimental measurements Example 2: Use of two standard deviations (2σ) of the test method results from reproducibility data from the validation studies. n Provides an approximation of the measurement uncertainty for non-critical values

Reporting n Uncertainty shall be documented but may not need to be reported n n Should be reported when result impacts customer Even if not reported, analysts shall be cognizant of the uncertainty associated with their results

When to Report Uncertainty? n Jurisdictional n n n Analytical n n n Prevailing statutory requirement Relevant governing body (agency) requirements Customer requests Potential exculpatory value Qualitative results where limitations are known (e. g. , inability to differentiate isomers) Quantitative measurements where a value is critical (e. g. , weight or purity level close to statutory threshold) Laboratory accreditation requirements

Qualitative Reporting Examples n n n Contains Ephedrine or Pseudoephedrine Item tested: 5. 2 grams net Visual examination determined that the physical characteristics are consistent with a Schedule IV pharmaceutical preparation containing Diazepam. There was no apparent tampering of the dosage unit and no further tests are being conducted. Contains cocaine (salt form not determined)

Quantitative Reporting Examples n n Positive for cocaine in the sample tested Net weight of total sample: 5. 23 grams ± 0. 03 grams Quantitation: 54. 7% ± 2. 8% Sample tested positive for cocaine Net weight: 5. 23 grams Purity: 54. 7% Confidence Range: ± 2. 8%* Calculated net weight of drug: 2. 8 grams of cocaine *Confidence range refers to a 95% confidence level

Training Recommendations n Individuals responsible for determining, evaluating and documenting uncertainty shall be capable of demonstrating familiarity with foundational concepts and principles of estimating uncertainty n n n General metrology (terminology, symbols, etc. ) Concepts of random and systematic error, accuracy, precision, propagation of error, etc. Reporting conventions (sig. figs, truncating, rounding) Basic statistics (i. e. , confidence interval, probability, etc) All analysts shall be capable of explaining their labs procedures for evaluating uncertainty of qualitative and quantitative analyses

Supplemental Documents to Follow n n n Uncertainty budget Control chart data applications Demonstration of balance control using standard weight sets Summing weights from individual exhibits Expression of sampling uncertainty based on confidence interval using multiple samplings

Core Committee • DEA – Nelson Santos (Chair) • Secretariat – Scott Oulton (non -voting) • FBI - Eileen Waninger • ASCLD – Garth Glassburg • NIST - Susan Ballou • ASTM and NEAFS- Jack Mario

Core Committee • CAC & NWAFS - Jerry Massetti • MAFS - Richard Paulas • MAAFS - Linda Jackson • SAFS – Christian Matchett • Toxicology – Dr. Robert Powers • Educator – Dr. Chris Tindall • Educator – Dr. Suzanne Bell

Core Committee • Canada - Richard Laing • Japan – Mr. Osamu Ohtsuru • United Kingdom - Dr. Sylvia Burns • Australia - Catherine Quinn • Germany - Dr. Udo Zerell • ENFSI - Dr. Michael Bovens • UNODC - Dr. Iphigenia Naidis

THANK YOU Visit us at www. swgdrug. org