Method Validation and Verification An Overview Patricia Hanson

Method Validation and Verification: An Overview Patricia Hanson, Biological Administrator I Florida Department of Agriculture and Consumer Services, Food Safety Microbiology Laboratory

Introduction • 17 years in the microbiology section of the Florida Department of Agriculture and Consumer Services, Food Laboratory • Laboratory moved from primarily cultural methods in 1998 to a leader in the implementation of new technology in 2015 • Numerous partnerships with state, federal and industrial customers to validate and implement new methods

Goals • Distinguish between method verification and method validation • Based on our laboratory’s experience, identify the critical components of both a method verification and a method validation • Offer insight to approaches we have found useful, specifically in the microbiology section • Share lessons learned

Verification vs Validation • Verification is a demonstration that your laboratory can perform a Standard Method or other well documented method and produce acceptable results • Validation is both a demonstration that your lab can run the method and that the method or change to a method is fit for purpose

Aspects of a Verification • Run entire method, start to finish, exactly as it is written • Include both positive and negative samples • Represent all matrix types • Be spiked at a level that is fit for the purpose of the method if spiked samples are used

Verifications can be ongoing…. • For the purpose of this discussion, “Verification” is being used to describe what the laboratory does prior to implementing a method in the laboratory • The term “Verification” is often also used to describe the ongoing process of verifying a method with each run – in our lab this it the use of Process Controls, Cultural Controls, and Media Controls

Validation • Laboratory developed method • Matrix Extensions • ANY technically significant change to a Standard Method or previously validated method

Overview of an Existing Guideline • Level One: Urgent Use, Single Lab Validation • Level Two: Single Lab Validation • Level Three: Multi-Lab Validation • Level Four: Full Collaborative Study

Level 2: Single Lab Validation Highlights • 50 different strains of target organism, if available • 30 different strains of “nearest neighbor” organisms • At least two target analyte levels • 20 replicates per food type per analyte level • Perform side by side if possible

Aspect of a Verification • Be extensive enough to show that the method is fit for the intended use and meets the customer’s needs

What is “Extensive Enough” Items to be considered: Ø Matrix: number and types Ø Target: source and levels Ø Method Evaluation: side by side or use of standard reference material When considering these items during the development of the validation, keep in mind what is needed to show the method is fit for the intended use and meets the customer needs.

Matrices • Validation or verification needs to be performed on each food type • Meat (raw, smoked, cooked, etc) • Vegetable (whole, cut, sprout, etc) • Dairy (liquid, solid, etc) • Other (spices, candies, etc)

Replicates • • Exact number depends on method Verification, normally triplicate Validation, normally six replicates per level Important to do the same number of replicates of “un-spiked”

Naturally Contaminated Samples • Naturally contaminated pros and cons • Best test of the method • Level of analyte is unknown • Little or no control of analyte level

Spiked Samples • Spiked sample pros and cons • Ability to control organism level • Methods often perform differently on spiked samples than on naturally contaminated samples

Aging of Spiked Cultures • Spiking samples 48 -72 hours prior to initiation of testing can better represent naturally occurring contamination • Sometimes the cells are so stressed, they die off • Consider the method or part of method being validated and type of validation

Selection of Organisms • Select at least one organism for each target • “Nearest Neighbor” organisms – not often used but good to consider

“Side by Side” Validations • Normally involves running the method being validated side by side with a Standard or established method • Twice the work but very valuable for determining if the “new” method is equivalent or better than the Standard or established method • Best choice when changing part of a method – for example a new enrichment broth but same screening method

How Much Confirmation? • Screening Method Validations: Confirm everything • You can get by with an abbreviated confirmation, for example just biochemical panel and/or serology but do enough to show you “got out what you put in. ”

Why Do So Many Confirmations? • Ensure you can confirm samples that screen positive • Ensure samples you can confirm screen positive • Spiked samples that don’t screen positive and don’t confirm point to limitations of the method or problems with the spiking procedure

Data Evaluation • The measures used to evaluate data depend on the purpose and type of the method and the needs of the customer Acc n o ura i t cy a t i t n a u Range Q of t i m e i t a R L e Precision iv t i s o P False

Qualitative Data Evaluation • Detection Limits – can the method detect the target at the levels required by the customer? • Specificity – can the method detect the target in the matrix? • Selectivity – can the method differentiate between the target and other analytes in the matrix? • Robustness – can the method be repeated on the sample with the same results?

Quantitative Data Evaluation • Detection Limits, Specificity, Selectivity, and Robustness • Precision – Does the method produce the same results as established method? • Linearity – Does the method produce proportional results with in the customer’s target range?

Personnel Considerations • Personnel designing and approving verifications and validations should be knowledgeable about the technology as well as the needs of the lab

Personnel Considerations • Personnel carrying out verification and validations should be competent in all techniques • Work on verification and validation studies can be used to demonstrate competency but should not be used as training

Pit-falls of Method Validation • Repeating tests until you get the result you want • Drifting from method validation into method development • Adjusting criteria for acceptance AFTER the data is obtained

Overcoming Pit-falls • Have a well thought out, documented and approved plan and stick to it • Go back and start over, incorporating any changes into new validation • All data should still be included in evaluation

Documentation • • • Approval of plan prior to running Raw Data Interpretation of raw data Final approval from customer Reference validation in SOP If SOP is a modified Standard Method, list deviations from the Standard Method in a section in the SOP

References • U. S. Department of Health and Human Services, Food and Drug Administration (2014, August 29). Volume II – Methods, Method Verification and Validation ORALAB. 5. 4. 5. Retrieved from http: //www. fda. gov/Science. Research/Field. Science/ucm 1 71877. htm • U. S. Department of Health and Human Services, Food and Drug Administration (2011, September 8). Guidelines for the Validation of Analytical Methods for the Detection of Microbial Pathogens in Foods. Retrieved from http: //www. fda. gov/downloads/Science. Research/Field. Sc ience/UCM 273418. pdf

Questions