Introduction to the basic principles of quality assurance

Introduction to the basic principles of quality assurance (QA) and quality control (QC) in untargeted metabolomics Prof. Warwick (Rick) Dunn and m. QACC friends School of Biosciences, Phenome Centre Birmingham and Institute of Metabolism and Systems Research, University of Birmingham, UK Email: w. dunn@bham. ac. uk Twitter: @rickdunnblade

The next 20 minutes o What is quality assurance in metabolomics? – definition and processes o What is quality control in metabolomics? – definition and processes o Use of QC samples in untargeted metabolomics

What is Quality Assurance (QA) and Quality Control (QC)?

Question 1: which of the following statements about quality assurance (QA) and quality control (QC) is true? QA and QC is performed before data is collected 2. QA and QC is performed after data is collected 3. QA is performed before data is collected 4. QA is performed after data is collected 1.

What is Quality Assurance (QA) and Quality Control (QC)? q Assurance: The act of giving confidence, the act of making certain q Quality Assurance: The planned and systematic activities implemented in a quality system so that quality requirements for a service will be fulfilled q Control: An evaluation to indicate needed corrective responses q Quality Control: The observation techniques and activities used to evaluate and report quality Quality Assurance occurs before data is collected q Quality Control occurs during and after data is collected q http: //asq. org/learn-about-quality/quality-assurancequality-control/overview. html

QA: Some important points q The value of chemical measurements depends upon the level of confidence that can be placed in the results q Increasingly, the chemical testing community is adopting QA principles which, whilst not actually guaranteeing the quality of the data produced, increases the likelihood of it being soundly based and fit for its intended purpose q The business of QA is to manage the frequency of quality failures - the greater the effort taken, the smaller the number of quality failures that can be expected q However, it is important for both laboratories and their customers to realise that QA cannot guarantee that 100% of the individual results will be reliable. http: //asq. org/learn-about-quality/quality-assurancequality-control/overview. html

Some important Quality Assurance Procedures q q q q q A quality management system Suitable laboratory environment Educated, trained and skilled staff Training procedures and records Equipment suitably maintained and calibrated Documented and validated methods Checking and reporting procedures Preventative and corrective actions Proficiency testing Requirements for reagents, calibrants, measurement standards & reference materials http: //asq. org/learn-about-quality/quality-assurancequality-control/overview. html

Important Quality Control Processes q q q q Analysis of reference materials/measurement standards Analysis of blind samples Use of quality control samples & control charts Analysis of blanks Analysis of spiked samples Analysis in duplicate Proficiency testing http: //asq. org/learn-about-quality/quality-assurancequality-control/overview. html

What is ‘fit-for-purpose’ in targeted and untargeted assays?

‘Fit-for-purpose’ for a targeted assay q The objective of a targeted assay is to calculate the concentration of a metabolite(s) with high quality and confidence The Food and Drug Administration (FDA) have defined this q Guidance for Industry – Bioanalytical Method Validation q http: //www. fda. gov/downloads/drugs/guidancecompliancereg ulatoryinformation/guidances/ucm 368107. pdf q q A validated method will demonstrate q selectivity q accuracy, precision and recovery q calibration curve linearity q sensitivity q stability

‘Fit-for-purpose’ for an untargeted assay The objective of an untargeted metabolomics assay is to provide an unbiased snapshot of metabolism in the system studied q Here ‘fit-for-purpose’ for a targeted assay does not match the objective q This is fine! q We can define what ‘fit-for-purpose’ for the analytical method is in these cases q q ‘Fit-for-purpose is ensuring the analytical method fulfills a certain criteria and (in my view) performs this applying validated methods which show reproducibility and repeatability q You can still validate these methods to show they are ‘fit-forpurpose’, that they give the same results every time within acceptance criteria q Broad coverage of detectable metabolites (high sensitivity and low selectivity) q High precision (reproducibility and repeatability)

QC samples: providing evidence for data quality in targeted and untargeted studies

QC samples in targeted metabolomics q Quality Control samples are applied in each analytical batch q low, medium and high concentration QC samples analysed multiple times q Acceptance criteria are set for QC samples q FDA has provided guidelines as to what is acceptable q QC data assessed to ensure the acceptance criteria are met for the analytical batch q If the acceptance criteria are met then proceed with data analysis q If the acceptance criteria are not met then the data should be reacquired

QC samples in untargeted metabolomics q Quality Control samples are applied in each analytical batch q typically a single pooled QC sample is analysed multiple times q Acceptance criteria are set for QC samples q FDA has not provided guidelines as to what is acceptable q different groups apply different criteria (which is OK as long it it is clearly recorded and reported) q e. g. some groups use RSD<20% and some use RSD<30% q QC data assessed to ensure the acceptance criteria are met for the single analytical batch q If the acceptance criteria are met then proceed with data analysis q If the acceptance criteria are not met then the data should be re-acquired

Question 2: How many different types of QC sample can we apply in an untargeted metabolomics study? 1. 2. 3. 4. 5. One Two Three Four Five

What types of QC samples are applied in untargeted metabolomics? q q q System suitability blank Process blank Pooled QC sample (intra-study sample) Reference materials (RMs), standard reference materials (SRMs) and long term reference (LTR) samples – intra-study, interstudy, intra-laboratory, interlaboratory) Process internal standards Broadhurst, D. , et al. Metabolomics, 14(6), p. 72.

What are pooled QC samples in untargeted metabolomics? q Quality Control (QC) samples are a single (or set) of samples representative of the qualitative and quantitative composition of the study samples being analysed q ideally they are an average of the composition of all samples studied q they contain all of the metabolites and matrix components expected to be present in study samples q ‘theoretically’ a biologically identical sample intermittently analysed during analytical experiments q replicate injections of the sample so should provide exactly the same data for each sample injected q in reality, we observe variation and is representative of deviation in sample preparation, data acquisition or data pre-processing (analyst or instrument-introduced deviation) q Early origins derive from Ian Wilson and Jan van der Greef Broadhurst, D. , et al. Metabolomics, 14(6), p. 72.

Assessing data quality in untargeted metabolomics Assess visually and quickly applying PCA q Assess quantitatively for EACH metabolite feature q calculate the % detection rate (all samples, each class, QC only? ) q calculate the relative standard deviation for QC samples (RSD) q calculate the % blank contribution q Apply this test to all metabolite features detected individually q features with RSD greater than limit are removed (? ) q features with % detection rate lower than limit removed (70%) q features with % blank contribution greater than limit are removed (5 or 10%) q In authors experience 10 -30% of features removed q

Question 3: What should be the acceptance criteria for RSD in quality filtering? 1. 2. 3. 4. 5. 15% 20% 30% 50% Other

Summary Quality Assurance: The planned and systematic activities implemented in a quality system so that quality requirements for a service will be fulfilled q occurs before data is collected q e. g. training, SOPs and preventative maintenance q Quality Control: The observation techniques and activities used to evaluate and report quality q occurs during and after data is collected q e. g. system suitability samples, pooled QC samples, reference materials q q Ensure assays are fit-for-purpose q Use of a range of QC samples is appropriate q System suitability blank q Process blank q Pooled QC sample q Reference materials (RMs), standard reference materials (SRMs) and long term reference (LTR) q Process internal standards