So how good are your results An introduction
- Slides: 38
So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincent’s Hospital, Sydney
Overview • Standard QC • Quantitative QC – How good are we? – How good do we need to be? • Bias • A short course in what we will need to know
Quality System • Staff – Choosing – Training • Instruments – Choosing – Optimising – Maintaining • QC – Planning – Performing – Responding • Quality Assurance – Performance – Interpretation – Action • Sample management • Result management
Quality System • Staff – Choosing – Training • Instruments – Choosing – Optimising – Maintaining • QC – Planning – Performing – Responding • Quality Assurance – Performance – Interpretation – Action • Sample management • Result management
Quality Terminology • QA - Quality Assurance – Planned, systematic actions providing confidence that a quality output will be produced – Laboratory Procedures • QC - Quality Control (QC) – Procedures use to assess validity of results in real time, controls release of results – QC material run with patient samples • EQA - External Quality Assessment (PT) – Procedures operated by an external agency which allow retrospective review of performance – RCPA-AACB
Running a QC Program • • Selection of material (matrix) Selection of levels (decision points) Setting of targets and ranges Decision on frequency (batch vs RA) Decision on number of QC samples (n) Decision on rules and interpretation Response to out-of-range values • Quality planning
QC Quiz Your new trainee scientist asks you the following question: “In our lab, how far can the results of an assay vary from the actual concentration in the sample? ” What is the answer: +/- 1 SD; 2 SD; 3 SD; 4 SD; 5 SD ?
Assay Characteristics • • Stable assays: Performance defined by mean and SD QC never fails Results “always” within +/- 2 SD
Stable Assay Mean = 20, SD = 1 95% of QC results between 18 and 22. Interpretation: A result of 20 has a 95% confidence interval of 18 to 22.
Unstable Assays • Mean drifts over time (fluctuating bias) • QC process used to detect drifts • Variation in results due to scatter plus drift
Unstable Assay Mean = 20, SD = 1, Plus fluctuating mean. Interpretation: Result of 20 has 95% confidence limit of 18 – 22, PLUS bias at time of assay.
Unstable assays • How bad can this be? • How can we measure this?
Original Westgard Multi-rules
Original Westgard Multi-rules
Original Westgard Multi-rules What does “In Control” mean?
www. westgard. com/
Probability of Rules Firing Power Function Chart N=2 13 s/22 s/R 4 s Shift in Mean (multiples of SD)
Power Function Charts
Probability of Rules Firing Power Function Chart N=2 13 s/22 s/R 4 s Shift in Mean (multiples of SD) 90% error detection at 3. 2 x SD
Shifts and Results (unstable assay) • Imprecision: up to 2 SD. • Undetected shifts in mean: 3 SD • Total spread: up to 5 SD With the assay still “in control”! +3 SD +2 SD +5 SD
QC Quiz • The result may differ from the correct result by addition of: – random variation of the assay (up to 2 x SD) – the undetected bias at the time of the assay (up to 3 x SD). • Example – CV cholesterol assay: 2. 0% – At 6 mmol/L, 5 x SD = 0. 6 mmol/L • Accumulation of errors all in the same direction is rare, but can happen.
Understanding our assays • For any assay, with the QC protocol in place, we should be able to say how much analytical error may occur. “These rules have the power to cause a STOP 90% of the occasions when there is a shift in the assay of 2. 8 x LSD and cause a PAUSE 90% of the occasions when there is a shift in the assay of 2. 6 x LSD. ” - Syd. Path Quality Control SOP
Westgard - Quantifying QC • With the rules I have in place, what shifts in assay performance can I detect. or • How can I be sure that I can detect important changes – Capability – Setting QC protocols – What are “important changes”
Capability • Capable assays are easily able to “do their job” • Capable assays (almost) never produce results outside important limits. • Poorly capable assays will produce results outside the set limits. Capable assay Incapable assay
Capability • Good assays (capable) have an analytical performance (SD) which is much less than the clinically important change. • This can be quantified as the Capability Index: Cp=ALP/SD (ALP=Allowable limit of Performance) >6 great; 4 -6 OK; <4 poor
Capability Required shift detection Cp = 6, good Cp = 4, OK 4 SD Shift 2 SD spread 2 SD Shift Cp = 3, poor 1 SD Shift Important Change
Capability • Capability is the concept we use to discuss quality of assay performance. • Relates assay precision to required precision.
Reverse Engineering QC • If we have limits to our assay performance we want QC protocols which allow us to detect assay problems before “wrong” results may be issued. • Choose QC protocols which allow appropriate error detection. • Use Power Function Charts….
Power Function Graph (n=2) 12 s 12. 5 s MR 13. 5 s
Setting QC Protocols • Capable assays – simple rules • Poorly capable assays – Need: – More rules – More QC • Incapable assays – will not achieve target performance – Have to accept less chance of finding shifts – Or choose better assay
Quality Specifications Hierarchy How good do we need to be: 1. Proven analyte-specific data on clinical decision making 2. General-clinical decision making – Based on biological variation – Based on medical opinions 3. Professional recommendations 4. Regulations or EQA targets 5. Published state-of-the-art data
Within-subject biological variation • Variability in patient results due to changes in the patient. • www. westgard. com – Archives – Reference Material (near bottom on right) – Biological Variation Database • www. westgard. com/biodatabase 1. htm • eg Sodium: 0. 7%; ALT 24. 3%
Analytical v Biological CV CVa/CVb 2 1 0. 5 0. 25 0. 125 1. 01 1. 0 Relative total CV 2. 2 1. 4 1. 08
CVa / CVb Percent increase in CV
Precision Targets • Within-person biological variation provides limits to benefits of improved assay precision • Many assays clearly capable (optimal) • Some assays not able to reach targets
Bias • Most discussion thus far has related to precision • Attention to bias will be the next major issue
Summary (1) • Some aspects of QC can be quantified • Power function charts are a tool • This process allows us to use appropriate QC • can either – Know how good we are or – Aim to reach certain targets
Summary (2) • Targets can come from various sources • Within-person biological variation provides a useful reference point. • Bias will need to be addressed • Control of bias will provide many advantages
- Insidan region jh
- Good thoughts, good words, good deeds
- Hii good afternoon
- Good afternon animado
- If you are
- Buenas tardes
- You are good you are good and your love endures
- Prayer for success in exams
- A good or helpful result or effect in research
- Give us your hungry your tired your poor
- Introduction, method, result, and discussion
- Good morning lady and gentlemen
- Excellent very good good fair poor scale
- Real and apparent goods examples
- Good morning, class!
- Good food good life
- Ethics is a personal and individual affair
- Positive consumption externality
- Looking good feeling good
- He who finds a wife finds a good thing
- Good morning message to my lecturer
- Good morning and welcome to vietcombank
- Let's be good friends
- Law is order and good law is good order
- Hello friend! i am glad to see you!
- Good communication occurs when the
- It is a good condition that good communication occurs
- Mips instruction set architecture
- Look good feel good quote
- Good charlotte the young and the hopeless
- War good good feeling
- Good morning good lookin
- Real and apparent good
- Good morning
- Good morning,students
- Gentlemen good night ladies good morning
- A good that replaces another demanded good
- A good that replaces another demanded good
- Qualities of a good paragraph