Gynecologic Consensus Conference Work Group 3 PAP Proficiency

Gynecologic Consensus Conference Work Group 3: PAP Proficiency Testing, General Quality Practices, and Workload cap. org June 4, 2011 v. 1

Working Group 3 • Joseph Tworek, MD (Senior Author) • Lydia P. Howell, MD (Chair) • Ritu Nayar, MD • Sana O. Tabbara, MD • Barbara Winkler, MD • Lynnette Savaloja, SCT • Nicole E. Thomas, MPH, CT(ASCP), (CAP Staff) © 2011 College of American Pathologists. All rights reserved. 2

PT History • First mandated in Clinical Laboratory Improvement Amendments of 1988 Final Rule (42 CFR Part 405). Fed Reg 1992, 57: 7001 -7186. (CLIA 88); 100 -578; Oct. 31, 1988 • Not implemented until almost two decades later. © 2011 College of American Pathologists. All rights reserved. 3

Today • Despite initial controversies: o Positive correlation between PT and work performance o Failure is uncommon − 99. 6% passed after 3 tests (2006 CAP PT) − Initial failure rate: 5% CT, 16% 1 o screening path, 6% 2 o screening path − First time failure rate has plateaued at 3% for past 3 years − Keenlyside et al. Am J Clin Pathol 1999; 112: 769 -776. − Moriarty A et al. Arch Pathol Lab Med 2009; 133: 1757 -1760. © 2011 College of American Pathologists. All rights reserved. 4

PAP PT Consensus Statements

Statement #1 • The laboratory should have a written policy that the director (and/or designee) actively monitors results of gynecologic cytology proficiency testing.

Justification • Required by Federal Regulation (CLIA ‘ 88) • Survey findings: 84. 2% of labs currently follow this practice

Vote#31 – Statement 1 • The laboratory should have a written policy that the director (and/or designee) actively monitors results of gynecologic cytology proficiency testing. • Do you agree with this statement? o A – Yes 86. 36% o B – No 12. 12% o C – Other 1. 52% © 2011 College of American Pathologists. All rights reserved. 8

Statement #2 • Pap PT pass-fail rates should be monitored globally for the laboratory and by individual practitioner (ie, CT and Path).

Justification • Survey: o 94. 7% of labs monitor pass-fail rate − Individual monitoring: CT= 84. 7%, Path = 83. 7% − Lab-wide monitoring: 51% o Expert opinion: Both help find outliers − Lab-wide data may be a baseline to find individual outliers − Monitoring individuals may facilitate identifying outliers © 2011 College of American Pathologists. All rights reserved. 10

Vote #32 – Statement 2 • Pap PT pass-fail rates should be monitored globally for the laboratory and by individual practitioner (ie, CT and Path). • Do you agree with this statement? o A – Yes o B – No o C – Other © 2011 College of American Pathologists. All rights reserved. 66. 67% 31. 82% 1. 52% 11

Statement #3 • Pap PT slides should be prescreened by CTs for pathologists who are secondary screeners. • Justification o Survey findings: 93. 9% = yes o Regulation and expert opinion: Test environment should reflect normal lab practice. © 2011 College of American Pathologists. All rights reserved. 12

Vote#33 – Statement 3 • PAP PT slides should be prescreened by CTs for pathologists who are secondary screeners. • Do you agree with this statement? o A – Yes o B – No 89. 06% 10. 94% © 2011 College of American Pathologists. All rights reserved. 13

Statement #4 • For a first-time PT failure (CT and Path): o Re-enrollment for re-testing is a CLIA requirement and is sufficient. o No other remedial actions required, unless supported by other performance indicators.

Justification • Survey: o Re-enroll: 83% CT, 86% Path o Options for remedial action did not exceed 13% for CTs or 9% for Paths o Labs may recognize that a single failure is not a significant finding © 2011 College of American Pathologists. All rights reserved. 15

Justification, con’t • Testing alters performance (anxiety? ) • Almost everyone passes eventually: o 99. 6% passed after 3 tests (2006 CAP PT) − Hughes J et al. Arch Pathol Lab Med 2009; 133: 279 -282. − Moriarty A et al. Arch Pathol Lab Med 2009; 133: 1757 -1760. © 2011 College of American Pathologists. All rights reserved. 16

Vote#34 – Statement 4 • For a first-time PT failure (CT and Path): o Re-enrollment for re-testing is sufficient. o No other interventions are required, unless supported by other performance indicators. • Do you agree with this statement? o A – Yes o B – No 94. 44% 5. 56% © 2011 College of American Pathologists. All rights reserved. 17

Statement #5 • Remedial action policy should not be applied for a passed but non-perfect test (ie, score <100%), even for multiple non-perfect test scores. • Justification o Survey: − 81. 4% report no policy to do so − Even in 2 consecutive non-perfect exams, only 5. 2% take remedial action o No literature to show that a non-perfect test is a significant finding. © 2011 College of American Pathologists. All rights reserved. 18

Vote#35 – Statement 5 • Remedial action policy should not be applied for a passed but nonperfect test (ie, score <100%), even for multiple non-perfect test scores. • Do you agree with this statement? o A – Yes o B – No 93. 75% 6. 25% © 2011 College of American Pathologists. All rights reserved. 19

Statement #6 • Monitoring of incorrect slide diagnoses on passed PT tests: o Discouraged from inclusion in lab PT policy o No interventions for this test finding are necessary • Justification o Survey findings ―This type of monitoring only done by 26% of labs. o Literature ―Recognition that test-taking alters performance (as in statement #3). ―No literature to show that incorrect results are a significant finding.

Vote#36 – Statement 6 • Monitoring of incorrect slide diagnoses on passed PT tests: o Discouraged from inclusion in lab PT policy o No interventions for this test finding are necessary • Do you agree with this statement? o A – Yes o B – No 76. 56% 23. 44% © 2011 College of American Pathologists. All rights reserved. 21

Additional Voting Questions Working Group 3 © 2011 College of American Pathologists. All rights reserved. 22

Voting WG 3 - PT 70. For labs performing more than one slide methodology, each methodology should be tested during PT. A. Strongly agree 14. 29% B. Agree 21. 43% C. Disagree 8. 93% D. Strongly disagree 41. 07% E. Unsure 10. 71% F. Other 3. 57% © 2011 College of American Pathologists. All rights reserved. 23

Voting WG 3 - PT 71. There is room for improvement in PT testing methods as a monitor, especially in light of new screening technologies. Do you agree? A. Yes 82. 76% B. No 15. 52% C. Other 1. 72% © 2011 College of American Pathologists. All rights reserved. 24