Improving Procedural Teaching by Understanding Features Associated with

Improving Procedural Teaching by Understanding Features Associated with Cognitive Load Justin L. Sewell MD, MPH Christy K. Boscardin, Ph. D John Q. Young, MD, MPP Olle ten Cate, Ph. D Patricia S. O’Sullivan, Ed. D

Procedural skills training is a balancing act! Learners Teachers, patients, system

Learners Case volume e Deliberate practic Feedback Autonomy

Case volume Teachers, patients, system Safety Quality Efficiency Patient-provider relationship

Learners Teachers, patients, system Case volume Safety Deliberate practice Quality Feedback Efficiency Autonomy Patient-provider relationship

Procedural skills training in HPE • Complexity of cognitive and psychomotor demands high risk of cognitive overload • HOWEVER, empirical evidence for contributors to cognitive load during procedural skills training is limited • Better understanding of such contributors could lead to: Curricular innovations Improved learning Better patient outcomes? ? ?

Cognitive Load Theory Capacity for sensory input ∞ Sweller J. Cogn Sci 1988. Figure adapted from Young JQ. Med Teach 2014. Working memory Long-term memory ∞

Types of cognitive load Optimize Minimize Maximize Intrinsic load Task complexity Learner knowledge and experience Extraneous load Learning environment Instructional design Germane load Learner effort and metacognitive skills

Cognitive load & working memory Working memory overloaded – no space for germane load Extraneous load Intrinsic load Space available for germane load Extraneous load Adapted from Young JQ. Med Teach 2014. Intrinsic Germane load

Colonoscopy • Complex and requires >300 cases to attain competence • Preferred method for colorectal cancer screening • “Middle of the road procedure” implications for spectrum of procedural skills

What affects cognitive load during procedural skills training?

Methods • Post-colonoscopy survey distributed to 1, 061 fellows in 2014 -15 academic year • Outcome: three cognitive load types • Predictors: procedural learning features • Anticipated relationships predicted between each feature and each type of cognitive load • Multivariable linear regression model developed for each type of cognitive load

Outcome: cognitive load types Cognitive Load Inventory for Colonoscopy (CLIC) Intrinsic items Extraneous items Germane items Estimate of IL, EL, GL Sewell JL et al. Med Educ 2016; 50(6): 682 -92, AERA 2016 • Psychometric instrument • Multiple items averaged to estimate degree of intrinsic, extraneous and germane load • Validity evidence presented

Predictors • • Year in training Prior experience Sleep Fatigue Supervisor • Junior versus senior • Engagement • Confidence Learner Cognitive load Also supervisor takeover Setting • • • Prep quality Tolerance Anesthesia Gender No. of maneuvers Patient/ task • • Queue order No. people in room On call Paged

Results – response rate • 477 (45. 0%) of 1, 061 invited fellows participated • 154 (95. 1%) of 162 programs represented

Number of fellows per year in training 1 st-year 2 nd-year 3 rd-year 4 th-year 200 179 180 174 160 140 120 110 100 80 60 40 20 0 12

Prior colonoscopy experience <50 51 -100 101 -150 151 -200 201 -250 251 -300 >300 250 203 200 150 100 50 50 25 0 41 49 55 52

Intrinsic load model Feature Category Coefficient (95% CI) β-coefficient P-value Year 2 fellow (vs year 1) Learner -0. 60 (-1. 09, -0. 14) -0. 17 0. 01 Year 3 or 4 fellow (vs year 1) Learner -0. 82 (-1. 38, -0. 26) -0. 23 0. 004 Prior colonoscopy experience Learner -0. 20 (-0. 31, -0. 09) -0. 24 <0. 001 Fatigue Learner 0. 01 (0. 008, 0. 02) 0. 21 <0. 001 Tolerated procedure well Patient/task -0. 67 (-1. 01 -0. 33) -0. 16 <0. 001 No. ancillary maneuvers Patient/task 0. 31 (0. 18, 0. 44) 0. 19 <0. 001 Supervisor took over Multiple 0. 51 (0. 24, 0. 77) 0. 16 <0. 001 Hours of sleep Learner 0. 11 (-0. 07, 0. 29) 0. 05 0. 22 Good bowel prep (vs excellent) Patient/task 0. 14 (-0. 17, 0. 44) 0. 04 0. 38 Fair or poor bowel prep (vs excellent) Patient/task 0. 17 (-0. 26, 0. 59) 0. 04 0. 44

Extraneous load model Feature Category Coefficient (95% CI) β-coefficient P-value Fatigue Learner 0. 01 (0. 005, 0. 02) 0. 18 0. 001 Queue order Setting 0. 07 (0. 01, 0. 13) 0. 10 0. 03 Somewhat or very disengaged (vs very engaged) Supervisor 0. 33 (0. 02, 0. 64) 0. 11 0. 04 Less than very confident Supervisor 0. 49 (0. 13, 0. 86) 0. 13 0. 009 Supervisor took over Multiple 0. 86 (0. 49, 1. 23) 0. 24 <0. 001 Year 2 fellow (vs year 1) Learner -0. 36 (-0. 84, 0. 12) -0. 12 0. 14 Year 3 or 4 fellow (vs year 1) Learner -0. 32 (-0. 89, 0. 26) -0. 10 0. 28 Prior colonoscopy experience Learner -0. 04 (-0. 16, 0. 07) -0. 06 0. 46 Hours of sleep Learner 0. 17 (-0. 01, 0. 35) 0. 10 0. 07 No. people in room Setting 0. 04 (-0. 09, 0. 18) 0. 03 0. 52 Paged Setting 0. 16 (-0. 19, 0. 51) 0. 04 0. 37 On call Setting -0. 23 (-0. 68, 0. 21) -0. 05 0. 30 Somewhat engaged (vs very engaged) Supervisor 0. 45 (-0. 03, 0. 94) 0. 10 0. 07 Neither engaged nor disengaged (vs very engaged) Supervisor 0. 15 (-0. 34, 0. 64) 0. 03 0. 55

Germane load model Feature Category Coefficient (95% CI) β-coefficient P-value Somewhat engaged (vs very engaged) Supervisor -0. 91 (-1. 83, 0. 001) -0. 10 0. 05 Neither engaged nor disengaged (vs very engaged) Supervisor -1. 33 (-2. 24, -0. 42) -0. 15 0. 004 Somewhat or very disengaged (vs very engaged) Supervisor -0. 85 (-1. 44, -0. 26) -0. 15 0. 005 Year 2 fellow (vs year 1) Learner -0. 30 (-1. 19, 0. 60) -0. 05 0. 52 Year 3 or 4 fellow (vs year 1) Learner -0. 62 (-1. 70, 0. 46) -0. 11 0. 26 Prior colonoscopy experience Learner -0. 14 (-0. 36, 0. 07) -0. 10 0. 19 Fatigue Learner 0. 01 (-0. 005, 0. 02) 0. 06 0. 26 No. maneuvers performed Patient/task -0. 06 (-0. 32, 0. 19) -0. 02 0. 62 Paged Setting 0. 28 (-0. 36, 0. 98) 0. 04 0. 39 On call Setting -0. 25 (-1. 08, 0. 58) -0. 03 0. 55 Queue order Setting 0. 03 (-0. 09, 0. 14) 0. 02 0. 65 Less than very confident Supervisor 0. 58 (-0. 10, 1. 27) 0. 08 0. 10 Supervisor took over Multiple -0. 43 (-1. 16, 0. 30) -0. 06 0. 25 Intrinsic load --- 0. 28 (0. 08, 0. 48) 0. 17 0. 006 Extraneous load --- 0. 25 (0. 60, 0. 45) 0. 14 0. 01

In other words… • Intrinsic load : fatigue, number of maneuvers, supervisor took over : year in training, prior colonoscopy experience, good patient tolerance • Extraneous load : fatigue, queue order, supervisor takeover : more engaged supervisor, more confident supervisor • Germane load : more engaged supervisor, intrinsic load, extraneous load

Implications for instructional design • To optimize IL: - Select (partial) task for learner’s competence and prior experience - Take over quickly when early learner is struggling • To minimize EL: - Monitor fatigue - Consider target number of procedures per session - Remain engaged and communicate confidence • To maximize GL: - Remain engaged with learner and procedure - Adjust support based on learner’s experience and procedural complexity

Limitations • Some features specific to colonoscopy • CLIC measures learner perceptions of cognitive load • No measures of learning • Associations, not causation

Summary • Different sets of features are associated with the three types of cognitive load among colonoscopy learners • Classes of features studied are generalizable and adaptable • Findings can inform future education interventions to help balance the scales during procedural skills training Learners Teachers, patients, system Case volume Safety Deliberate practice Quality Autonomy Efficiency Feedback Patient-provider relationship

justin. sewell@ucsf. edu