Cognitive approaches Current understanding and future trends Kevin

  • Slides: 40
Download presentation
Cognitive approaches: Current understanding and future trends Kevin W. Eva Department of Medicine Centre

Cognitive approaches: Current understanding and future trends Kevin W. Eva Department of Medicine Centre for Health Education Scholarship University of British Columbia

What do we (need to) know? • Interventional efforts • Assessment strategies

What do we (need to) know? • Interventional efforts • Assessment strategies

What do we know?

What do we know?

Summary • It is important to recognize that exclusive reliance on nonanalytic processes creates

Summary • It is important to recognize that exclusive reliance on nonanalytic processes creates error … • … but so does absolute aversion to those processes

Interventional Efforts • Representation scaffolds (Braun et al. , 2017) • Test-enhanced learning (Raupach

Interventional Efforts • Representation scaffolds (Braun et al. , 2017) • Test-enhanced learning (Raupach et al. , 2016) • Self-explanation (Chamberland et al. , 2015) • De-biasing (Croskerry, 2015)

Interventional Efforts • Knowledge oriented • Encouraging students to explain underlying mechanisms or compare

Interventional Efforts • Knowledge oriented • Encouraging students to explain underlying mechanisms or compare and contrast cases vs • Process oriented • Teaching students how to reason • Evidence for the former stronger than for the latter

“The essence of expertise is more a matter of knowing a lot about the

“The essence of expertise is more a matter of knowing a lot about the world than of possessing advanced reasoning strategies” John Anderson

What do we need to know?

What do we need to know?

How many jelly beans are there in this jar? You’re wrong but You’re all

How many jelly beans are there in this jar? You’re wrong but You’re all right

Many fallible judgments can be summed to create something truly exceptional

Many fallible judgments can be summed to create something truly exceptional

What matters? 1. Independence 2. Diversity 3. Volume 4. Decentralization

What matters? 1. Independence 2. Diversity 3. Volume 4. Decentralization

Summary • How can we take better advantage of movements towards collaborative and multi-professional

Summary • How can we take better advantage of movements towards collaborative and multi-professional practice? (see Kiesewetter, et al. , 2017) • How are technologies like Electronic Health Records or Social Media changing clinical reasoning? (see Varpio, et al. , 2015)

What do we need to know?

What do we need to know?

What’s your current opinion? • If we had 1000 cases for which the outcome

What’s your current opinion? • If we had 1000 cases for which the outcome was known and we were to present the relevant data to… • Expert judges vs An evidence-based algorithm • Which would win?

What’s your current opinion? • What about … • Expert judge armed with the

What’s your current opinion? • What about … • Expert judge armed with the decision generated by the algorithm vs The algorithm alone? • Which would win?

Summary • Where is deep learning from artificial intelligence going to lead us with

Summary • Where is deep learning from artificial intelligence going to lead us with respect to the need for clinical reasoning? (see Hodges, Jan. 2018)

What do we know?

What do we know?

Assessment Strategies • Incorporation of key features into exams (Surry et al. , 2017)

Assessment Strategies • Incorporation of key features into exams (Surry et al. , 2017) • Identifying cues to improve self-regulation (de Bruin et al. , 2017) • Preparation for future learning (Woods and Mylopoulos, 2015) • Assessment for learning (Agrawal et al. , 2012)

SSSSMNT DRVS LRNNG (Teunissen, 2017)

SSSSMNT DRVS LRNNG (Teunissen, 2017)

What do we need to know?

What do we need to know?

The real influence of experience 1 Accuracy 0. 9 0. 8 0. 7 0.

The real influence of experience 1 Accuracy 0. 9 0. 8 0. 7 0. 6 0. 5 0. 4 0. 3 0. 2 0. 1 0 30 40 50 60 70 Age (See Eva, 2002)

Which factors, personal or external, most influence students’ generation of learning goals? (Eva, et

Which factors, personal or external, most influence students’ generation of learning goals? (Eva, et al. , 2010)

“It is impossible for a man to learn what he thinks he already knows”

“It is impossible for a man to learn what he thinks he already knows” Epictetus

Summary • How can lifelong programmatic assessment be implemented for maintenance of competence in

Summary • How can lifelong programmatic assessment be implemented for maintenance of competence in a manner to which practitioners are receptive? (see Eva, et al. , 2016)

What do we need to know?

What do we need to know?

Ilgen, et al. (2011) • First impression condition • Read the case below and

Ilgen, et al. (2011) • First impression condition • Read the case below and as soon as you know the diagnosis enter it in the box. • Directed search condition • Give a one sentence summary of the problem; List diagnoses you are considering with supporting/refuting data; Rank the diagnoses; List any “must not miss” diagnoses

Inter-case Reliability 0. 7 Novices With experience, however, formal Intermediates knowledge becomes weaker at

Inter-case Reliability 0. 7 Novices With experience, however, formal Intermediates knowledge becomes weaker at Simple Complex differentiating candidates 0. 7 0. 6 0. 5 It is important to select cases of 0. 4 appropriate difficulty if hope to 0. 4 0. 3 differentiate between candidates 0. 3 Novices vary reliably in ability to 0. 2 think analytically and non- 0. 2 0. 1 analytically 0 0 -0. 1 First Impression Directed Search

Summary • How can assessment design be tailored to the level of practice given

Summary • How can assessment design be tailored to the level of practice given what we know about clinical reasoning?

Thanks kevin. eva@ubc. ca

Thanks kevin. eva@ubc. ca

Unavailable options

Unavailable options

Too much from too little ?

Too much from too little ?

Is pattern recognition evil? Error No Error System 1 5% ? System 2 5%

Is pattern recognition evil? Error No Error System 1 5% ? System 2 5% ? 10% 90% • Graber (2005) – 10 -15% error rate • Crowley, et al. (2012) – 2, 230 cases analysed - “ 49. 9% of cases in which a diagnostic occurred were associated with one or more of the measured biases”

Is pattern recognition evil? Error No Error System 1 5% ? System 2 5%

Is pattern recognition evil? Error No Error System 1 5% ? System 2 5% ? 10% 90% fit e en B Ha rm Error No Error System 1 5% 80% System 2 5% 10% Error 10% No 90% Error System 1 5% 10% System 2 5% 80% 10% 90% • Graber (2005) – 10 -15% error rate • Crowley, et al. (2012) – 2, 230 cases analysed - “ 49. 9% of cases in which a diagnostic error occurred were associated with one or more of the measured biases”

Unavailable options

Unavailable options

(Wilson, et al. , 1993)

(Wilson, et al. , 1993)

“Reasoning” aloud On. Line Postp. Hoc value Number of hypotheses 9. 7 5. 3

“Reasoning” aloud On. Line Postp. Hoc value Number of hypotheses 9. 7 5. 3 <. 01 Number of propositions 103. 5 37. 9 <. 001 6. 9 27. 6 <. 05 66. 9 80. 7 <. 05 22. 3 46. 1 <. 05 Position of first hypothesis/Total Position of final hypothesis/Total % of linking statements that were Conditional (vs Causal) (Eva, Brooks, and Norman, 2002)