CLT Conference Heerlen Does Cognitive Load Theory account

CLT Conference Heerlen Does Cognitive Load Theory account for the beneficial effects of worked examples in tutored problem solving? Ron Salden, Ken Koedinger, Vincent Aleven, & Bruce Mc. Laren (Carnegie Mellon University, Pittsburgh, USA)

Worked examples and tutored problem solving • Worked examples mostly investigated in untutored problem solving environments • Cognitive Tutor is Intelligent Tutoring System provides step-by-step guidance during complex problem solving practice

Worked examples and tutored problem solving • Cognitive Tutor provides a tougher control condition of tutored problem solving – It is this tutored part that in our view reduces extraneous load – And, sometimes, increases germane load

Cognitive Tutors • Longstanding tradition in improving students’ learning • Grounded in cognitive theory (ACT-R, Anderson & Le. Bière, 1998) • Methods for reducing WM load – Scaffolding or prompting of sub-goals – Step by step feedback & hints (i. e. , guided learning) • Use cognitive model of student thinking • Many full-year classroom evaluations show improved math competence (Koedinger & Aleven, 2007)

Cognitive Tutors Elements that reduce extraneous cognitive load.

Standard Cognitive Tutor: control condition 1 a. Corrective feedback 9/30/2020 Pittsburgh Science of Learning Center 6

Standard Cognitive Tutor: control condition 1 b. Implicit positive feedback 9/30/2020 Pittsburgh Science of Learning Center 7

Standard Cognitive Tutor: control condition 2. Stepwise hints: last hint level is bottom-out hint problem fading into example

Standard Cognitive Tutor: control condition 3. Problem sub-goals are given

Standard Cognitive Tutor: control condition 4. Student’s self explanation This feature is not so much about reducing extraneous load but about increasing germane load 9/30/2020 Pittsburgh Science of Learning Center 10

Studies • Shih et al (Geometry) • Mc. Laren et al (Chemistry) • Salden et al (Geometry) Not addressed in this talk • Anthony et al (Algebra) obtained similar results as the other three studies Does Cognitive Load Theory explain beneficial effects of examples in tutored problem solving?

Shih et al • Study: re-analyzing prior study (Aleven & Koedinger, 2002) – Logged response data on bottom-out hint usage • One type of “Gaming the system” behavior – Can be hint abuse due to students skipping abstract hints to obtain the concrete answer – Can also be helpful when bottom-out hints act as worked examples

Shih et al • Developed a model to distinguish between good student use of bottom-out hints from bad student use of bottom-out hints • Two key elements of model are time spent on: – Reflecting about prior step (after bottom-out hint) – Thinking about next step (prior to next action) • Subtraction method to isolate reflection (selfexplanation) time – Use other data, when bottom-hints are not requested, to estimate next step time

Shih et al results • High correlation of time spent reflecting on bottom-out hint with learning (pre-to-post gain) • Spending time on hints is beneficial to learning for all students • Difference between students’ hint usage: – Good usage = spending more time on bottom-out hint – Bad usage = spending less time on bottom-out hint • Thus students who study bottom-out hint as worked example obtain higher learning gains

Mc. Laren et al • Conducted three studies comparing – Tutored Alone vs. Worked Examples + Tutored • Examples are alternated with isomorphic problems

Stoichiometry Tutor: control condition

Worked Example condition • Students watch video of a worked example plus do prompted self-explanations following the example:

Mc. Laren et al results • No differences on posttest performance • BUT, students in Examples condition did learn more efficiently, using 21% less time to finish same problem set

Salden et al • Conducted lab and classroom study comparing: – Tutored problem solving – Fixed example fading – Adaptive example fading • Adaptive fading based on students’ selfexplanations of the example steps – Students who self explain well receive fewer examples than students who self explain poorly

Standard Cognitive Tutor: control condition 9/30/2020 Pittsburgh Science of Learning Center 20

Example-enhanced Tutor: experimental condition Worked out value step with calculation shown by Tutor Student still has to self explain the worked out value step!

Salden et al results • Lab study: – Adaptive fading condition needed fewer examples than fixed fading condition – Adaptive fading > both fixed conditions on posttest and delayed posttest • Classroom study: – Adaptive fading condition needed fewer examples on several theorems than fixed fading condition – Adaptive fading > problem solving on delayed posttest

Summary of results • Shih et al: Students can effectively use bottom-out hints as worked examples and achieve higher learning gains • Mc. Laren et al: Students working with examples can complete learning phase needing 21% less time while obtaining the same learning outcomes • Salden et al: Students learning from adaptively faded examples obtained higher immediate and delayed posttest performance • Fourth study by Anthony et al (using Algebra Tutor): Students who learned with examples attained better long term retention Ø Also measured mental effort: examples = tutored problem solving

Does CLT explain these beneficial effects of worked examples in tutored problem solving? • Cognitive Tutor is a harder control condition than untutored environments – Students can effectively use bottom-out hints as worked examples • The tutoring seems to reduce possible extraneous cognitive load – Anthony study even showed no difference in mental effort between control and experimental condition • Stepwise feedback & hints, self-explanation prompts geared to increase germane cognitive load

Does CLT explain these beneficial effects of worked examples in tutored problem solving? Possible explanations • • • Without the information (guidance) provided by examples, students waste time tackling new skills during problem solving – Mc. Laren study: examples lead to same learning gains but needed 21% less time – Two Freiburg lab studies: examples lead to same learning gains needing roughly 17. 5% and 25% less time Motivation – Goal of understanding v. performing (Shih et al) – Frustration after unsuccessful solution attempt Where is the cognitive load?

Questions?