Merging Code Comprehension and Writing Tasks Angus Mc

Merging Code Comprehension and Writing Tasks Angus Mc. Intosh

I’m very much just run it, run it. The run button is my best friend Usually I would just write it, see if it works, if it works great move on if it doesn’t work oh god ok why is not working.

Motivation Code comprehension is important • But it can feel fake Writing code is engaging • But there is a sense that some student do not understand their code Can we put code comprehension into code writing?

Doesn't Code Writing Already Teach Code Comprehension? • Yes, for some students. • But not all. • Many tutors and lecturers have experienced student that do not understand the code that they have written. • To be able to merge code comprehension and code writing it helps to understand why writing code doesn't always involve understanding it

Demonstration • You will answer two multiplication questions • Report when you know the answer

73 x 73 = ?

73 x 73 = 5329

How did you reach your answer? • Did the multiplication • Mental arithmetic • Long multiplication on paper • Got something else to do it for you • Used a calculator • Looked at somebody else's working • Already knew the answer • You have a very good knowledge of square numbers • You have already seen the presentation • Didn't reach an answer • Couldn't find a method • Failed at other methods • Didn't attempt to reach the answer

532 x 532 = ? sqrt(283024) = 532

532 x 532 = 283024 sqrt(283024) = 532

How did you reach your answer? • Did the multiplication • Mental arithmetic • Long multiplication on paper • Used Existing Knowledge • Used that the square root of 283024 is 532 • Got something else to do it for you • Used a calculator • Looked at somebody else's working • Already knew the answer • You have a very good knowledge of square numbers • You have already seen the presentation • Didn't reach an answer • Failed at other methods • Didn't attempt to reach the answer

Generalising • People take the easiest way to reach a goal that is sufficiently accurate[1] • They will try to avoid high levels of cognitive effort if they can whilst still reaching their goal[2] • This can even be subconscious[3] 1. Norman, Donald A. [1983], Some Observations on Mental Models, In collection: Mental Models, edited by: Stevens, Albert L. and Gentner, Dedre, Lawrence Erlbaum Associates, pages 7 -14 2. Schwartz, Daniel L. and Black, John B. [1996], Shuttling between depictive models and abstract rules: Induction and fallback, Cognitive Science, pages 457 -497, isbn: 0364 -0213 3. Cohen, Jonathan D. and Schooler, Jonathan W. [1997], Scientific approaches to consciousness, In

Ski Poles Ski poles are helpful when: • You are starting to learn to ski • You are an experienced skiier Ski poles are unhelpful when: • You are an intermediate skiier

Implication For Goal Driven Coding Tasks • Goal Driven Learning Tasks teach how to reach the goal • Coding tasks teach how to get code that matches a specification • There are ways to achieve their goal without understanding • Monkey Typewriter/Random Guessing • Pattern Matching/Educated Guessing • Help From Tutor/Peers/Internet • Success is reaching the goal • If a student struggles at understanding code they will avoid it but still see themselves as successful if their program works

Add Extra Goals The goals should: • Require understanding to achieve • Involve building new understanding of the code • Cannot be achieved only using knowledge of the specification • Feels relevant to coding • Not annoying to achieve

Ask Questions About Understanding • Before they see the program run, so they can't use that as a clue • Focus on code behaviour, instead of the programmer's intention • • Focus on structural instead of functional understanding They will often already know their intention; they just wrote it Structural behaviour can be demonstrated by running the code Structural questions can be automated • Ask questions about different sizes of code piece from subexpressions to loops • Build it into an automated system in the IDE

Example Questions • What is the value of x + 5 when x is 3? • After the first execution of the for loop what is the value of y? • Will the code inside the if statement execute the second time? Question Picking • Ask questions on topics that haven't been answered right twice in a row • Ask for the outcome of three iterations of a loop • Ask about iterations that go through different paths

Tutor Feedback • Should ask the questions in execution order • But should skip some parts • Concern that some students may not be able to answer structural questions at all

The Exploratory Study • Acted as an automated tutor asking questions • Participants completed coding tasks in Python • 2 participants from CS 1 P and 2 from CS 1 PX • Asked about their experience during semistructured interviews afterwards

They Looked Closely At Their Code I’m hitting that run button going this is going to run because I’ve looked at it and its like yeah nothing is getting past me because I’ve scrutinised it so much. I was a lot slower but I was scrutenising my code in a lot more. I was looking at it a lot more in depth than what I would usually do. I feel like this is good for making me pay more attention to details. Well after this I’m beginning to question a lot It made me suspicious of everything. Its like does this do what it does? I will be staring at my code for longer before I use the run button I guess maybe inspecting it more.

Effect On Confidence My confidence has gone down the attention to detail in like how my able to without testing the code right away actually going through it and thinking about it. , like how many errors could I catch if I pay more attention to detail. In this sense my confidence went definitely down. I’m beginning to say to myself in my head I’d better be revising.

Effect On Confidence : Why? How long it took to solve these kinds of questions because I don’t think it should have taken this long These are things that I have done and I would have expected more of myself The number of mistakes I made The part involving dictionaries and counting cause thats all something I couldn’t grasp last year and I’ve had no contact with it once again.

Effect On Confidence • In general their confidence dropped • Weak skills had been exposed • Goal driven learning had hidden the weaknesses • Using this technique at the beginning would prevent those weaknesses ever forming • This happened at all levels of ability

Summary of the Project • Survey of code comprehension literature and pedagogies • Survey of literature on demonstrations and mental models to show why demonstrations do not work • Analysis of running code as a demonstration of code behaviour • Tutor survey to collect best practices and feedback • Created a protocol to generate questions from student's code • Organising and running tutor sessions with students • Analysis and discussion of results

I’m hitting that run button going this is going to run because I’ve looked at it and its like yeah nothing is getting past me because I’ve scrutinised it so much.

Discussion Questions • In your experience, how many students avoid thinking deeply about their code? • Should they always be encouraged to understand all their code? When is it ok not to? • Are there other areas in Computing Science where it is easy to take an approach to a task that makes it less educationally effective?