A First Look at Novice Compilation Behavior Using

A First Look at Novice Compilation Behavior Using Blue. J By Cole Spicer

Introduction • Where does correct code come from? • When and What students choose to compile • Want to determine how students tend to program in lab • What types of errors do they encounter?

Introduction cont… • Look into novice compilation behavior • Goal: – Is to help inform the teaching of programming and the program development

Blue. J • Java programming development environment for small scale software development.

Previous Work • Most studies explore the cognitive psychology of novice programmers – Planning – Pedagogic IDE – Error Rates – Error Message Design – Characterizing Novices • Few studies explore the programmer’s behavior

Methods • Observed novice compilation behavior in classroom tutorial sessions. – Met once a week for one hour – 63 students – Worked through one or two problems to help illustrate concepts form that week’s lecture.

Methods cont… • Set Blue. J up to report at compile time – Complete source from students session – Metadata • Username • Research site • Client-side index indicating compilation number in the current sequence • Compilation result • Filename • Start time of compile • Etc; – Shipped to a server for storage and later analysis

Course Marks • 4 marked assignments – Three were take-home coursework – In class exam • Intended to provide feedback to students and instructors a sense of where they stood.

Grade Distributions Assessments

Attendance • University of Kent Programming Courses = One class session out of eight • Students in the study missed significantly more. • Only six students managed to attend seven of nine sessions • Reasons may be lack of motivation

Analysis of Results • A minority of different types of errors account for the majority of errors dealt with by students. • Mostly quick fix errors • This can be seen as the user is just letting the compiler do the thinking for them.

Error Types and Distribution • Total of 1, 926 errors • Of the 42 error types the five most common accounted for 58% of all the errors – Missing semicolons 18% – Unknown symbol: variable 12% – Brackett expected 12% – Illegal start of expression 9% – Unknown symbol: class 7%

Compilation Errors

Time Between Compilations • 51% of all compilation events occurred 30 seconds after the previous event • 20% of all compilation events involved more than two minutes of work time between events.

Time Between Compiles

Time Between Compiles cont… • The graph doesn’t tell us anything about compilation results. • So analyze the compile events as pairs – F means the compilation was an error – T means the compilation was a success

Compilation Pairs

Amount of Work • Most is done in the two minute span following a successful compile • Even though the student is compiling frequently doesn’t mean a lot of work is being done. • The three most common errors are typically handled in less than 30 seconds and require little change to the source.

Quick Recompile • Since most of the work is done after a successful compile. What are they doing when they recompile quickly.

Session Analysis • “Session” represents the sequence of compiles from one class period. • Student typically compiles 10 times per session.

Session Analysis cont…

Session Analysis cont… • We can see the students who have typical compilation patterns, and the students who compile more than the average. • Students who compile more than the average could be very meticulous or just sloppy • Some students do not even trust the error message reported by Blue. J

Session Analysis cont…

Shaping Behavior • Typical behavior of students from this experiment is to make changes and then come back and correct all the syntax errors that resulted from most recent addition of code. • Now what can be done to shape this behavior. What can be done to change the programmers behavior.

Shaping Behavior cont… • Encouraging them to make fewer semicolon mistake • Introduce highlighting of bracket pairs • Highlight spaces where a expected token is supposed to be • Now observe how students behavior changes

Shaping Behavior cont… • Don’t want students to become dependent of development environments. • Want all shaping of behavior to be in the students best interest.

Future Work • What behaviors are classified as good or bad? • How to detect these behaviors