How to make sure your Students Learn what

How to make sure your Students Learn what you want them to Claus Brabrand ((( brabrand@itu. dk ))) ((( http: //www. itu. dk/people/brabrand/ ))) Associate Professor, Ph. D. Software and Systems Section IT University of Copenhagen, Denmark

An Introduction to Constructive Alignment and The SOLO Taxonomy Claus Brabrand ((( brabrand@itu. dk ))) ((( http: //www. itu. dk/people/brabrand/ ))) Associate Professor, Ph. D. Software and Systems Section IT University of Copenhagen, Denmark

T Exercise: "What is good teaching? " Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [3] Oct 10, 2013

Outline (Oct 10, 2013) n 1) Introduction Constructive Alignment n The SOLO Taxonomy n n 2) From Content to Competence Advocate a shift in perspective n Elaborate on The SOLO Taxonomy n n 3) In Practice Concrete recommendations n Alignment implementation process n Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [4] Oct 10, 2013

Introduction to… n Constructive Alignment & SOLO Taxonomy: John Biggs’ popular and heavily cited book: “Teaching for Quality Learning at University - What the student does” [J. Biggs & C. Tang, 2007] “Teaching & Understanding” 19 min award-winning short-film on Constructive Alignment (available on DVD in 7 languages, epilogue by John Biggs) Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [5] Oct 10, 2013

T Neighbour Discussion: "What are the film's main messages (in your opinion)"? Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [6] Oct 10, 2013

Outline (Oct 10, 2013) n 1) Introduction Constructive Alignment n The SOLO Taxonomy n n 2) From Content to Competence Advocate a shift in perspective n Elaborate on The SOLO Taxonomy n n 3) In Practice Concrete recommendations n Alignment implementation process n Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [7] Oct 10, 2013

From Content to Competence n My old course descriptions (Concurrency 2004): n Given in terms of a 'content description'. n Essentially: Goal is…: To understand: deadlock interference synchronization. . . This is a bad idea for two reasons. . . ! Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [8] Oct 10, 2013

Problem 1 ! n Problem with 'content' as goals ! analyze. . . theorize. . . analyze systems explain causes agre a from ion dge it owle ed trad nt t kn s taci rch-ba y stude a b rese known not define deadlock describe solutions nt eme Goal is…: Stud. C To understand: Teacher deadlock interference synchronization. . . analyze systems explain causes name solutions recite conditons Stud. B Stud. A Censor Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [9] Oct 10, 2013

Problem 2 ! n Problem with 'understanding' as goals ! Goal is…: To understand: deadlock interference synchronization. . . 'concept of deadlock' ? ! The answer is simple: It cannot be measured ! Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 10 ] Oct 10, 2013

Competence ! n 'Competence' as goals ! n Competence : = knowledge + capacity to act upon it Have the student do something; and then observe (evaluate) the product and/or process Objective ! To learn how to: analyze systems for. . . explain cause/effects. . . prove properties of. . . compare methods of. . . Note: 'understanding' is of course pre-requisitional ! Note': inherently operational (~ verbs) 'SOLO' = Structure of the Observed Learning Outcome Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 11 ] Oct 10, 2013

T Neighbour Discussion "How does this content vs competence relate to YOUR courses? " Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 12 ] Oct 10, 2013

Note: the list is non-exhaustive Graphic Legend SOLO (elaborated) QUANTITATIVE SOLO 2 SOLO 3 ”uni-structural” “multi-structural” define identify count name recite paraphrase follow (simple) instructions … Q R combine structure describe classify enumerate list do algorithm apply method … Q Claus Brabrand, IT University of Copenhagen R SOLO 4 “relational” problem / question / cue known related issue - given! hypothetical related issue - not given! R student response Q QUALITATIVE SOLO 5 “extended abstract” analyze compare contrast integrate relate explain causes apply theory (to its domain) … Q University Pedagogy, Danish Technical University R theorize generalize hypothesize predict judge reflect transfer theory (to new domain) … Q [ 13 ] R R' Oct 10, 2013

SOLO verbs Mapped by: § B. Dahl & C. Brabrand (Natural science context!) With help from: § 3 Educational research colleagues (medicine) § J. Biggs & C. Tang Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 14 ] Oct 10, 2013

SOLO Advantages n Advantages of The SOLO Taxonomy: Linear hierarchical structure (good for progression) n Aimed at evaluating student learning n Converges on research (at SOLO 5) n Research: Production of new knowledge Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 15 ] Oct 10, 2013

T Exercise "Key competences in YOUR course? " Concurrency: analyze systems compare models Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 16 ] Oct 10, 2013

The BLOOM Taxonomy (1956) The BLOOM Taxonomy: Synthesis Evaluation ive SOLO Analysis at Qualit 4+5 n titative SOLO 2+3 Quan Application Comprehension Knowledge ” ”[…] really intended to guide the selection of items for a test rather than to evaluate the quality of a student’s response to a particular item” Claus Brabrand, IT University of Copenhagen -- (Biggs & Collis, 1982) University Pedagogy, Danish Technical University [ 17 ] Oct 10, 2013

Outline (Oct 10, 2013) n 1) Introduction Constructive Alignment n The SOLO Taxonomy n n 2) From Content to Competence Advocate a shift in perspective n Elaborate on The SOLO Taxonomy n n 3) In Practice Concrete recommendations n Alignment implementation process n Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 18 ] Oct 10, 2013

Implementation Process 1) Think carefully about: overall goal of course (what students learn to do? ) 2) Operationalize these goals and formulate them as SOLO intended learning outcomes learning incentive 3) Choose carefully the form(s) of assessment (~ intended learning outcomes) Claus Brabrand, IT University of Copenhagen alignment learning support 4) Choose carefully the form(s) of teaching (~ intended learning outcomes) University Pedagogy, Danish Technical University [ 19 ] Oct 10, 2013

Concrete Recommendations Intended Learning ulation': 1) Use 'standard form on the student a) puts learning focus "to be able to" n: io at ul rm fo e nc te pe Outcomes [Genetics 101] b) com After the course, the students are expected to be able to: locate V genes. N on chromosomes 4) Avoid 'understanding-goals': do V simple calculations N: (e. g. , recombination"To frequencies, understand X", in-breeding coefficients, Hardy-Weinberg, "Be familiar with Y", evolutionary equilibria). "Have a notion of Z" ! describe V and perform V connexion-analysis N • describe V fundamental genetic concepts: (e. g. , mutation variation, in-breeding, natural selection). describe V and analyze V simple inheritancies N analyze V inheritance N of multiple genes simultaneously 2) List sub-goals a s 'bullets': Clearer than text Claus Brabrand, IT University of Copenhagen 3) Use 'Verb + Noun' formulation: What the student is expected to do V with a given matter N. University Pedagogy, Danish Technical University [ 20 ] Oct 10, 2013

Implementation Process 1) Think carefully about: overall goal of course (what students learn to do? ) 2) Operationalize these goals and formulate them as SOLO intended learning outcomes learning incentive 3) Choose carefully the form(s) of assessment (~ intended learning outcomes) Claus Brabrand, IT University of Copenhagen alignment learning support 4) Choose carefully the form(s) of teaching (~ intended learning outcomes) University Pedagogy, Danish Technical University [ 21 ] Oct 10, 2013

Teaching/Learning Activities functional knowledge (problem oriented) Project work Case teaching teacher centric student centric Lecture Claus Brabrand, IT University of Copenhagen Exercise class declarative knowledge (discipline oriented) University Pedagogy, Danish Technical University [ 22 ] Oct 10, 2013

Lecture n Teacher activity: n n n n [ Biggs & Tang 2007, p. 137 ] (about Application) vs. Student activity: n Introduce Explain Elaborate Discuss application Give examples Show PPT slides Questions on slides Winding up Claus Brabrand, IT University of Copenhagen n n n n Listen (maybe take notes) Understand? (correctly? deeply? ) Listen (maybe take notes) Watch (maybe note points) Write answers to questions Possibly ask a question active teacher vs. passive student University Pedagogy, Danish Technical University [ 23 ] Oct 10, 2013

Learn about vs. Learn to do n Learning (about): n n n n n about application about cooking about programming about designing about analysis about construction about relating. . . Learning (to do): n n n n to apply to cook to program to design to analyse to construct to relate. . . student listening (to something about something) vs. student doing !! Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 24 ] Oct 10, 2013

Student Activation "The (in-famous) Learning Pyramid": Average retention rate Note: these percentages are "bogus" active student Listening: Learning about s ac tud tiv en at t io n passive student Doing: Learning to do Lecture 5% Reading 10% Audiovisual 20% Demonstration 30% Discussion group 50% Practice by doing 75% Teaching others 80% [ NTL Institute for Applied Behavioral Science, Bethel, Maine ] Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 25 ] Oct 10, 2013

Constructive Alignment § Make explicit ILO's Teaching § Learning = ILO's = Activities ( Intended Learning Outcomes ) Assessment § (…and tell this to students) vs. SUSAN: intrinsically motivated Claus Brabrand, IT University of Copenhagen ROBERT: extrinsically motivated University Pedagogy, Danish Technical University [ 26 ] Oct 10, 2013

Questions. . . R Q R' Association new ~ old ert Susan & Rob My research ing" "understand teaching and e c mpeten content co models Teacher 2 -3 levels 1 - The Film Cognitive proce ss es Course descriptions The Book 'TLA' Teaching / Learn ing Activities The SOLO Taxo nomy ? Students at University ? " g od teachin o "What is g Student activ ation Tips'n'Tricks MAT CS v. NAT v. s John Bigg Claus Brabrand, IT University of Copenhagen recite generalize 15% program Top Competences ming Constructive Alignment University Pedagogy, Danish Technical University [ 27 ] Oct 10, 2013

Now, please: "2 -minute recap" n T Please spend 2' on thinking about and writing down the most important points from the talk – now!: Immediately After 1 day After 1 week After 2 weeks Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University After 3 weeks [ 28 ] Oct 10, 2013

Key References n ”Teaching for Quality Learning at University” John Biggs & Catherine Tang Society for Research into Higher Education, 2007. Mc. Graw-Hill. n ”Evaluating the Quality of Learning: The SOLO Taxonomy” John Biggs & Kevin F. Collis London: Academic Press, 1982 n ”Teaching & Understanding” Claus Brabrand & Jacob Andersen 19 minute award-winning short-film (DVD) Aarhus University Press, Aarhus University, 2006 n "Constructive Alignment & The SOLO Taxonomy: a Comparative Study of University Competencies in Computer Science vs. Mathematics" Claus Brabrand & Bettina Dahl CRPIT, Vol. 88, ACS 3 -17, R. Lister & Simon, Eds. , 2007 Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 29 ] Oct 10, 2013

Thank You! Film's homepage: ((( http: //www. daimi. au. dk/~brabrand/short-film/ )))

Tips'n'Tricks (activation) n Neighbour discussions: n Post-It exercise: § focus: zoom in § anonymous (!) § swap'able § everyone will engage § empathetic control § shared knowledge pool § more questions (students dare ask them) § better questions (students had a chance to discuss) [Phil Race] 1 -2 min timeout Form variation: n lecturing blended with in-class activation exercises Claus Brabrand, IT University of Copenhagen Frequent breaks: pulse reader measurements: n University Pedagogy, Danish Technical University [ 31 ] Oct 10, 2013

Tips'n'Tricks (cont'd) n Use many examples: (build on student pre-knowledge) NEW n vs. Explicit structure: 1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4. wwwwwww § self evident to you [ teacher ] § not to a learner [ student ] (esp. during learning process) OLD "Less-is-more": § analyze § compare § relate n n Student 'recap' at end: common deadlock, uncommon deadlock, A-synchronization, Bsynchronization, hand-shake, multi-party synchronization, multi-party hand-shake, binary semaphores, generalized semaphores, blocking semaphores, recursive locks, . . . now after 1 day after 1 week after 2 weeks after 3 weeks Emphasize depth over breadth (coverage) Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 32 ] Oct 10, 2013

Problematic Courses n E. g. course: ”Databases” (at RUC/Roskilde): § obtain knowledge about the structure of database systems; § be familiar with design of databases by use of special notations like E/R and analysis through normalization; § get an overview of the most important database models and a detailed knowledge about the most important model - the relational model as well as the language SQL; § get an overview of database indexing and query processing; § obtain knowledge about application programming for DB systems. n Note: almost entirely non-operational(!) n i. e. measure how? ! Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University Familiar with ? ! [ 33 ] Oct 10, 2013

UNALIGNED COURSE Teacher’s intention e. g. - explain - relate - prove - apply Student’s activity "Dealing with the test" Exam’s assessment e. g. - memorize - describe Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 34 ] Oct 10, 2013

ALIGNED COURSE Teacher’s intention e. g. - explain - relate - prove - apply Student’s activity Exam’s assessment e. g. - explain - relate - prove - apply Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 35 ] Oct 10, 2013

Student Motivation § Susan: (”intrinsic motivation”) - wants to…: learn ! § Robert: (”extrinsic motivation”) - to…: pass exams ! Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 36 ] Oct 10, 2013

Constructivism § ”Transmission is Dead…” : ! active teacher & (lectures risk = passive students) § Knowledge is… Actively Constructed ! Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 37 ] Oct 10, 2013

SOLO Taxonomy § Hierarchy for Competences: 5: 4: 3: 2: generalize, theorize, predict, … explain, analyze, compare, … describe, combine, classify, … recite, identify, calculate, … § Deep learning (not surface) ! Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 38 ] Oct 10, 2013

Stud Learning Focus § Focus on Student Learning ! (instead of ”what teacher does” & labelling students: ’good/bad’) § Student activitation learning Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 39 ] Oct 10, 2013

Alignment § Make explicit ILO’s (Intended Learning Outcomes): § Exam = ILO’s = Teaching § (…and tell this to students) Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 40 ] Oct 10, 2013

Acquisition of Competence n Acquisition of competence progresses according to the following stages of learning: 1) Unconscious incompetence n 2) Conscious incompetence n 3) Conscious competence n 4) Unconscious competence n n 5) Capacity for moving consciously between stages 3) and 4): (which is required by a teacher) Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 41 ] Oct 10, 2013

On the Role of Examination n Constructive Alignment: A systemic theory (a teaching system w/ cause/effects) n A theory of planning (over the course of a course) n A theory of motivation (and incentive) n "The exam does not come after, but before the course!" n From the exam as a. . . : "Necessary evil" application of alignment . . . to: Claus Brabrand, IT University of Copenhagen Motivational and learning-guiding pedagogical tool for the teacher(!) University Pedagogy, Danish Technical University [ 42 ] Oct 10, 2013

Definition: “Good Teaching” n Definition: ”Good teaching is getting most students to use the higher cognitive level processes that the more academic students use spontaneously” -- “Teaching for Quality Learning at University”, John Biggs, 2003 n Good news (we now know how to do this): Alignment!!! n Explicitly defined course objectives (as verbs)! n Discourage surface-learning! n Encourage depth-learning! n “Less-is-more”: depth rather than breadth of coverage! n Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 43 ] Oct 10, 2013

The SOLO Taxonomy (1982) SOLO 1: (Pre-Structural) n Here the subject does not have any kind of understanding but uses irrelevant information and/or misses the point altogether. Scattered pieces of information may have been acquired, but they are unorganized, unstructured, and essentially void of actual content or relation to a topic or problem. SOLO 2: (Uni-Structural) n The subject can deal with one single aspect and make obvious connections. The subject can use terminology, recite (remember things), perform simple instructions/algorithms, paraphrase, identify, name, count, etc. SOLO 3: (Multi-Structural) n At this level the subject can deal with several aspects but these are considered independently; i. e. , not in connection with one another. Metaphorically speaking; the subject sees the many trees, but not the forest. He or she is able to enumerate, describe, classify, combine, apply methods, structure, execute procedures, etc. SOLO 4: (Relational) n At level four, the subject may understand relations between several aspects and how they might fit together to form a coherent whole. The understanding forms a structure and now he or she sees how the many trees form a forest. A subject at this level may compare, relate, analyze, apply theory, explain in terms of cause and effect, etc. SOLO 5: (Extended Abstract) n At this level, which is the highest, the subject may generalize structure beyond what was given, may perceive structure from many different perspectives, and transfer ideas to new areas. He or she may generalize, hypothesize, criticize, theorize, etc. Claus Brabrand, IT University of Copenhagen University Pedagogy, Danish Technical University [ 44 ] Oct 10, 2013