LH 216 V Utveckla lrandet med betygskriterier Trff

LH 216 V Utveckla lärandet med betygskriterier, Träff 1 Lärare: Viggo Kann and Anna-Karin Högfeldt Handledare: Svea Ekelin viggo@nada. kth. se; akhog@kth. se 1

Dag 1 12. 15 -12. 35 Välkomna och introduktion 12. 35 -12. 50 Varför betygskriterier? 13. 00 -14. 00 Exempel på betygskriterier, Viggo Kann 14. 15 -14. 45 Prova på! 14. 45 -15. 00 Kursinformation, webbregistrering och miniutvärdering 2

Betygskriterier/Grade Descriptors / Grading Criteria / Grading Rubrics Prestationsnivå i relation till förväntade lärandemål /Levels of achievements in relation to the intended learning outcomes

Vad är det? Obs att detta är ett utdrag ur hel kurs för att visa på tanken med kriterierna! Grading criteria for the Advanced Pavement Engineering Analysis and Design course (AH 2905). Example with one ILO Carry out mechanical analyses of pavement structures, report the analyses results to the client, formulate and motiative assumptions and approximations used ILO 1: A B C D E [B criterion included] using out of the course material; implications of using different analysis strategies are evaluated [C criterion included] with alternative analysis strategies proposed based on the course material [D criterion included] for a more involved technical problem [assessment task] With analysis approach, assumptions and approximations justified based on the course material With a given hint on analysis approach; assumptions and approximations thoroughly documented

Varför använda betygskriterier? Diskutera möjligheter, utmaningar och hinder. Utgå gärna från olika perspektiv 1. …studenterna 2. …kursansvarig/examinator 3. …lärare i andra kurser / programansvariga / GA / studierektorer… 4. …avnämare/potentiella arbetsgivare Redovisa på tavlan efteråt!

När betygskriterier tillämpas brukar det ofta leda till förändringar på examinationen och lärandemålen Betygskriterier behöver vara integrerat med examinationen och bedömningen. Annars blir de lätt ett spel för gallerierna (gesture politics).

Välj ut ett lärandemål från en kurs. Diskutera med 1 -2 pers hur det här lärandemålet kan uppnås/bedömas i olika nivåer (om du har A-F). Om du har P/F, kan du gå djupare och diskutera specifika kriterier för att säkerställa att lärandemålet är uppfyllt. PROVA PÅ!

Bloom, Feisel-Schmitz, SOLO, … TAXONOMIES FOR LEARNING

Feisel-Schmitz Technical Taxonomy Judge: (värdera) Solve: (lösa problem) Explain: (förklara) Compute: (räkna typtal) Define: (återge) To be able to critically evaluate multiple solutions and select an optimum solution Characterize, analyze, and synthesize to model a system (provide appropriate assumptions) Be able to state the outcome/concept in their own words Follow rules and procedures (substitute quantities correctly into equations and arrive at a correct result, Plug & Chug) State the definition of the concept or is able to describe in a qualitative or quantitative manner

SOLO taxonomy Biggs p. 79

Bloom’s taxonomy [Bloom 1956]

Bloom’s revised taxonomy in 2 dim. Krathwohl (2002) The Cognitive Process Dimension The Knowledge Dimension 1. Remember 2. Understand 3. Apply 4. Analyze A. Factual Knowledge B. Conceptual Knowledge C. Procedural Knowledge D. Metacognitive Knowledge Different forms of learning Different types of knowledge 5. Evaluate 6. Create

The Cognitive Process Dimension The Knowledge Dimension A. Factual Knowledge Aa. Knowledge of terminology Ab. Knowledge of specific details and elements B. Conceptual Knowledge Ba. Knowledge of classifications and categories Bb. Knowledge of principles and generalizations Bc. Knowledge of theories, models, and structures C. Procedural Knowledge Ca. Knowledge of subject-specific skills and algorithms Cb. Knowledge of subject-specific techniques and methods Cc. Knowledge of criteria for determining when to use appropriate procedures D. Metacognitive Knowledge Da. Strategic knowledge Db. Knowledge about cognitive tasks, including appropriate contextual and conditional knowledge Dc. Self-knowledge 1. Reme mber 2. Und ersta nd 3. Apply 4. Analyz e 5. Evaluate 6. Create

The Cognitive Process Dimension The Knowle dge Dimens ion A. Factual Knowle dge B. Concep tual Knowle dge C. Proced ural Knowle dge D. 1. Remember 1. 1 Recognizing 1. 2 Recalling 2. Understand 2. 1 Interpreting 2. 2 Exemplifying 2. 3 Classifying 2. 4 Summarizing 2. 5 Inferring 2. 6 Comparing 2. 7 Explaining 3. Apply 3. 1 Executin g 3. 2 Impleme nting 4. Analyze 4. 1 Differentia ting 4. 2 Organizing 4. 3 Attributing 5. Evaluate 5. 1 Checking 5. 2 Critiquing 6. Create 6. 1 Generati ng 6. 2 Planning 6. 3 Producin g

1 -5: ILOs X: Learning activity E: Examination

Kursinformation Kurs-PM: https: //kth. instructure. com/courses/287/pages/kurs-pm? module_item_id=6433

https: //kth. instructure. com/courses/287/assignments INLÄMN. UPPG 1 A. DEADLINE FÖRE TRÄFF 2

Uppgift 1 – Designa betygskriterier för din kurs Följa gärna arbetsgången nedan. Summera i en kort rapport som även beskriver kursen kortfattat. 1. Börja med att skriva om lärandemålen om det behövs. 2. Om du bara har/tänker ha ett enda examinationsmoment i kursen och vill skriva holistiska betygskriterier: hoppa till steg 5! 3. Bryt ner lärandemålen i delmål så att varje delmål examineras på samma sätt i (den tänkta examinationen av) kursen. Notera hur varje delmål examineras. 18

4. Gör en matris som visar vilka mål som examineras var (se ADK-exemplet). 5. Utforma betygskriterier för E (godkäntnivån) för varje delmål. 6. Markera vilka delmål som inte behöver ha kriterier på högre nivå än E. 7. Titta igenom gamla examinationsuppgifter och fundera över vad som brukar krävas för olika betyg. Vad borde krävas? 8. Utforma betygskriterier för A-nivån för varje delmål. Om du anser att ett delmål inte behöver ha kriterier på högre nivå än till exempel E så skriver du kriterier för den högsta nivån som används! 9. Utforma betygskriterier för C-nivån. 10. Skissa på hur betygskriterierna ska användas i examinationen. Kan flera delmål examineras med samma uppgift? Behövs flera uppgifter för att examinera ett delmål? Behövs olika uppgifter för olika betygskriterier? 19

MINIUTVÄRDERING 20

Vi ses på träff 2! Dag 2 studie om betygskriterier, betygskriterier i din egen kurs, testning och granskning av betygskriterier, sammanvägning av betyg, fallgropar och tips 21

Analyzing and ”fixing” the learning outcomes – this final part of the slides is not included in the lecture but can be useful when analyzing your ILOs. [All steps for the design of grading criteria are found in instructions for Assignment 1, and details/examples will be given by Viggo] 22

What is Intended Learning Outcomes (ILOs)? “Objectives that describe intended learning outcomes as the result of instruction are usually framed in terms of (a) some subject matter content and (b) a description of what is to be done with or to that content. ” (Krathwohl, 2002) The student should be able to develop and implement algorithms with data structures and analyze them with respect to correctness and efficiency Example: 23

The objectives define the course To the course teachers To the study program To the students For mobility and comparison between institutions For employers 24

Criteria for the formulation of learning outcomes A. B. C. D. RESULT ORIENTED OBSERVABLE HINTING AT THE LEVEL OF UNDERSTANDING AND REALISTIC BASIS FOR ASSESSMENT, PLANNING OF TEACHING, FOLLOW-UP AND QUALITY ASSURANCE 25

A. RESULT ORIENTED Not a learning outcome, but important in course description The course gives an overview of the development of history of technology from the 70’s to today, and the students have the possibility to specialize within a chosen area Possible aspects to consider for criteria Examples on ILOs Students should be able to - Summarize the development of the history of technology from 1970 to 2014 - Relate the needs of society to technical solutions To whom? Somebody with non-technical background? In what ways? How many important steps to present? What needs? What parts of society? Deep investigation or a wide approach? 26

An ILO is result oriented when – The student is the subject, not the course – When you’re not telling what you are doing within the course, but as a result of the course 27

There is nothing wrong with wanting the students to understand. But how do we know that the students understand? B. OBSERVABLE The students shall understand different processes for product development A possible learning outcome, depending on what you mean with understand BUT, you can keep these specifications to the different levels in the grading criteria! The student shall be able to describe, compare and critically examine different product development processes as well as their properties 28

An ILO is OBSERVABLE when It describes what observable performancies the students should show as reasonable ”proofs” for understanding 29

C. Hinting at the level of understanding and Realistic Pretty vague on what level we are talking about The student must be able to describe properties of the material Many of these aspects can be used in the grading criteria instead of the ILO The student shall be able to - explain the properties of materials in relation to chemical binding - choose the appropriate materials for a given construction with regard to function, conditions of operation, economical and environmental factors A possible learning outcome, depending on what you mean 30

The level of understanding The context and the work carried out The contributions the course makes to the study program, progression and red thread 31

Bloom, Feisel-Schmitz, SOLO, … TAXONOMIES FOR LEARNING 32

Feisel-Schmitz Technical Taxonomy Judge: (värdera) Solve: (lösa problem) Explain: (förklara) Compute: (räkna typtal) Define: (återge) To be able to critically evaluate multiple solutions and select an optimum solution Characterize, analyze, and synthesize to model a system (provide appropriate assumptions) Be able to state the outcome/concept in their own words Follow rules and procedures (substitute quantities correctly into equations and arrive at a correct result, Plug & Chug) State the definition of the concept or is able to describe in a qualitative or quantitative manner 33

SOLO taxonomy Biggs p. 79 34

Bloom’s taxonomy [Bloom 1956] 35

Bloom’s revised taxonomy in 2 dim. Krathwohl (2002) The Cognitive Process Dimension The Knowledge Dimension 1. Remember 2. Understand 3. Apply 4. Analyze 5. Evaluate 6. Create A. Factual Knowledge B. Conceptual Knowledge C. Procedural Knowledge Different forms of learning Different types of knowledge D. Metacognitive Knowledge 36

The Cognitive Process Dimension The Knowledge Dimension 1. Reme mber 2. Und ersta nd 3. Apply 4. Analyz e 5. Evaluate 6. Create A. Factual Knowledge Aa. Knowledge of terminology Ab. Knowledge of specific details and elements B. Conceptual Knowledge Ba. Knowledge of classifications and categories Bb. Knowledge of principles and generalizations Bc. Knowledge of theories, models, and structures C. Procedural Knowledge Ca. Knowledge of subject-specific skills and algorithms Cb. Knowledge of subject-specific techniques and methods Cc. Knowledge of criteria for determining when to use appropriate procedures D. Metacognitive Knowledge Da. Strategic knowledge Db. Knowledge about cognitive tasks, including appropriate contextual and conditional knowledge Dc. Self-knowledge 37

The Cognitive Process Dimension The Knowle dge Dimens ion 1. Remember 1. 1 Recognizing 1. 2 Recalling 2. Understand 2. 1 Interpreting 2. 2 Exemplifying 2. 3 Classifying 2. 4 Summarizing 2. 5 Inferring 2. 6 Comparing 2. 7 Explaining 3. Apply 3. 1 Executin g 3. 2 Impleme nting 4. Analyze 4. 1 Differentia ting 4. 2 Organizing 4. 3 Attributing 5. Evaluate 5. 1 Checking 5. 2 Critiquing 6. Create 6. 1 Generati ng 6. 2 Planning 6. 3 Producin g A. Factual Knowle dge B. Concep tual Knowle dge C. Proced ural Knowle dge D. 38

1 -5: ILOs X: Learning activity E: Examination 39

E. BASIS FOR ASSESSMENT, PLANNING OF TEACHING, FOLLOW-UP AND QUALITY ASSURANCE 40

Learning outcomes What should the students be able to do as a result by the course? Constructive alignment [Biggs] Activities What work must the students do in order to reach the desired outcomes? Assessment What should the students do in order to prove that they have reached the learning outcomes? 41

Time to analyze your ILOs A. B. C. D. RESULT ORIENTED OBSERVABLE HINTING AT THE LEVEL OF UNDERSTANDING AND REALISTIC BASIS FOR ASSESSMENT, PLANNING OF TEACHING, FOLLOW-UP AND QUALITY ASSURANCE 42