Lars Heinemann Modeling and Measuring Competence Maurer Heinemann

  • Slides: 12
Download presentation
Lars Heinemann Modeling and Measuring Competence Maurer, Heinemann, Rauner

Lars Heinemann Modeling and Measuring Competence Maurer, Heinemann, Rauner

Notions of Competence vs. Performance Competence as generic skills Competence as 'mental conditions necessary

Notions of Competence vs. Performance Competence as generic skills Competence as 'mental conditions necessary for cognitive, social, and vocational achievement' Maurer, Heinemann, Rauner

 • Competence vs. • Knowledge • Examination/Assessment • Qualifications • Bildung Maurer, Heinemann,

• Competence vs. • Knowledge • Examination/Assessment • Qualifications • Bildung Maurer, Heinemann, Rauner

General Assumptions Competence Model Measurement Model Test Tasks Maurer, Heinemann, Rauner

General Assumptions Competence Model Measurement Model Test Tasks Maurer, Heinemann, Rauner

Maurer, Heinemann, Rauner

Maurer, Heinemann, Rauner

Maurer, Heinemann, Rauner

Maurer, Heinemann, Rauner

Maurer, Heinemann, Rauner

Maurer, Heinemann, Rauner

Andrea Maurer, Felix Rauner, Lars Heinemann Ensuring inter-rater reliability in a large scale competence

Andrea Maurer, Felix Rauner, Lars Heinemann Ensuring inter-rater reliability in a large scale competence measurement project in China Maurer, Heinemann, Rauner

Demand is… Functionality fully met rather not met Does the solution work? Does the

Demand is… Functionality fully met rather not met Does the solution work? Does the solution represent the state of the art? Does the solution consider practical feasibility? Does the solution explain the functional relations? Are descriptions and explanations correct? Maurer, Heinemann, Rauner

Rater Training 1. Introduction (Plenum) - the KOMET-Competence Model - the KOMET-Measurment Model -

Rater Training 1. Introduction (Plenum) - the KOMET-Competence Model - the KOMET-Measurment Model - the Items 3 to 5 hours Test rating using exemplary solutions of all test tasks 30 to 60 Minutes (Plenum) Introduction of first task and its solution space Introduction of solutions Constituting Work groups of 5 to 6 persons Test rating in work groups: 1, 5 to 2 hours - Individual rating - Group discussion - Group rating 4. Plenum Presentation of Rating results, discussing problems Analysing all results in comparison to results of skilled raters - Introduction of 2 nd task and solutions 1 to 2 hours 5. Rating in groups (cf. task 1) 1 to 1, 5 hours 6. Presentation and discussion of group results in Plenum 1 to 2 hours 7. 3 rd and 4 th task as before Ca. 2 to 3 hours per task 8. Final individual rating of all solutions Ca. 15 to 20 min. per solution Maurer, Heinemann, Rauner

Maurer, Heinemann, Rauner

Maurer, Heinemann, Rauner

Inter-rater reliability (Finnunjust), based on Chinese ratings Solution code Test task Day 1 Day

Inter-rater reliability (Finnunjust), based on Chinese ratings Solution code Test task Day 1 Day 2 (morning) Day 2 (afternoon) Day 3 (morning) Day 3 (afternoon) H 0282 Signal installation . 41 . 82 H 0225 Signal installation . 54 . 79 H 0176 Drying room . 80 . 84 H 0234 Drying room . 75 . 80 H 0265 Roof window control . 84 . 82 H 0102 Roof window control . 82 . 83 H 0336 Gravel plant . 86 . 85 H 0047 Gravel plant . 79 Maurer, Heinemann, Rauner