Itembank statistics in Methods Education Dimitris Pavlopoulos Idea

Itembank statistics in Methods Education Dimitris Pavlopoulos

Idea and development • Many old exams… • Plus (free-to-use) questions from publishers 2

First step: database with questions • Tailored to course-needs • Clustered, with feedback, eventually with clips – Easy to make (summative and formative) tests 3

Educational aim: intensification • Weekly formative tests (MVA: 35 -60 vragen) • Weekly tutorials (SPSS) • Weekly summative test (‘theory’) • Weekly summative test (‘SPSS’) 4

Aim 2: using contact hours… • Courses run well: good success rates, good evaluations… • …but we are still dissastisfied College Toetsing Practicum • Lecture attendance is low − − But not in international groups (Anthropology) Is it due to slidecast lectures or not? • BSc students’ participation is low − − Few questions, little interaction But not in international groups (Anthropology) • Students find tutorials boring 5

SPSS tutorials: • Till last year: − − Students got an assignment (in Canvas) and data They could work independently In the tutorial they could ask questions (sometimes it takes long…) Then they got the solutions • Students don’t come • Or (if it’s compulsory) they come but find it boring… 6

Tutorials: digitalizing • Instruction, links to electonic sources • Students can work independently 7

Aim Tutorials-new form: differentiation • Addressing diversity. Students have: − − − Different background (e. g. MVA MSc) Different level of motivation They fear statistics at different levels… 2 experiments: • BIS/DIS/PMC-BIS courses. Tutorial assignment split in 2 parts − − Basic: first hour ‘Advanced’: second hour • MVA MSc. Students were offered a choice − − Do the assignment independently: tutorial with extra material Follow a ‘classical’ tutorial 8

Midterms • Weekly split in 2 tests − − Theory: answer questions without data SPSS: analyse a dataset and answer questions • Questions − − − Multiple choice (as few as possible…) Numerical (Multiple) fill in the blanks – numerical and text Matching Open 9

Questions: numerical • Work very well, but don’t forget… − − …to set a range (when a calculation is involved) …to check if a completely different answer is correct as well − This is not necessary in a multiple choice! 10

Questions: fill in the blanks • Numerical: the same • Text: work also well but be carefull! − Correct answers that you have not foreseen • A typo, an extra space etc can cause it to be wrong • You need to count is as correct with ‘Modify’ 11

Questions: matching • Works also very well! − Check that the matching is exclusive 12

Questions: open • Compared to paper exams: day and night! − Easy and fast to correct • But even more so for questions with short answers 13

Involve your students: comments • Option for comments in Test. Vision. Dare using it! - Students do react immediately Improve questions you make. Improve feedback Make students co-owners of the formative test 14

Results: no clear image • MVA MSc − − Mean grade: 7. 02 (2017: 7. 17) Percentage pass: 94% (2017: 87. 1%) • BIS − − BSc: 5. 45 PMC: BCO-6. 27, CW (BIS plus)-5. 99, SOC-5. 98 15

Evaluations: very clear image! • MVA MSc • Differentiatie (advanced groep) 16

Evaluations: very clear image! • BIS 17

Way open for: blended learning • Should we change the structure of courses? • Should we offer tutorial online only? − − Give comments Maybe with targeted meetins afterwards • Lectures? Possible structure − − Monday: 3 -hour lecture Tuesday-Thursday: work with online assignment and practice Friday: 2 -hour practice (all together) Biweekly: 2 -hour remedial or advanced meeting (differentiation) 18

Itembank statistics in Methods Education • • • We welcome suggestions! We welcome collaborations! Towards management: we need investments!