Community Updates Michaela Hesse Project Manager telc Rebecca

Community Updates

Michaela Hesse Project Manager, telc Rebecca Rosenlund Project Manager, telc Sean Mc. Donald Head English Exams, telc

NOT from paper to on-screen

Speakers Michaela Hesse Project Manager Test Development Sean Mc. Donald Head of English Assessment Rebecca Rosenlund Project Manager Digital Development 25. 11. 2020 telc g. Gmb. H 4

telc (the european language certificates) • Based in Frankfurt, Germany • Language tests based on international standards • More than 80 different examinations in 10 languages • Subsidiary of the German Adult Education Association (DVV) 25. 11. 2020 telc g. Gmb. H 5

25. 11. 2020 telc g. Gmb. H 6

Product Vision & Test Specification Business Goals Stakeholder Research • Pilot project for digital processes • Benefits for test development • Workshops with teachers & examiners (process, content) • General process rationalization Standards • CEFR • National curricula • Surveys (English & media usage) telc English A 2·B 1 From School to Job 25. 11. 2020 • Digital English exam using Surpass • Target group: students in middle schools (grade 9/10) • English for the (first) job • Reading, Listening, Writing/Mediation, Speaking/Mediation • Certificate as USP for applications telc g. Gmb. H 7

Using Surpass • • 25. 11. 2020 Decision to use Surpass Training “Playtime” and making mistakes Internal trials with telc employees telc g. Gmb. H 8

Item authoring 25. 11. 2020 telc g. Gmb. H 9

Item authoring 25. 11. 2020 telc g. Gmb. H 10

25. 11. 2020 telc g. Gmb. H 11

Proof of concept: delivery in schools • Trials in 10 German middle schools • PC labs, Laptops, online/offline delivery • Surpass-Prototype (~20 min. exam time) • Feedback on process, IT infrastructure, usability Surpass 25. 11. 2020 telc g. Gmb. H 12

Proof of concept: Validity • • • 25. 11. 2020 Piloting Pre-testing Standard Setting Pre-testing II Standard Setting II telc g. Gmb. H 13

Proof of concept: Validity 25. 11. 2020 telc g. Gmb. H 14

Aims § gather data about the test § define cut-off scores § modify tasks § refine rating criteria § develop training material ü Validity argument 25. 11. 2020 telc g. Gmb. H 15

Validated test / New item pool 25. 11. 2020 telc g. Gmb. H 16

Delivery - What have we learned? ‘I liked the test. Good that you’re moving with the times and going digital. ’ (Student who took part in content trial) 25. 11. 2020 telc g. Gmb. H 17

Test Development - What have we learned? 25. 11. 2020 telc g. Gmb. H 18

Thank you! Any questions? m. hesse@telc. net s. mcdonald@telc. net r. rosenlund@telc. net 25. 11. 2020 telc g. Gmb. H 19