RELANG Relating language examinations to the common European
- Slides: 10
RELANG Relating language examinations to the common European reference levels of language proficiency: promoting quality assurance in education and facilitating mobility From Learning to Testing European Centre for Modern Languages and European Commission cooperation on Innovative Methodologies And Assessment In Language Learning
Linking Procedures in the Manual • Familiarisation with the CEFR • Linking on the basis of specification of examination content • Standardization and Benchmarking • Standard setting • Validation: checking that exam results relate to CEFR levels as intended
Claims through Specification and Standardization • Claimed link to CEFR is based on specification only – no indication of the quality of a performance or the score on a test required for CEFR-level in question • Claims to be substantiated through standardisation – Learning objectives in relation to CEFR levels.
Steps in the standardization phase 1. Adequate familiarisation with the CEFR. 2. Training in rating productive skills • tables and scales in the CEFR or the Manual & scales or specific rating scales 3. Training in rating receptive tasks • tables in the Manual & specifications developed for the examinations or tests in question 4. Benchmarking performance samples 5. Standard setting of receptive tasks
Group Decisions • Benchmarking and standard setting are procedures requiring group decisions – to be carefully prepared by appropriate training.
Standard Setting for Receptive Skills • For Learning Objectives that are tested through indirect tests with a numerical score, performance standards have to be set – Receptive skills (reading, listening) – Underlying competences (grammar, vocabulary) • A performance standard is the boundary (cut-off score) between two levels on the scale reported by a test • Process to arrive at a cut-off score: standard setting
How to arrive at Standards? • • • Group decisions (panel) Group is familiar with CEFR Test content has been specified in terms of the CEFR Standard setting procedures have been formalized Careful selection and training of panel members
Benchmarking of Productive Skills The judgment on the CEFR level is direct – Assistance needed for raters in giving valid judgments Benchmarking: providing one (or more) typical sample(s) to illustrate performance at a given level
Validation What is Validation? • The degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests. • Does the test measure what it intends to measure?
Content Validity • Does the test accurately reflect the syllabus on which it is based AND reflect the descriptors in the CEFR? • Does the content specification reflect all areas to be assessed in suitable proportions?
