The impact of the CEFR on language testing

The impact of the CEFR on language testing (and assessment): looking back to move forward Neus Figueras nfiguera@xtec. cat

Why are we here? • • Impact of the CEFR in Europe and beyond Language policies in our respective countries Share views, experiences and worries Work towards a better understanding of linking procedures

And also…. .

There was the Barcelona Alte meeting 2001…. And there was the Helsinki meeting 2002…. Is my B 1 like your B 1?

cations “It is imperative that examinations today are q Honest q Reliable q Valid q Transparent q Portable “ John Trim ( 1924 -2013) John Trim’s address to ACTFL-CEFR Conference in Provo, USA. August 4 th, 2011

The 2003 Pilot Manual

The CEFR in language testing & assessment Meetings and Seminars - April 2004 Strasbourg - November 2007 Cambridge - May 2008 Athens - June 2008 Sèvres Projects – CEFR Dutch Grid Relex Relang Documents DVD with Eurocentres performances DVD with CIEP performances (French) ALTE written performances Co. E Item CD Co. E CIEP performances (5 languages) . . /. . Publications

“The Common European Framework of Reference for Languages (CEFR) and the development of language policies: challenges and responsibilities” Intergovernmental Language Policy Forum Strasbourg, 6 -8 February 2007 REPORT Francis Goullier

Recommendation CM/Rec(2008)7 of the Committee of Ministers to member states on the use of the Council of Europe’s Common European Framework of Reference for Languages (CEFR) and the promotion of plurilingualism (Adopted by the Committee of Ministers on 2 July 2008 at the 1031 st meeting of the Ministers’ Deputies)

Standard setting in Practice: How to cut the mustard Barcelona 2007 Turku 2007 Budapest 2008 Siena 2010 Copenhagen 2013 Figueras et al. (2013) Relating a test of Reading comprehension to the CEFR levels. A case of standard setting in practice with a focus on judges and items.

The 2009 Manual January 2009 Following the best traditions of Council of Europe Relating Language Examinations to action in promoting language education, however, the Common European Framework of Reference for Languages: Learning, the Manual has wider aims to actively promote and Teaching, Assessment (CEFR) facilitate co-operation among relevant institutions and Language Policy Division experts in member countries. The Manual aims to: www. coe. int/lang • contribute to competence building in the area of linking assessments to the CEFR; • encourage increased transparency on the part of examination providers; • encourage the development of both formal and informal national and international networks of institutions and experts. (Manual 2009 : 1)

Steps • • • Chapter 1: The CEFR and the Manual Familiarisation Chapter 2: The Linking Process Specification Chapter 3: Familiarisation Standardisation Chapter 4: Specification What Chapter 5: Standardisation & Benchmarking Chapter 6: Standard setting procedures Standard setting Chapter 7: Validation Appendix A. Forms and scales for description and specification Who Appendix B. Content Analysis Grids Selecting the Panel Appendix C. Forms and scales for How Selection of approach standardisation and benchmarking Completion of procedures How much/many

Building an argument Familiarisation with the CEFR

The Manual offers… . . . guidance to users to … • describe the examination coverage, administration and analysis procedures; • relate results reported from the examination to the levels presented in Chapter 3 of the CEFR; • provide supporting evidence that reports the procedures followed to do so , BUT • it is not the only book for linking exams or systems to reference levels, (Cizek & Bunch 2007; Zieky, Perie & Livingston 2008, …);

Not easy…. 18

Policy Definition Mandatory Curricula Elaborated Description Test Content Cut Score on Test Reckase 2008

How does the Manual help? G O O D P R A C T I C E D T R A N S P A R E N C Y • Content analysis of an examination • Relating to the CEFR content and scales • Profiling to CEFR levels • Claim of linkage O C U M E N T A T I O N

Guidelines for Good Practice in Language Testing and Assessment

Guidelines for Good Practice in Language Testing and Assessment

Guidelines for Good Practice in Language Testing and Assessment

So……. . • What’s my B 1 like? • What’s your B 1 like? Thank you