English proficiency standards in Thai universities Richard Watson

English proficiency standards in Thai universities Richard Watson Todd King Mongkut’s University of Technology Thonburi Keynote speech at the Assumption University Annual Faculty Seminar 2016

Judging students’ level of English (in the 1990 s) • Generic level – Pretty good – Lower intermediate • Context-specific level – Textbook level 3 – Fundamental English 3 level • Exam-related level – TOEFL 550 became TOEFL CBT 213, and is now TOEFL i. BT 80 • ACTFL guidelines

ACTFL guidelines • Published in 1986 • 9 levels (from Novice-Low to Superior) • Descriptors for the 4 skills Superior-level writers are characterized by the ability to: Novice-level writers are characterized by the ability to: express themselves effectively in most informal and formal writing on practical, social, and professional topics treated both abstractly as well as concretely. produce lists and notes and limited formulaic information on simple forms and documents.

Education in the 1990 s and 2010 s • 1990 s – Little quality assessment – Little time spent on bureaucratic formfilling – Academics focused on academic work – Some quality problems • 2010 s – Extensive quality assessment – Academics spend time on bureaucratic formfilling – Less time for academic work – Still some quality problems

The bureaucratization of education • In Italian universities between 1990 and 2001: – The number of researchers decreased by 17% – The number of administrative staff increased by 150% • Source: Coccia, 2009 • We can confidently predict more of what bureaucratization has been producing for many decades--less and less real learning, less and less creativity, more and more stagnation • Source: Douglas, 1991

Bureaucratization and language standards • 1990 s: ACTFL guidelines had very little impact • 2010 s: Desire for standardization of language levels • CEFR levels published in 2001 • Major impact in Europe in the 2000 s • Broadening impact throughout the world in the 2010 s

CEFR levels • CEFR = Common European Framework of Reference for Languages • Produced by the Council of Europe • Six levels: A 1, A 2 (Basic user), B 1, B 2 (Independent user), C 1, C 2 (Proficient user) • Provides descriptors for overall proficiency and each of the 4 skills

CEFR descriptors for overall proficiency • • A 2 Can understand sentences and frequently used expressions related to areas of most immediate relevance (e. g. very basic personal and family information, shopping, local geography, employment). Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters. Can describe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need. • • C 1 Can understand a wide range of demanding, longer texts, and recognise implicit meaning. Can express him/herself fluently and spontaneously without much obvious searching for expressions. Can use language flexibly and effectively for social, academic and professional purposes. Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organisational patterns, connectors and cohesive devices.

What does CE(uropean)FR have to do with Thailand? • 2010 s: Desire for standardization of language levels • Growing influence of CEFR internationally • Applicability of Can-Do statements

Standardization of Thai education quality • National Education Act of 1999 Section 53: “Teachers. . . shall have professional licenses as provided by the law. ” • “The provisions in this section shall not apply to the faculty staff. . . for higher education at the degree level. ” • Regulations of the Teachers Council of Thailand on Professional Standards and Ethics (2005): 9 areas where teachers should show knowledge (does not include language)

Standardization of language in Thai education • Ministry of Education: all secondary school English teachers should have English at CEFR C 1 level • In 2015, 43, 000 secondary school teachers were tested • 6 teachers at C 2 level • 900 teachers at B 2 or C 1 level • Proposal to create CEFR-T test for all purposes

Standardization of language in Thai higher education • Ministry of Education wants clear target levels for university staff and students • Proposal for a university exit exam • Flexibility for individual universities to use their own exams (if standardized) • Universities don’t have to use CEFR, but. . .

How to measure CEFR levels • • Claim to be at a certain level Self-evaluation Use a test designed to match CEFR Use an existing standardized test that has been equivalenced with CEFR

CEFR-T • Proposed by Ministry of Education • To be designed by the Professional Qualifications Institute • Single test for all purposes – For primary students, for university teachers, foreign teachers, for professionals • 10 levels • Goal: Prathom 3 = A 2 level; University undergraduates = B 1 level

Using existing tests • Most major tests have been equivalenced against CEFR (e. g. TOEFL, TOEIC, IELTS) • Thai-produced tests (e. g. CU-TEP, TETET) undergoing equivalencing

Equivalencing existing tests with CEFR • 62 different methods – Modified Angoff method – Combined Judgment Empirical method – Item Score String Estimation – Up-and-Down method • Experts making judgments – For each item, experts assign a probability that a B 2 level candidate would answer the item correctly

Equivalencing existing tests with CEFR • TOEFL i. BT research showing minimum TOEFL scores for each CEFR level CEFR 2008 2015 B 1 57 42 B 2 87 72 C 1 110 95

CEFR levels - problems • Descriptors are all Can-Do statements, and say nothing about what a person cannot do • The framework is based on an idiosyncratic model of communicative competence • The descriptors describe behavior only (ignoring e. g. knowledge and potential) • There is no clear link with teaching and learning • The descriptors are broad and not contextspecific • Interpretation is subjective

CEFR levels - benefits • Clear, easy-to-understand descriptors of expected language performance at different levels • Default way of describing language proficiency

Thai higher education context • Evaluation, measurement and standards are emphasized • CEFR is the default approach to measuring language proficiency

CEFR at KMUTT • Applicants for academic staff • Ph. D applicants • Undergraduate and Masters students dependent on type of program and individual department

CEFR and applicants for academic staff at KMUTT • Applicant must show evidence of level of English proficiency on application • 4 -skills test with academic orientation only • Test must be equivalenced with CEFR • IELTS, TOEFL i. BT, TETET • Goal: C 1 level

Measuring CEFR level for KMUTT academic staff • Identifying test scores for C 1 level • Different sources give different scores CEFR IELTS TOEFL i. BT C 1 6. 5 – 7. 5 7 -8 79 – 95 95 – 120 110 - 120 • Judging which score to use – IELTS 6. 5; TOEFL i. BT 85

CEFR and applicants for academic staff at KMUTT Test before application B 1 or lower B 2 C 1 or higher STOP Continue application If accepted, 3 years’ probation to improve to C 1 English language improvement support

CEFR and KMUTT undergraduates • Individual departments set their own standards • Some departments require standardized test scores for graduation • Typically high B 1 or B 2 (e. g. IELTS 4. 5 or 5) • TETET as university exit exam, but no minimum requirements

CEFR and Thai universities • CEFR is simply a set of descriptors of language proficiency at different levels and is not a panacea • Using CEFR reinforces the emphasis on measurement, standardization and bureaucratization in education • Judgments of CEFR levels are subjective • CEFR provides a sensible, readily understandable common language to help make decisions