Study Group 4 Best Practices in STANAG 6001

Study Group #4 Best Practices in STANAG 6001 Testing Peggy Garza Birgitte Grande

Best Practices in STANAG 6001 Testing Nations Represented • • • Bosnia and Herzegovina Canada Czech Republic Denmark France Germany Georgia Hungary Italy • • • Lithuania Netherlands Norway Slovenia Spain Sweden Ukraine United Kingdom United States of America

Best Practices in Testing • Considered the European Association of Language Testing and Assessment (EALTA) – Guidelines for Good Practice in Language Testing and Assessment • Discussed the uniqueness of STANAG 6001 testing best practices • BILC documents – Self Assessment for STANAG 6001 Testing Programs – LTS materials: Test Development Process – Roadmap to a Validity Argument

Four Categories • • Test purpose Test design Test development procedures Test administration procedures

STANAG 6001 Test Purpose The purpose of STANAG 6001 proficiency tests can be viewed as three inseparable tasks • Assess an individual’s general language proficiency using the criteria captured in STANAG 6001’s 0 through 5 proficiency scales, i. e. measure the individual’s ability to consistently complete the real-world communication tasks in the specified situations with the level of accuracy expected in those situations. • Determine which STANAG 6001 level describes the level of sustained ability. • Report that proficiency level to the appropriate stakeholders. • Because the purpose of STANAG 6001 tests is to assess an individual’s spontaneous abilities in frequently-occurring real-world communicative settings, STANAG 6001 tests are different from curriculum-based tests

STANAG 6001 Test Design Test specifications should include (not necessarily in the following order): The purpose of the test, The target testing population, What is going to be tested (skills, subskills), Levels according to STANAG 6001 (single, bi-, multi-level test) Format of the test (how long, how many texts, tasks, items…), Criterion referenced testing, Types of tasks (MCQ, open –ended, matching, writing and speaking tasks…), • The language of tasks (mother tongue, target language), • List of topics, • Authenticity of the materials • •

STANAG 6001 Test Development Test Construction Process General recommendations • Item writers should not work on their own, but in and as a team • Before the team starts developing items, there should be consensus about what and how to test (as laid out in the test specs) • Make sure that each item and prompt measures unrehearsed general language proficiency, and not discrete-point grammar and vocabulary, or curriculum-based performance. Quality control measures • Keep the testing section separated from the teaching section in order to avoid conflicts of interest • Item writers should be properly trained and (re)normed against the STANAG scale • Keep all testing materials secure at all times and provide access only on a need to know basis • Update items regularly to avoid them becoming outdated or compromised •

STANAG 6001 Test Administration Procedures Prior to test administration • • Familiarization – on MOD websites, clear instructions prior to the exam To achieve proper planning, incl. accommodation etc. – Is a formal schedule published with the same time, date, and location of the testing sessions? Are the proctors given an official list of approved names prior to the testing session? Is there a clear policy to prevent and penalize cheating of any kind? Test administration • • • Ensure all candidates have the same conditions Teachers should not administer tests to their own students Regulations concerning the number of speaking tests conducted each day All speaking tests should be recorded Develop security procedures – preventing test compromise, storing of test materials, Item database, archive, prevent the use of electronic devices (smartwatch etc. )

Best Practices in STANAG 6001 Testing Thank you for your input. This is a work in progress. The final product will be posted on the BILC website.