ICAO Circular 318 Adrian Enright EUROCONTROL ICAO Regional

ICAO Circular 318 Adrian Enright EUROCONTROL ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

ICAO Circular 318 Ø Gives more detailed guidance Ø Has recommended criteria for selection of aviation language testing programmes Ø An indispensable guideline to CAA regulators, aircraft operators and air navigation service providers ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Captain? Captain can you hear me? …have you found my copy of ICAO Circular 318? An indispensable guideline to aircraft operators … ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Circular 318 Ø ICAO LPR apply to all languages, but Ø Circular 318 focuses on English as the language for which most training and testing programmes need to be developed Ø Circular 318 will not replace the extensive standards, guidelines and principles found in the literature on language testing ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Background Ø 1998 32 nd ICAO Assembly Ø 2003 comprehensive set of SARPs ● ● ● Annex 1 Personnel Licensing Annex 6 Operation of Aircraft Annex 10 Aeronautical Communications Annex 11 Air Traffic Services PANS-ATM Ø March 5 th 2011 final deadline for implementation of LPRs ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Fundamental test principles Ø Validity Ø Reliability Ø Practicability ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Test purposes Ø Diagnostic Ø Placement Ø Progress Ø Achievement Ø Aptitude Ø Proficiency ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Test focus Ø Speaking and listening proficiency Ø High stakes test Ø Work related context Ø ICAO Rating Scale, and Ø Holistic descriptors ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Use of ICAO phraseology Ø Testing phraseology is a operational issue, but Ø Use of phraseology in a work-related context is acceptable Ø Can be used as a warm-up, or Ø In setting the context ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Recommended criteria Ø Self-contained statements with supplementary information ● What it means ● Why it is important ● Additional information ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Test Design and Construct Ø Tests should address the specific and unique requirements of aviation language testing Ø A separate listening test can provide independent information about comprehension Ø Tests should be specific to aviation operations – ● Visual AND non-visual communications ● customize tests for pilots or controllers Ø Language proficiency tests should not be designed to evaluate technical knowledge Ø The final score is the LOWEST of the six rated criteria ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Rating Ø Rating should be carried out by a minimum of two raters Ø An independent 3 rd rater when the 2 raters cannot agree between level 4 and level 3 ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Test familiarisation Ø Complete sample test freely available Ø Helps to ensure that test takers are not surprised by the format of the test and are properly able to demonstrate their language proficiency Ø Test description or a marketing brochure is not sufficient ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Test security Ø Test security is essential for ensuring reliability of, and confidence in, the testing system Ø A full description of security measures should be available to all decision makers ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Testing Team Ø Recommended 2 raters ● language specialist expertise ● operational expertise ● Should demonstrate at least the highest level of proficiency expected of test takers Ø Familiar with aviation English vocabulary and structures Ø Complete recurrent training at least once a year ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Rater reliability Ø Intra-rater reliability ● Extent to which a particular rater consistently uses a proficiency scale Ø Inter-rater reliability ● Extent to which a group of independent raters agree on a test taker’s performance ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Checklist Ø Chapter 3 provides a simple checklist Ø Cross-referenced to the sub-paragraphs Ø Assist States in determining: ● how far they have gone ● how far they have still to go in implementing valid and reliable language proficiency tests ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

What? That’s not the pre-flight checklist… Is the rating process documented? ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Glossary Ø Useful glossary of testing terms and acronyms Ø “Everything you wanted to know about testing but were afraid to ask” Ø Not quite, but a good start. ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

Let us not forget just one reason why we set off on this journey to improve aviation English Tenerife March 1977 ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010

A final thought … Ø We have come a long way in the last 12 years in improving language proficiency in aeronautical communication, but the journey is not over yet. Our commitment today is in laying the foundation for generations to come. ICAO – Regional Workshop on Language Proficiency Requirements Implementation Rome, March 3 rd to 5 th 2010 © EUROCONTROL 2010