Bureau for International Language Coordination Language Needs Analysis

Bureau for International Language Coordination Language Needs Analysis of NATO Posts Part 1 Julie J. Dubeau Associate BILC Secretary Chief Standards, Foreign Language Programmes, Canadian Defence Academy www. NATOBILC. org Madrid, 4 May 2015

Bureau for International Language Coordination Language needs analysis (LNA) background • Problem statement from SACEUR • LNA procedure & methodology • Preliminary observations • Next steps… www. NATOBILC. org

LNA BACKGROUND • November 2008 Chairmen’s Meeting of the NTG in Brussels results in tasking to conduct: LANGUAGE NEEDS ANALYSIS OF NATO CRISIS ESTABLISHMENT (CE) POSTS • The aim of this study was to show whether language requirements appeared to be set at appropriate levels to enable military personnel to perform their duties adequately in the NATO OPS context, in this case ISAF. www. NATOBILC. org

FINDINGS LNA 2009 BILC STUDY Ratings Total = 609 CE Post JDs A - At Level 355 (58. 2%) H - High 203 (33. 3%) L - Low 41 (6. 7%) H/L - Mixed 10 (1. 6%)

PROBLEM STATEMENT FROM SACEUR'S ANNUAL GUIDANCE ON NATO EDUCATION – SAGE 2014 “Language competencies are identified as an important shortfall and actions are needed to mitigate and resolve this problem. ” & “ACT is requested to conduct an assessment of the language requirements for collective groups, NATO posts, along with the review of Job Descriptions. ” ACT helped organize & subsidize 1 st visit, with support from Vice Admiral Javier Gonzalez-Huix, DCOS Joint Force Trainer www. NATOBILC. org

LANGUAGE NEEDS ANALYSIS (LNA) BACKGROUND • What are the issues? • • Standardized Language Profiles (SLPs) are set ‘arbitrarily’ by HR managers Inflated requirements (req) may lead to inflated SLPs (and test scores) Nations have difficulty training to unrealistically high requirements Inflated req/SLPs undermine the validity of the job descriptions themselves • Some misconceptions: • • As rank increases so do language proficiency requirements and abilities Military (subject matter) expertise can substitute for language proficiency Most individuals can get ‘trained’ to level 3 (and 4) Once members are on post, and after a while…

LNA PROCEDURE & METHODOLOGY Goal of exploratory study: to examine whethere is concordance between language requirements and actual language proficiency required for the successful performance of work-related tasks in the NATO/SHAPE workplace. Phase 1 – Pre-analysis Jan 2015: • • Questionnaire & interview protocols developed & framework set-up Representative posts selected Questionnaires sent to respondents in advance Respondents scan JD and submit for review/coding of language functions/levels • Respondents submit samples of their written work, (first drafts of Point Papers, letters, directives etc. ) • Reading materials scanned & sent for review/coding - reports, papers, correspondence, directives, etc. www. NATOBILC. org

LNA PROCEDURE & METHODOLOGY • Most respondents should have SLP 4 in JD (ex. 4343), others 3333 • They should have been working at their posts for at least 6 months in order to have intimate knowledge of what is required of the job. • They and their superiors (either Section, Branch or Division/Directorate Head who can assess their work) should be present at HQ at the time of LNA. • Respondents filled in Q ranking tasks in terms of complexity, importance and frequency. • Respondents provide BILC SMEs with national SLP certificate • BILC provides a standard interview protocol for all respondents and their supervisors • SMEs check task on the JD in terms of complexity, importance and frequency and numerically rate the tasks in terms of minimal language required. • SMEs interview the individuals / supervisors to see if language ability matches requirements and their performance on the job

PRELIMINARY OBSERVATIONS… • • • 31 officers, 3 NCO interviewed + 26 supervisors… Ranks from Capt. to Col… 5 Directorates, 4 divisions, 14 sections… 15 NATO nations + 3 Partner nations… Majority of SLPs of positions reviewed were at 3333 (3 were at 4343, 3 at 3322)… • • • Job descriptions seem to have accurate SLPs Officers interviewed did require L 3 proficiency Majority of respondents had SLPs matching (or exceeding JD req on paper Not all interviewees seemed to have proficiency reflected on their certificates… No L 4 reading or writing material was submitted… www. NATOBILC. org

NEXT STEPS • Individuals are tested against the BAT (anonymously for ACT but not for BILC) • Phase 2 repeated at SHAPE and NATO HQ/IMS • All data is compiled/analysed and coded (so that no individual, country, section, branch or HQ is identified) • Report drafted and presented to ACT. www. NATOBILC. org