LEVEL 4 IAW STANAG 6001 A CONCEPTUAL MODEL

LEVEL 4 IAW STANAG 6001 - A CONCEPTUAL MODEL Jana Vasilj-Begovic BILC Secretary/Canada Gerard Seinhorst Head, Language Office, Defence Institute Security & Intelligence, Netherlands

SCOPE OF PRESENTATION Background Methodology Conceptual Framework Implications for Testing Way ahead

Background BILC conference 2010 – Istanbul Formation of level 4 WG Members: Bulgaria, Canada, Denmark, Germany, Italy, Netherlands, Shape, Sweden, USA

Methodology • Survey and compile literature • Develop a conceptual framework • Find illustrative samples • Discuss testing implications Provide guidelines for test development

Conceptual Framework Aims to answer the questions: o how to interpret the STANAG 6001 descriptors for Level 4? o what are the typical and shared (linguistic) characteristics of Level 4 language users? o what are the implications for assessing Level 4 language ability? Purpose: to enhance the common understanding of Level 4 proficiency.

Level 4 Characteristics • Linguistically complex • Low redundancy • Multiple meanings in socio-cultural context • Rarely encountered in daily life • Unfamiliar cultural concepts • Highly individualized / sophisticated Ø interpretation of the layers of explicit and implicit meaning

Types of Comprehension • Literal • Inferential • Evaluative Ø Understanding meaning “beyond the lines” Ø Evaluating the significance of the message

The Level 4 Language User STANAG 6001: “Expert” This level of proficiency is typically, although not necessarily and not exclusively, achieved by individuals who use the foreign language extensively to handle intellectually highly demanding issues as part of their profession or specialisation

The Level 4 Language User • Very taxing academic and professional settings • Deal with demanding topics at the highest levels • Examples of military-related tasks: Ø Spokesperson responsible for press releases and press conferences requiring nuanced, culturally appropriate communications • Arbiter between warring factions • Providing official interpretation services • Analyzing the hidden communicative intent of diplomatic pronouncements

The Level 4 Language User • Adapts and tailors language to suit the purpose and situation • Understands/expresses subtleties and shades of meaning • Consistently maintain high degree of grammatical and lexical accuracy • Uses metaphors, clichés and proverbs spontaneously • May have fossilized lapses in pronunciation and intonation

Significant Differences (1) LEVEL 3 LEVEL 4 • Understands/Can produce formal and informal language, for most everyday social and work-related situations. • Understands/Can produce highly sophisticated language appropriate for almost all topics, and social as well as professional situations. • Understands/Can produce language relating to abstract concepts and hypotheses. • Readily adapts language to any situation. Shows a firm grasp of various levels of style and can understand/express subtle nuances and shades of meaning. • Uses the language with great precision for all social and professional purposes, including persuasion, negotiation, or the representation of an official point of view. • Understands/Can produce extended discourse and conveys meaning correctly and effectively. Understands arguments/Can argue for and against different options.

Significant Differences (2) LEVEL 3 LEVEL 4 • Can understand/express meaning “between the lines”. • Can understand/express meaning “beyond the lines”. • Understands/Demonstrates humour and irony, but may not fully understand some allusions, as well as implications of nuance and idioms. • Good command of idiomatic expressions, figures of speech and colloquialisms. • Rarely requests repetition. Has a natural flow, without searching for words. • Consistently maintains a high degree of grammatical and lexical accuracy. Uses the language effortlessly. • Occasional errors rarely disturb the native listener/reader. • Errors are rare and without pattern.

Implications for Testing receptive skills (listening and reading) Tasks Text type Sample Testing techniques

Text Type (listening) Adaptation of Clifford’s Text Characteristics Speaker Purpose Typical Text Type Listener Purpose PROJECT lines of thought beyond the expected; connect previously unrelated ideas or concepts; present complex ideas with nuanced precision and virtuosity. Extended discourse that is tailored for the message and the intended audience. To achieve the desired tone and precision of thought, the speaker will often skilfully use lowfrequency vocabulary, cultural and historical concepts, and demonstrate understanding of the audience’s shared experience and values. Listen “beyond the lines”, understand the speaker’s sociolinguistic and cultural references, follow innovative turns of thought, and interpret the message in view of its wider cultural, societal, and political setting.

Text Type (listening) Adaptation of Clifford’s Text Characteristics Speaker Purpose Typical Text Type Listener Purpose PROJECT lines of thought beyond the expected; connect previously unrelated ideas or concepts; present complex ideas with nuanced precision and virtuosity. Extended discourse that is tailored for the message and the intended audience. To achieve the desired tone and precision of thought, the speaker will often skilfully use lowfrequency vocabulary, cultural and historical concepts, and demonstrate understanding of the audience’s shared experience and values. Listen “beyond the lines”, understand the speaker’s sociolinguistic and cultural references, follow innovative turns of thought, and interpret the message in view of its wider cultural, societal, and political setting.

Text Type (listening) Non-participatory Situations Speaker Purpose Typical Text Type Listener Purpose Analyse issues, present complex or novel ideas or arguments on professional, political, economic, cultural or societal issues (through public media, overheard conversations). Lectures, broadcast interviews and discussions, briefings, conference presentations. For example: an interview contrasting critical thinking with emotional intelligence. Understand unpredictable shifts of subject matter and tone; understands highly sophisticated language and references embedded in a wider sociocultural context.

Text Type (listening) Participatory Situations Speaker Purpose Elaborate on various highly abstract topics Typical Text Type Negotiations Debates Professional meetings. Counsel/persuade or For example: influence using influencing others to nuanced language change a policy specifically tailored to position the listener(s). Listener Purpose Understand highly complex discourse and all styles of speech (highly formal and abstract or highly informal/colloquial). Respond and contribute satisfactorily, although not always in the projective mode.

Text Type (reading) Clifford’s Overview of Text Characteristics PROJECT lines of thought beyond the expected; connect previously unrelated ideas or concepts, or present complex ideas with nuanced precision and virtuosity. Extended discourse that is tailored for the message being sent and the intended audience. To achieve the desired tone and precision of thought, the author will often demonstrate the skilful use of lowfrequency vocabulary, cultural and historical concepts, and an understanding of the audience. Read “beyond the lines”, understand the author’s sociolinguistic and cultural references, follow innovative turns of thought, and interpret the text in view of its wider cultural, societal, and political setting.

SAMPLE LISTENING TEXT Extracted from the Charlie Rose Show Topic: Science and Health http: //www. charlierose. com/view/interview/1 2276

SAMPLE READING TEXT AN AMORAL MANIFESTO, Joel Marks (Philosophy Now) Hold onto your hats, folks. Although it is perhaps fitting that the actual day on which I sit here at my computer writing this column is April 1 st, let me assure you that I do not intend this as a joke. For the last couple of years I have been reflecting on and experimenting with a new ethics, and as a result I have thrown over my previous commitment to Kantianism. In fact, I have given up morality altogether! This has certainly come as a shock to me (and also a disappointment, to put it mildly). I think the time has come, therefore, to reveal it to the world, and in particular to you, Dear Reader, who have patiently considered my defences of a particular sort of moral theory for the last ten years. In a word, this philosopher has long been labouring under an unexamined assumption, namely, that there is such a thing as right and wrong. I now believe there isn’t.

Testing Techniques Alignment of text and task level Constructed response vs. multiple choice What line of argumentation does the author present for his relinquishment of morality? One skill vs. combined skill testing Rating decisions

Implications for Testing productive skills (speaking and writing) Tasks Sample Testing techniques

What L 4 Speakers/Writers Can Do • Tailor speech to a specific audience • Counsel • Persuade • Advocate a position at length • Elaborate on abstract concepts • Express nuances and shades of meaning

Sample of Speaking Proficiency • Task: discuss abstract topic • Testing/elicitation technique (OPI): descriptive prelude • Level of task aligned with level tested

Sample of Writing Proficiency • Task: analyse abstract issue (NATO related topic) • Testing technique/writing prompt: descriptive prelude • Level of task aligned with level tested

SAMPLE WRITING PROMPT One of the most significant trends in recent decades in the arena of international military and political affairs has been the perceived change in the scope and purpose of NATO commitments and engagements. In the post-Soviet Era, the mission of NATO has apparently shifted from “defending the free world” to “saving the innocent from ethnic cleansing or genocide”—whether at the hands of their own rulers or of neighboring nations—through intervention in an armed struggle on the side of a party chosen on the grounds of moral righteousness and justice. The policy of intervening to prevent genocide or ethnic cleansing is seen as the moral equivalent of humanitarian aid to save innocent human lives.

Sample of Writing Proficiency In the year 2000, the UN published the Brahimi report on peacekeeping and one of its probative findings was that peacekeeping was significantly more effective, when there was regional ownership of the mission. There are several reasons for this, but one of the major contributing factors is that few conflicts can be adequately solved, unless there is a solid regional backing for the peace process. Additionally, peacekeeping soldiers from the same region and culture are generally much more attuned to the cultural particularities of any given conflict zone and as such they may be better able to understand confront the underlying cleavages in a society torn by war and atrocities.

Implications for Testing • Finding appropriate texts • Selecting appropriate testing techniques • Determining proficiency/performance ratio • Sampling widely to elicit a representative/ratable sample of proficiency • Reconciling requirements for wide sampling with those of practicality • Validating (target population) • Training testers/raters

WG Status • Paper finalized and disseminated (available on BILC Website www. natobilc. org) • Format and protocol for level 4 testing under consideration • Pilot project of level 4 testing protocol planned

QUESTIONS THANK YOU!