Layering methods to analyse the relationship between language

Layering methods to analyse the relationship between language use and attainment among OU undergraduate students Maria Leedham, Elizabeth J. Erling, Lina Adinolfi Contact: m. e. leedham@open. ac. uk Faculty of Education and Language Studies The Open University

This paper will • report on a research project at the OU into the relationship between use of academic language and attainment among a small cohort of Health and Social Care undergraduate students. Layers 1 & 2 – MASUS and contextual research • explore further research which ‘layers’ methods to investigate the texts • Layers 3 & 4 - intuitive identification of lexical chunks, corpus linguistic extraction of chunks

Context: a Health and Social Care course (HSC) • K 204: Working with Children and Families • “This course is designed to meet the educational and training needs of those who work (or will work) with children and their families across social care, childcare, health, education, and leisure settings. ” • Level 2, 60 points, distance-learning course, • 6 essays + 1 exam • 26 students • 72 HSC texts were analysed • 7 students interviewed

LAYERS 1 & 2: THE ORIGINAL STUDY

Layer 1: MASUS Measuring Academic Skills of University Students Categories A Use of source material B Structure and development of text C Control of academic writing style D Grammatical correctness E Qualities of presentation Text analysis: Two language specialists and one subject specialist Rating 4 Excellent/no problems/ accurate/very appropriate 3 Good/minor problems/mainly accurate/largely appropriate 2 Only fair/some problems/often inaccurate/often inappropriate 1 Poor/major problems/inaccurate/inappropriate 1. 0 -2. 4 = low score (Bonanno and Jones 1997)

Layer 1: MASUS Findings: Attainment and language

Layer 1: MASUS Findings: MASUS scores for cohort (n=26) Assign. 1 (n=25) A Use of source material B Structure and development C D Assign. 3 (n=24) Assign. 5 7 27% 5 20% 2 8% 15 58% 14 56% 15 63% Academic writing style 8 31% 7 28% 4 17% Grammatical correctness 4 15% 7 28% 6 25% The number of students in sample with low language scores (1. 0 – 2. 4)

Layer 2: Contextual investigation • Analysis of course materials insight into important key concepts and language of the field • Tutorial visits a contact point for researcher and students • Interviews with students insights into students’ views of their own writing • Collaboration between language and discipline specialists ensured that both language and discipline expertise were represented

LAYERS 3 & 4: SEARCHING FOR LEXICAL CHUNKS

Lexical chunks • Conventionalised word combinations • Contribute to ease of language production and reception • Demonstrate membership of academic community • • Layer 3: corpus linguistic research on chunks E. g. work from Douglas Biber, Ken Hyland Layer 4: intuitive identification of chunks E. g. work from Alison Wray, Pauline Foster

Layer 3: Corpus linguistic extraction of lexical chunks • • 28, 000 words 15 HSC texts Assignments 1, 3, 5 Used Word. Smith Tools v. 5 Extracted 3, 4, 5 word chunks Compared no. of types of chunks over time Ongoing… could extend to cover keyness

Layer 3: Corpus linguistic extraction of chunks Assignment 1 : Why should we listen to children? • quality of life (31, 4/4 texts) • listening to children (17, 4) • children and families (10, 4) • right to be heard (6, 4) Assignment 3: Child development theories can have both a positive and a negative influence on how practitioners respond to children. Discuss. • in the reader (27, 3/6 texts) • negative influence on (7, 4) • as well as (5, 4) • I will discuss (5, 3) Assignment 5: Critically discuss how the quality of life of children living away from home can be improved by those working with children and their families. • living away from home (49, 4/5) • quality of life (37, 5) • it is important (8, 3) • if we are to (4, 2)

Layer 3: Corpus linguistic extraction of chunks Assignment 5 if we are to (4, 2) = Example of students taking on the role of HSC worker?

Layer 4: Intuitive identification of lexical chunks Advantage of using intuition • Can identify single chunks • Simulates the tutor’s experience Difficulties with intuition • Subjective • Difficult to justify • Problem with boundaries • Partial solution is to adopy Wray and Namba’s 11 criteria (2003) “By my judgement…” Two types of chunks focused upon: • Linkers B Structure and development of text • Discipline-specific C Control of academic writing style

Layer 4: Intuitive identification of chunks Case study A: Laura - Assignment 1: Extract Key: Red = ill-formed or inappropriate chunks Blue =well-formed and appropriate chunks

Layer 4: Intuitive identification of chunks Case study A: Laura - Assignment 5: Extract

Layer 4: Intuitive identification of chunks Case study B: Sally - Assignment 1: Extract

• Layer 4: Intuitive identification of chunks Case study B: Sally - Assignment 5: Extract Key: Red = ill-formed or inappropriate chunks Blue =well-formed and appropriate chunks

Layer 4: Intuitive identification of chunks Findings • There is a correlation between the grades given to students and the intuitive assessment of the nature of chunks they use in their writing (ie the number and range of discipline-specific chunks used appropriately) • Low achieving students appear to use fewer disciplinespecific chunks than higher achieving ones. • There appears to be an improvement in the range and number of chunks produced accurately by individual learners over time, whether low achieving or higher achieving.

Conclusions • Students need more direct language development • Investigation into the features of highly valued academic language can help in developing language support • MASUS – offers a way of gathering judgements on aspects of texts from multiple raters, including a discipline-based perspective • Contextual data – provides explanations for changes in students’ texts and differences in students’ abilities/desires to acquire and use the ‘highly valued’ features of academic language • Corpus linguistics – offers a way of quantifying the number and range of chunks across texts & gives insights into disciplinary differences • Intuitive extraction of chunks – provides a language specialist’s view of which chunks are most marked – and how this markedness might affect student attainment

References • BONNANO, H & JONES, J. (1997) Measuring the Academic Skills of University Students: The MASUS procedure. University of Sydney. • ERLING, E. J. (2009) An investigation into the relationship between the use of academic language and attainment – with a focus on students from ethnic minorities. Open University. • WIKTORSSON, M. (2003) Learning idiomaticity: a corpus-based study of idiomatic expressions in learners' written production. Lund, Lund University • WRAY, A. & NAMBA, K. (2003) Formulaic language in a Japanese-English bilingual child: a practical approach to data analysis. Japan Journal for Multilingualism and Multiculturalism, 9, 24 -51.

Acknowledgements • We would like to thank Student Services at the Open university, who funded the research project The relationship between academic language use and attainment – with a focus on students from ethnic minorities in 2008 -2009. • We would also like thank the researchers involved in the project: Elizabeth Erling, David Hann, John Kearsey, Christina Healey, Christine Buller, Kerry Bannister, Chris Lee, Zoe Doye, Harish Mehra; and the project supervisor, Jim Donohue.

Maria Leedham Centre for Language and Communication Faculty of Education and Language Studies The Open University Walton Hall Milton Keynes MK 7 6 AA m. e. leedham@open. ac. uk Dr. Elizabeth J. Erling Open English Language Teaching Dept. of Languages Faculty of Education &Language Studies The Open University Walton Hall Milton Keynes MK 7 6 AA e. j. erling@open. ac. uk Lina Adinolfi Open English Language Teaching Dept. of Languages Faculty of Education & Language Studies The Open University Walton Hall Milton Keynes MK 7 6 AA l. adinolfi@open. ac. uk