Graduate Outcomes of Undergraduate Students with English as

  • Slides: 43
Download presentation
Graduate Outcomes of Undergraduate Students with English as an Additional Language (EAL): Patterns and

Graduate Outcomes of Undergraduate Students with English as an Additional Language (EAL): Patterns and Implications DR PAMELA HUMPHREYS MACQUARIE UNIVERSITY ENGLISH LANGUAGE CENTRE UECA Assessment Symposium 2018

Overview Introduction Methods Results Discussion

Overview Introduction Methods Results Discussion

Introduction

Introduction

Introduction Literature • English language proficiency (ELP) in higher education (Arkoudis & Doughney, 2014;

Introduction Literature • English language proficiency (ELP) in higher education (Arkoudis & Doughney, 2014; Birrell et al. , 2006; IEAA, 2013) • Exit testing (Craven, 2011; Humphreys & Mousavi, 2010; Knoch, Rouhshad, Oon & Storch, 2015; O’Loughlin & Arkoudis, 2009)

Introduction Regulatory environment • Principles and standards (AUQA, 2012; DEEWR, 2008; Barthel, 2012) •

Introduction Regulatory environment • Principles and standards (AUQA, 2012; DEEWR, 2008; Barthel, 2012) • Focus on ELP by national regulators (Australian Government, 2015; Australian Government, 2015)

Introduction: Research Questions RQ 1 • What are the graduating outcomes of undergraduate international

Introduction: Research Questions RQ 1 • What are the graduating outcomes of undergraduate international EAL students (as measured by IELTS (Academic) and GPA)? RQ 2 • Is there any systematic variability in these graduating outcomes correlating with the variables of i) gender, ii) first language, iii) discipline, iv) entry pathway to the University, or v) reason for taking the IELTS test?

Methods

Methods

Methods: Data collection • Single site Australian university • Undergraduate EALs • IELTS (Academic)

Methods: Data collection • Single site Australian university • Undergraduate EALs • IELTS (Academic) exit test • Voluntary; subsidised • Captive sample • n = 564

Methods: Data collection • ID • Date of birth • Gender • First language

Methods: Data collection • ID • Date of birth • Gender • First language • Entry pathway • Undergraduate degree program • Grade Point Average (GPA) at graduation • IELTS (Academic) at graduation • Reason for taking IELTS test

Methods: Data analysis • R Statistical Software version 3. 2. 1 • Descriptive statistics

Methods: Data analysis • R Statistical Software version 3. 2. 1 • Descriptive statistics • Preliminary data plots • Linear regression (Casella & George, 1992; Kaplan, 2014; Madsen & Thyregod, 2011)

Methods: 2 Dependent Variables IELTS (Academic) 0 -9 Grade Point Average (GPA) 0 -7

Methods: 2 Dependent Variables IELTS (Academic) 0 -9 Grade Point Average (GPA) 0 -7

Methods: 5 Independent Variables IELTS Academic Grade Point Average (GPA) Gender First language Discipline

Methods: 5 Independent Variables IELTS Academic Grade Point Average (GPA) Gender First language Discipline Entry pathway Reason for taking the IELTS test

Methods: Independent Variables IELTS Academic Gender (2) First language (43 aggregated* to 8) Discipline

Methods: Independent Variables IELTS Academic Gender (2) First language (43 aggregated* to 8) Discipline (4) Entry pathway (4) Reason for taking the IELTS test (4) Male English Business Proficiency test Migration Female Chinese Health Employment Vietnamese Science & Technology Language of Instruction (LOI) Korean *Indian languages Grade Point Average (GPA) *African languages *Indo-European languages *Austronesian languages Arts & Social Sciences Diploma pathway partner (PP) Direct Entry Program pathway (DEP) Further study Other

Results

Results

Results: descriptive statistics

Results: descriptive statistics

IELTS Overall Scores (n = 564) Variable IELTS Overall (0 -9) n 564 Minimum

IELTS Overall Scores (n = 564) Variable IELTS Overall (0 -9) n 564 Minimum 4. 0 Maximum 9. 0 Mean 6. 61 Median SD 6. 5 . 85 N. B. Undergraduate entry level at time of data collection was IELTS 6. 0

IELTS Overall Scores (n = 564) Variable IELTS Overall (0 -9) n 564 Minimum

IELTS Overall Scores (n = 564) Variable IELTS Overall (0 -9) n 564 Minimum 4. 0 Maximum 9. 0 Mean 6. 61 Median SD 6. 5 . 85 Paired t-test: (M = 6. 91, SD =. 78), t(86) = 6. 34, p <. 000 (two-tailed)

Cross-tabulation (n = 86) PRE/POST SCORES (χ2 (28, n = 86) = 87. 09,

Cross-tabulation (n = 86) PRE/POST SCORES (χ2 (28, n = 86) = 87. 09, p =. 000) Mean increase of. 38 of a band score

GPA Scores (n = 564) Variable GPA (0 -7) n 564 Minimum 2. 96

GPA Scores (n = 564) Variable GPA (0 -7) n 564 Minimum 2. 96 Maximum 6. 9 Mean 4. 67 Median 4. 57 SD. 77

Correlation between GPA & IELTS Variable GPA Note: ***p <. 001 GPA Listening 1

Correlation between GPA & IELTS Variable GPA Note: ***p <. 001 GPA Listening 1 . 39*** Reading . 41*** Writing . 32*** Speaking . 19*** Overall IELTS . 38***

Linear regression

Linear regression

Coefficients of linear regression (IELTS)

Coefficients of linear regression (IELTS)

Coefficient Intercept Females Males Business Mean LCI UCI 8. 14 7. 83 8. 45

Coefficient Intercept Females Males Business Mean LCI UCI 8. 14 7. 83 8. 45 0 -0. 14 -0. 27 -0. 02 0 English 0 African -0. 33 -0. 65 0. 00 Austronesia -0. 78 -1. 18 -0. 38 Chinese -1. 45 -1. 72 -1. 18 Indian -0. 91 -1. 20 -0. 62 Indo European -0. 35 -0. 67 -0. 03 Korean -1. 39 -1. 70 -1. 10 Vietnamese -1. 56 -1. 90 -1. 20 Social sciences -0. 06 -0. 29 0. 17 Health -0. 18 -0. 38 0. 03 0. 15 -0. 04 0. 35 0 -0. 57 -0. 81 -0. 32 LOI -0. 18 -0. 35 0. 00 PP -0. 40 -0. 57 -0. 24 0 Further study -0. 23 -0. 42 -0. 06 Migration -0. 04 -0. 19 0. 11 Other 0. 03 -0. 17 0. 23 Sigma 0. 65 0. 62 0. 70 Science/Technology TEST DEP Employment in Australia

Coefficient Intercept Females Males Business Mean LCI UCI 8. 14 7. 83 8. 45

Coefficient Intercept Females Males Business Mean LCI UCI 8. 14 7. 83 8. 45 0 -0. 14 -0. 27 -0. 02 0 English 0 African -0. 33 -0. 65 0. 00 Austronesia -0. 78 -1. 18 -0. 38 Chinese -1. 45 -1. 72 -1. 18 Indian -0. 91 -1. 20 -0. 62 Indo European -0. 35 -0. 67 -0. 03 Korean -1. 39 -1. 70 -1. 10 Vietnamese -1. 56 -1. 90 -1. 20 Social sciences -0. 06 -0. 29 0. 17 Health -0. 18 -0. 38 0. 03 0. 15 -0. 04 0. 35 0 -0. 57 -0. 81 -0. 32 LOI -0. 18 -0. 35 0. 00 PP -0. 40 -0. 57 -0. 24 0 Further study -0. 23 -0. 42 -0. 06 Migration -0. 04 -0. 19 0. 11 Other 0. 03 -0. 17 0. 23 Sigma 0. 65 0. 62 0. 70 Science/Technology TEST DEP Employment in Australia

Coefficient Intercept Females Males Business Mean LCI UCI 8. 14 7. 83 8. 45

Coefficient Intercept Females Males Business Mean LCI UCI 8. 14 7. 83 8. 45 0 -0. 14 -0. 27 -0. 02 0 English 0 African -0. 33 -0. 65 0. 00 Austronesia -0. 78 -1. 18 -0. 38 Chinese -1. 45 -1. 72 -1. 18 Indian -0. 91 -1. 20 -0. 62 Indo European -0. 35 -0. 67 -0. 03 Korean -1. 39 -1. 70 -1. 10 Vietnamese -1. 56 -1. 90 -1. 20 Social sciences -0. 06 -0. 29 0. 17 Health -0. 18 -0. 38 0. 03 0. 15 -0. 04 0. 35 0 -0. 57 -0. 81 -0. 32 LOI -0. 18 -0. 35 0. 00 PP -0. 40 -0. 57 -0. 24 0 Further study -0. 23 -0. 42 -0. 06 Migration -0. 04 -0. 19 0. 11 Other 0. 03 -0. 17 0. 23 Sigma 0. 65 0. 62 0. 70 Science/Technology TEST DEP Employment in Australia

Posterior estimates of linear regression for First Language (IELTS) English

Posterior estimates of linear regression for First Language (IELTS) English

Posterior estimates of linear regression for Entry pathway (IELTS) Proficiency test

Posterior estimates of linear regression for Entry pathway (IELTS) Proficiency test

Coefficients of linear regression (GPA)

Coefficients of linear regression (GPA)

Coefficient Intercept Females Males Business African Mean LCI UCI 8. 14 4. 80 5.

Coefficient Intercept Females Males Business African Mean LCI UCI 8. 14 4. 80 5. 50 0 -0. 17 -0. 31 -0. 03 0 -0. 41 -0. 78 -0. 04 Austronesia -0. 17 -0. 61 0. 29 Chinese -0. 27 -0. 57 0. 03 Indian -0. 52 -0. 84 -0. 19 0. 05 -0. 30 0. 42 Korean -0. 26 -0. 60 0. 08 Vietnamese Indo European -0. 02 -0. 42 0. 38 Social sciences 0. 07 -0. 20 0. 33 Health 0. 05 -0. 19 0. 28 Science/Technology 0. 32 0. 10 0. 53 0 DEP -0. 32 -0. 60 -0. 04 LOI -0. 12 -0. 31 0. 08 PP -0. 22 -0. 41 -0. 03 0 -0. 09 -0. 29 0. 10 -0. 11 -0. 27 0. 06 0. 14 -0. 08 0. 36 0 0. 74 0. 70 0. 79 TEST Employment in Australia Further study Migration Other English Sigma

Coefficient Intercept Females Males Business African Mean LCI UCI 8. 14 4. 80 5.

Coefficient Intercept Females Males Business African Mean LCI UCI 8. 14 4. 80 5. 50 0 -0. 17 -0. 31 -0. 03 0 -0. 41 -0. 78 -0. 04 Austronesia -0. 17 -0. 61 0. 29 Chinese -0. 27 -0. 57 0. 03 Indian -0. 52 -0. 84 -0. 19 0. 05 -0. 30 0. 42 Korean -0. 26 -0. 60 0. 08 Vietnamese Indo European -0. 02 -0. 42 0. 38 Social sciences 0. 07 -0. 20 0. 33 Health 0. 05 -0. 19 0. 28 Science/Technology 0. 32 0. 10 0. 53 0 DEP -0. 32 -0. 60 -0. 04 LOI -0. 12 -0. 31 0. 08 PP -0. 22 -0. 41 -0. 03 0 -0. 09 -0. 29 0. 10 -0. 11 -0. 27 0. 06 0. 14 -0. 08 0. 36 0 0. 74 0. 70 0. 79 TEST Employment in Australia Further study Migration Other English Sigma

Coefficient Intercept Females Males Business African Mean LCI UCI 8. 14 4. 80 5.

Coefficient Intercept Females Males Business African Mean LCI UCI 8. 14 4. 80 5. 50 0 -0. 17 -0. 31 -0. 03 0 -0. 41 -0. 78 -0. 04 Austronesia -0. 17 -0. 61 0. 29 Chinese -0. 27 -0. 57 0. 03 Indian -0. 52 -0. 84 -0. 19 0. 05 -0. 30 0. 42 Korean -0. 26 -0. 60 0. 08 Vietnamese Indo European -0. 02 -0. 42 0. 38 Social sciences 0. 07 -0. 20 0. 33 Health 0. 05 -0. 19 0. 28 Science/Technology 0. 32 0. 10 0. 53 0 DEP -0. 32 -0. 60 -0. 04 LOI -0. 12 -0. 31 0. 08 PP -0. 22 -0. 41 -0. 03 0 -0. 09 -0. 29 0. 10 -0. 11 -0. 27 0. 06 0. 14 -0. 08 0. 36 0 0. 74 0. 70 0. 79 TEST Employment in Australia Further study Migration Other English Sigma

Posterior estimates of linear regression for First Language (GPA) English

Posterior estimates of linear regression for First Language (GPA) English

Posterior estimates of linear regression for Entry pathway (GPA) Proficiency test

Posterior estimates of linear regression for Entry pathway (GPA) Proficiency test

Discussion

Discussion

Research Questions RQ 1 • What are the graduating outcomes of undergraduate international EAL

Research Questions RQ 1 • What are the graduating outcomes of undergraduate international EAL students (as measured by IELTS (Academic) and GPA)? RQ 2 • Is there any systematic variability in these graduating outcomes correlating with the variables of i) gender, ii) first language, iii) discipline, iv) entry pathway to the University, or v) reason for taking the IELTS test?

Discussion Within the identified variables • • Limited language improvement during degrees Language is

Discussion Within the identified variables • • Limited language improvement during degrees Language is not guaranteed to improve L 1 is a legitimate explanation for increased potential linguistic challenges: the greater the distance from English, the greater the challenge The same patterns by L 1 were not found for academic outcomes as linguistic outcomes Different linguistic groups require differentiated support Language is a key enabler but not guarantee of academic success Tendencies rather than predictions of future student performance: beware generalisations Is first language an intervening variable masking other aspects of within-group homogeneity (Cummins & Swain, 1984; Phakiti, Hirsh & Woodrow, 2013)?

Agent management Foreign Service Institute typology of language difficulty 1 French Italian Spanish Swedish

Agent management Foreign Service Institute typology of language difficulty 1 French Italian Spanish Swedish 3 Estonian Finnish Thai Vietnamese http: //aboutworldlanguages. com/language-difficulty 2 4 Czech Russian Turkish Urdu Arabic Mandarin/Cantonese Korean Japanese

Discussion Beyond the identified variables

Discussion Beyond the identified variables

Discussion Beyond the identified variables • • Individual differences (Cotton & Conrow, 1998; Storch

Discussion Beyond the identified variables • • Individual differences (Cotton & Conrow, 1998; Storch & Hill, 2008; Phakiti, Hirsh & Woodrow, 2013) Cognitive factors (Criper & Davies, 1998) Socioeconomic factors (Bailey & Butler, 2007; Cummins, 2000) Further research needed

Implications for ELICOS providers • play an important role in ensuring strong language proficiency

Implications for ELICOS providers • play an important role in ensuring strong language proficiency at commencement of higher ed program • can message to students about likely limited ELP improvement once commence degree • should encourage students to develop skills to continue to develop their ELP

Implications for higher ed providers Higher ed providers should • have (and communicate) realistic

Implications for higher ed providers Higher ed providers should • have (and communicate) realistic expectations of ELP improvement • explicitly attend to ELP during degrees • target academic language support for speakers of first languages more typologically distant from English • target academic skills support for speakers of African and Indian languages

References Arkoudis, S. , & Doughney, L. (2014). Good practice report – English language

References Arkoudis, S. , & Doughney, L. (2014). Good practice report – English language proficiency. Sydney: Office for Learning and Teaching. Retrieved from http: //www. cshe. unimelb. edu. au/research/teaching/docs/GPR_English_language_2014. pdf Australian Government. (2015). Draft national strategy for international education. ISBN: 978 -1 -74361 -889 -9. Retrieved from https: //internationaleducation. gov. au/Internationalnetwork/Australia/International. Strategy/Documents/Draft%20 National%20 Strategy%20 for%20 International%20 Education. pdf Bailey, A. L. , & Butler, F. A. (2007). A conceptual framework of academic English language for broad application to education. In A. L. Bailey (Ed. ), The language demands of school: Putting academic English to the test (pp. 68 -102). New Haven: Yale University Press. Barthel, A. (2012). English Language Standards for Higher Education. Retrieved from http: //www. aall. org. au/sites/default/files/Barthel. Cam. Rev 26 Jun 12_0. pdf Birrell, B. , Hawthorne, L. , & Richardson, S. (2006). Evaluation of the General Skilled Migration Categories Report. Commonwealth of Australia. Retrieved from http: //www. flinders. edu. au/sabs/nilsfiles/reports/GSM_2006_Full_report. pdf Casella, G. & George, E. (1992). Explaining the Gibbs Sampler. The American Statistician 46(3), 167 -174. Cotton, F. , & Conrow, F. (1998). An investigation into the predictive validity of IELTS amongst a group of international students studying at the University of Tasmania. IELTS Research Reports, 1 , 72 -115. Canberra: IELTS Australia Pty Limited. Craven, E. (2012). The quest for IELTS 7. 0: Investigating English language proficiency of international students in Australian universities. IELTS Research Reports, 13, 1– 61. Criper, C. , & Davies, A. (1988). ELTS validation project report. ELTS Research Report 1(i). Cambridge: British Council and University of Cambridge Local Examinations Syndicate. Cummins, J. (2000). Language, Power & Pedagogy: Bilingual Children in the Crossfire. Clevedon, UK: Multilingual Matters. Cummins. J. , & Swain, M. (1983). Analysis-by-rhetoric: Reading the text or the reader’s own projections? A reply to Edelsky et al. Applied Linguistics, 4(1), 23 -41. Department of Education Employment & Work Relations (DEEWR). (2008). Good Practice Principles for English Language Proficiency for International Students in Australian Universities – Final Report. Retrieved May 5 2009, from http: //www. deewr. gov. au/Higher. Education/Publications/Pages/Good. Practice. Principles. aspx Humphreys, P. , & Mousavi, A. (2010). Exit-testing – a whole of university approach, Language Education in Asia, 1(1), 8– 22. Retrieved from http: //dx. doi. org/10. 5746/LEi. A/10/V 1/A 03/Humphreys_Mousavi Humphreys, P. (2015). English Language Proficiency in Higher Education: Student conceptualisations and outcomes. Unpublished doctoral dissertation. Griffith University, Queensland, Australia. International Education Association of Australia (IEAA). (2013). Five years on: English language competence of international students. Outcomes Report, June 2013. Retrieved from http: //www. ieaa. org. au/research-projects/englishlanguage-competence Kaplan, D. (2014). Bayesian statistics for the social sciences. New York, NY: Guilford Press. Knoch, U. , Rouhshad, A. , Oon, S. P. , & Storch, N. (2015). What happens to ESL students’ writing after three years of study at an English medium university? Journal of Second Language Writing, 28, 39 -52. Madsen, H. , & Thyregod, P. (2011). Introduction to general and generalized linear models. Boca Raton, FL: CRC Press. O’Loughlin, K. , & Arkoudis, S. (2009). Investigating IELTS exit score gains in higher education. IELTS Research Reports, 10. IELTS Australia Pty Ltd, Canberra Phakiti, A. , Hirsh, D. , & Woodrow, L. (2013). It’s not only English: Effects of other individual factors on English language learning and academic learning of ESL international students in Australia. Journal of Research in International Education, 12(3), 239 -258. Storch, N. & Hill, K. (2008). What happens to international students’ English after one semester at university? Australian Review of Applied Linguistics, 31(1), 04. 1– 04. 17. Tertiary Education Quality and Standards Agency (TEQSA). (2013). Quality assessment: English Language Proficiency Terms of Reference , March 2013. Retrieved from http: //www. teqsa. gov. au/sites/default/files/English. Language. Proficiency. QATerms. pdf

Questions?

Questions?