Assessment of Listening Notetaking Skills on EAP courses

  • Slides: 14
Download presentation
Assessment of Listening & Note-taking Skills on EAP courses: Two Test Methods Donal Crawford

Assessment of Listening & Note-taking Skills on EAP courses: Two Test Methods Donal Crawford International Pacific College July 2010

Content �Assessment of Note-taking skills � 2 methods – ‘Retrieval’ & ‘Direct’ �Authenticity �Practicality

Content �Assessment of Note-taking skills � 2 methods – ‘Retrieval’ & ‘Direct’ �Authenticity �Practicality �Reliability �Validity �The Way Ahead?

Notetaking Skills in EAP �Note-taking skills in lectures widely seen as important component of

Notetaking Skills in EAP �Note-taking skills in lectures widely seen as important component of EAP courses (Rost. 2002; Jordan, 1997; Flowerdew, 1994 a) �Studies on note-taking & lecture comprehension (Flowerdew 1994 b; Rost, 1990, 2002) and published materials (e. g. Lebauer, 2000; Sanabria, 2004) �Very little information on assessment of this skill (King , 1994) � 2 methods used in final examination assessment within EAP Foundation Year or year 1 programmes

Retrieval Method �Assessment via questions which candidates answer using their notes �Procedure: �Listen to

Retrieval Method �Assessment via questions which candidates answer using their notes �Procedure: �Listen to recorded mini-lecture and make notes �Listen again and improve notes �Answer set of short-response questions using notes �Marked wrt marking scheme

Direct method �Assessment of the notes themselves �Procedure: � Listen to Section 1 of

Direct method �Assessment of the notes themselves �Procedure: � Listen to Section 1 of recorded lecture + Power. Point outline and make notes � Repeat for Section 2 � Repeat twice for Section 3 �Marking: Each section of notes marked/20 based on coverage and accuracy Each section divided into 4 parts with up to 5 marks for each part. � Markers meet and compare a sample of papers to agree on required levels of coverage. � Marks reduced for missing and/or inaccurate information. �

Authenticity �Scripted �Lack of visual input �Repetition

Authenticity �Scripted �Lack of visual input �Repetition

Practicality �Recorded vs. ‘live’ �Live recording? �Writing questions for Retrieval method �Pre-marking meetings for

Practicality �Recorded vs. ‘live’ �Live recording? �Writing questions for Retrieval method �Pre-marking meetings for Direct method

Reliability �Inter-marker reliability �Likely to be higher for Retrieval method �Routine sampling & double-marking

Reliability �Inter-marker reliability �Likely to be higher for Retrieval method �Routine sampling & double-marking for Direct �Inter-test reliability �Likely to be higher for Direct method

Validity “The centrality of the purpose for which the test is being devised or

Validity “The centrality of the purpose for which the test is being devised or used cannot be understated” (Alderson, Clapham & Wall, 1995) Powers (1986) key lecture listening activities: 1) identifying major themes or ideas of lectures 2) identifying relationships among major ideas in a lecture 3) identifying the topic of the lecture 4) retaining information through notetaking 5) retrieving information from notes 6) inferring relationships between information supplied in the lecture 7) comprehending key information presented in the lecture 8) following the spoken mode of the lecture 9) identifying supporting ideas and examples in the lecture.

Validity (Internal) �Face (credibility) �Both require students to take notes. �Students certainly take the

Validity (Internal) �Face (credibility) �Both require students to take notes. �Students certainly take the exams seriously �Content �Retrieval clearly tests Powers (5), and, depending on questions, (1), (6), (7), (8), (9) �Dependent on lecture content, delivery & Q-setting �Direct clearly tests Powers (1), (3), (8), (9) �Dependent on lecture content & delivery �Response – not yet considered

Validity (External) �Concurrent �Retrieval compared with ‘IELTS-style’ listening test, n=97, r=+0. 74 �Direct data

Validity (External) �Concurrent �Retrieval compared with ‘IELTS-style’ listening test, n=97, r=+0. 74 �Direct data not yet available �Predictive – no data

The Way ahead? �Power. Point & Moodle �Students often given lecture notes in advance

The Way ahead? �Power. Point & Moodle �Students often given lecture notes in advance �Listening & understanding more important than notetaking (Shin, 2008) �Lecture comprehension �Use of notes

References Alderson, J. C. , C. Clapham & D. Wall (1995). Language Test COnstruction

References Alderson, J. C. , C. Clapham & D. Wall (1995). Language Test COnstruction & Evaluation. Cambridge: Cambridge University Press. Flowerdew, J. (1994 a). Research of relevance to second language lecture comprehension - an overview. In J. Flowerdew, editor, Academic Listening Research Perspectives (pp. 7 -29). Cambridge: Cambridge University Press Flowerdew, J. (1994 b). Academic Listening Research Perspectives. Cambridge: Cambridge University Press Jordan, R. R. (1997). English for Academic Purposes. Cambridge: Cambridge University Press. King, P. (1994). Visual and Verbal Messages in the Engineering Lecture: Notetaking by postgraduate L 2 students. In J. Flowerdew, editor, Academic Listening Research Perspectives (pp. 219 -38). Cambridge: Cambridge University Press Lebauer, R. S. (2000) Learn to Listen, Listen to Learn. White Plains, NY: Pearson Education. Powers, D. E. (1986). Academic demands related to listening skills. Language Testing 3 (1): 1 -38. Rost, M. (1990). Listening in Language Learning. Harlow: Longman. Rost, M. (2002). Teaching and Researching Listening. Harlow: Pearson Education. Sanabria, K. (2004). Academic Listening Encounters: Life in Society. Cambridge: Cambridge University Press. Shin, I. (2008). Necessary Skills in English for Korean Postgraduate Engineering Students in London. Educate - Special London Edition, September 2008, 50 -61 [online version]. http: //www. educatejournal. org

Student NoteTaking Effective NoteTaking for Students Student NoteTakingStudent NoteTaking Effective NoteTaking for Students Student NoteTaking
Student NoteTaking Effective NoteTaking for Students Student NoteTakingStudent NoteTaking Effective NoteTaking for Students Student NoteTaking
ACADEMIC LISTENING Listening 1 Using Notetaking Symbols ListeningACADEMIC LISTENING Listening 1 Using Notetaking Symbols Listening
NoteTaking Skills Improve Your Notetaking Skills and ImproveNoteTaking Skills Improve Your Notetaking Skills and Improve
Chapter 4 Listening Skills Listening Skills Introduction ListeningChapter 4 Listening Skills Listening Skills Introduction Listening
Chapter 4 Listening Skills Listening Skills Introduction ListeningChapter 4 Listening Skills Listening Skills Introduction Listening
Chapter 4 Listening Skills Listening Skills Introduction ListeningChapter 4 Listening Skills Listening Skills Introduction Listening
Active Listening Listening Skills Listening for understanding isActive Listening Listening Skills Listening for understanding is
LISTENING SKILLS LISTENING SKILL Listening exercises from theLISTENING SKILLS LISTENING SKILL Listening exercises from the
EAP Writing Assessment Preparation MiniUnit The EAP OverviewEAP Writing Assessment Preparation MiniUnit The EAP Overview
Early Assessment Program EAP Tiamo De Vettori EAPEarly Assessment Program EAP Tiamo De Vettori EAP
Listening Skills and Blocks Listening Skills Stop TalkingListening Skills and Blocks Listening Skills Stop Talking
Listening Skills Steps and Skills for Listening UnitListening Skills Steps and Skills for Listening Unit