Asia CALL HCMC Open University Viet Nam Oct

Asia. CALL @ HCMC Open University, Viet Nam, Oct 25, 2017 Standardized Web-based Test of EFL Learner's Normal Reading Speed Eiichi Yubune (Professor of Language & Information Sciences at Toyo University) Akinobu Kanda (Associate Professor of English at Tokyo Metropolitan University) Masahiro Suzuki (Associate Professor of English at Seibu Bunri University) Takane Yamaguchi (Assistant Professor of English at St Margaret's Junior College) Ken-ichi Ohyama (Assistant Professor of English at Edogawa University) Ryuji Tabuchi (Computer Programmer. President of Mint Phonetics Research Institute)

1. Background (Theoretical) Reading skills for communication Comprehension Reading rate Summarization ↑ How do we measure? Scanning / skimming And more. . .

1. Background (Technical) We developed MEWM, the web app. measuring a learner’s reading rate (wpm), comprehension, and reading efficiency.

2. Purpose - to develop a web-based standardized test measuring EFL learners' normal reading speed that may reflect his/her phonological coding load.

3. What is “normal reading speed”? And more. . . Stop and think Temporary pause Reread / garden path Cruising

4. Method: How do we measure the cruising speed? 1) prepared 99 different levels of English short passages of appx. 100 words with two original comprehension questions after reading. 2) used four different levels of passages from the existing reading test in Japan (STEP: The Society of Testing English Proficiency): 1) Semi 1 st level (14 th grader) 34 passages 2) 2 nd level (12 th grader) 29 passages 3) Semi 2 nd level (10 th grader) 24 passages 4) 3 rd level (9 th grader) 12 passages

4. Method: How do we measure the cruising speed? Need to control three factors: Readability (MGJP: Mint Grade Level for Japanese Learners) Vocabulary level (VGL: Vocabulary Grade Level) Comprehension Quiz quality & difficulty level

4. Method: Why need questions after reading? - Cruising speed is supposed to be 150~200 wpm (words per minute); otherwise, the reading rate with 70% comprehension (Iwaki, 1980; Smith, 1982). - We need to select the data of a learner’s reading rate with around 70% comprehension. - A group of leaners’ reading rate is directly proportional to comprehension; beginners are low in both skills while advanced learners tend to be high in both skills (Carver, 1990).

5. Preliminary Test Participants: 204 students from five different universities (TOEIC 250 -800) Date: July, 2017 The number of samples collected after data cleaning: 1) Semi 1 st level (14 th grader) 51 passages 2) 2 nd level (12 th grader) 76 passages 3) Semi 2 nd level (10 th grader) 200 passages 4) 3 rd level (9 th grader) 291 passages

Data Cleaning Guidelines: 1. Data with the reading time of the passage less than 20 seconds are deleted 2. Data with answering time less than 10 seconds are deleted 3. Data with the reading rate faster than 250 wpm are deleted (Carver, 1990: 90 -91, Yamaguchi, et al. , 2014) 4. Data with the remaining time more than 10 minutes are deleted (14 minutes is the answering limit for reading 6 -8 passages and comprehension questions).

Example passage and the questions My family and I are moving to Minneapolis! My father‘s company is opening a new office next year, and he’ll be the manager there. Next month, my parents are going to Minneapolis to look for a new house. While they are away, my sister and I will stay at my grandmother‘s house here in Fairfield. We’ll move to Minneapolis at the end of December. I‘m excited about moving to a big city, but I love Fairfield. I’ve lived here since I was born. You‘re my best friend, and I’m going to miss you so much! Maybe you can visit me in Minneapolis next summer. When you come, we can go to some museums and go sightseeing around the city. (MGJP=11. 5) Q 1 本文は主に何について書かれたものですか ？ a) 父の転勤 b) アメリカの大都市と小さな町の比較 c) 引っ越しの予定 d) ミネアポリスの観光 Q 2 本文の内容と合っているものを 1つだけ選 んで下さい。 a) 引っ越す先の友人に早く会いたい b) 引っ越す前の現在の町に住む友人が名残惜 しい c) ミネアポリスはエキサイティングな街だ d) ミネアポリスの夏は涼しい

5. 1 Preliminary Test Results

Correct answer rate broken down by readability 正答率 100, 00% 90, 00% 80, 00% 70, 00% 60, 00% 50, 00% 9 10 11 12 13 14 15

Readability (MGJP) vs. correct answer rate (r=-0. 15 n. s. ) 2 1, 8 1, 6 1, 4 1, 2 1 0, 8 0, 6 0, 4 r= -. 15 n. s. 0, 2 0 5 7 9 11 13 cf. MGJP vs. VGL 15 17 r=. 68 ** 19 21 MGJP

Vocabulary level (VGL) vs. correct answer rate (r=-0. 02 n. s. ) 2 1, 8 1, 6 1, 4 1, 2 1 0, 8 0, 6 0, 4 r= -. 02 n. s. 0, 2 0 5 7 9 11 13 15 17 19 VGL

Reading rate (WPM) vs. correct answer rate (r=-0. 02 n. s. ) 2 1, 8 1, 6 1, 4 1, 2 1 0, 8 0, 6 0, 4 r= -. 01 n. s. 0, 2 0 0 50 100 150 200 250 wpm

Multiple regression analysis (against comprehension score) For passages with comprehension rate below 50% MGJP n. s. VG n. s. wpm n. s. . 001 Score For passages with comprehension rate above 50% MGJP ‐. 11**. 012 VG . 09* wpm . 09*** Score

Correlation coefficient For passages with comprehension rate below 50% MGJP VG wpm . 75** . 11* VG . 032 wpm For passages with comprehension rate above 50% MGJP VG VG wpm . 73** . 02*. 07* wpm *p<. 05, **p<. 01, ***p<. 001

6. Tentative conclusion • According to the preliminary test results, comprehension rate has little relationship with readability and vocabulary level. • In other words, we cannot measure a learner’s comprehension rate for a particular passage simply based on its readability and vocabulary level. • Instead, it largely depends on quiz quality and difficulty level.

7. Further challenges • We need to make further efforts to prepare valid quizzes to appropriately measure a leaner’s comprehension rate. • We need to validate how many percent correction rate can best calibrate the learners’ comprehension of passages when we use four-item multiple choice questions.

A guidelines for making validate comprehension quizzes (Tabuchi & Yubune, 2013) 1. Less load on grammar and sentence structures. 2. A plain and easy contents and comprehension is not dependent on background knowledge. 3. The right quantity of text independent of memory strategies. 4. No interval time before starting comprehension quizzes. 5. Two question or less for one passage, and little cognitive load on understanding each item of the four-options. 6. Each quiz asks the gist of the passage not a specific pinpoint information appeared locally in the passage.

Thanks to: Japan Society for the Promotion of Science (JSPS) @ The Ministry of Education, Culture, Sports, Science and Technology (MEXT) Grant-in-Aid for Scientific Research(C) 15 K 01082 Leading Researcher：Akinobu Kanda， 15 K 02733 Leading Researcher ：Eiichi Yubune, and 17 K 02896 Leading Researcher ：Masahiro Suzuki

References Carver, R. C. (1990). Reading Rate A review of Research and Theory. Academic Press. Smith, F. （1982). Understanding Reading. 3 rd ed. Austin. Holt, Rinehart and Winston. Tabuchi, R. & Yubune, E. (2016). A Developmental Study of Readability Formula for Japanese EFL Learners based on Estimated Phonological-coding Time. Language Education & Technology, 52, 359 -388. Tabuchi, R. & Yubune, E. (2013). 「 1次的読解速度予測に基づく日本人英語学習者向けリーダビリティ公式の 開発」, 外国語教育メディア学会(LET)第 54回(2014年度)全国研究大会発表要項, pp. 116 -117. Tabuchi, R. (2017). 「日本人英語学習者向け語彙レベル適応学年算出公式の試験的開発」, LET Kanto Journal, 1, 25 -35. 山口高領、神田明延、湯舟英一、田淵龍二、池山和子、鈴木政浩 (2014). Effects of Chunk-based Reading Aloud on Silent Reading Speed and comprehension. Language Education & Technology, 51, 243 -266. 岩城禮三. （1980 ). 「精読と速読」. 『英語教育』. 9. 21 -23.