TestTakers Views on the TOEIC Speaking Test Masaya

Test-Takers’ Views on the TOEIC Speaking Test Masaya Kanzaki Kanda University of International Studies JALT 2016, November 27, Room 1102

Consumer validity Subjects’ attitudes and feelings towards the test (Coniam, 1999) This term could be used to cover attitudes and feelings of both test-takers and test-users, the after-effects of testing procedures, and the possible debilitative effects on test scores of aspects of test design and test administration. (Bradshaw, 1990)

Purpose To find out how test-takers felt about the TOEIC Speaking test

Participants 313 university students attending a private university specializing in foreign languages 45 male, 268 female 2013: 84 participants 2014: 94 participants 2015: 54 participants 2016: 81 participants 19 of them participated twice.

Participants 1 st year: 21, 2 nd year: 160, 3 rd year: 77 4 th year: 55 International Communication: 165, English: 86 International Business Career: 33, Spanish: 10 Portuguese: 6, Chinese: 5, Indonesian: 4 Vietnamese: 3, Thai: 1 L 1 Japanese: 305, Chinese: 6, Korean: 2

Materials • TOEIC Listening and Reading test • TOEIC Speaking test • A questionnaire

TOEIC LR: 200 multiple questions, 2 hours, 10 -990 points Listening Part Task 1 For each question with a photo, listen to four sentences and choose the one that best describes the image. 2 Listen to a question or statement followed by three responses and choose the most appropriate response. 3 4 Listen to a conversation and answer comprehension questions. # 10 30 30 Listen to a short talk and answer comprehension questions. 30

TOEIC LR: 200 multiple questions, 2 hours, 10 -990 points Reading Part Task # Choose a word or phrase to fill in a blank in a sentence. 5 40 Choose words or phrases to fill in three blanks in a 6 passage. 12 Read a passage or a set of two passages and answer 7 comprehension questions. 48

TOEIC Speaking: Computer-based

TOEIC S: 11 questions, 0 -200 points, 20 minutes # 1– 2 3 4– 6 Task Read aloud the text that appears on the screen. Describe the picture on the screen. Answer three questions about a single topic as though you are participating in a telephone interview. 7– 9 Read the information on the screen and answer three questions about it as if responding to a telephone inquiry. 10 11 Listen to a recorded message about a problem and propose a solution for it. Express an opinion about a specific topic.

A questionnaire 12 questions Slightly different each year Five of them are the same

Results Descriptive Statics (N = 313) Mean SD Minimum Maximum S (0 -200) 116. 45 21. 57 40 180 L (5 -495) 372. 83 57. 81 170 495 R (5 -495) 272. 00 100 475 LR 644. 82 119. 63 310 945

S Scatter plots 200 180 160 140 120 100 80 60 40 20 0 300 400 500 600 LR 700 800 900

Correlations (N = 313) S L R LR . 48 . 52

1. How did you find it? (Multiple answers allowed) 0 50 Easy 100 150 12 Difficult 177 Effortless 13 Effortful 109 Fun Boring 2 143 1

2. Which do you prefer? (N = 313) 42 38 147 86 Speaking L&R Both Neither

3. Were you at a loss during the test? 36 121 156 Many times A few times Not at all

4. Do you want to take it again? 26 7 13 118 149 Very much so Yes, I do I'm not sure Not very much Not at all

5. Do you think your score will go up next time? 5 32 20 176 80 Very much so Yes, I do I'm not sure Not very much Not at all

6. How well were you able to show your speaking ability? (N = 133) 17 9 Perfectly Quite well Not very well Not at all 107

7. Did you feel talking to a computer is more difficult than to a person? (N = 135) 2 38 36 59 Very much Yes, I did Not so much Not at all

8. Do you think it is a good test to measure speaking ability? (N = 135) 30 17 Very much Yes, I do No, I don't Not at all 88

9. What do you think of taking tests like TOEIC? (Multiple answers allowed) 2 34 2 110 81 1) 2) 3) 4) 5)

Q&A Thank you. Acknowledgement This study was supported by JSPS KAKENHI Grant Number 25370727.