Speaking and Listening Sounds Straightforward but Whats the

Speaking and Listening: Sounds Straightforward, but What’s the Real Story? NCSA June 22, 2016 Wes Bruce – ELPA 21 Technology Consultant Carsten Wilmes, Ph. D. – WIDA Director of Assessment Terri Schuster – Nebraska Title III Director Jen Paul – EL and Accessibility Assessment Consultant, Michigan Department of Education

Who are the English Language Proficiency Consortia ELPA 21 WIDA • Funded by USED Assessment Grant • Originally funded by 2002 USED Grant • Oregon is the lead state • Housed at University of Wisconsin-Madison • Brand new assessment based on the ELP standards developed by West. Ed • Directed by WIDA Board of Directors (SEAs) • Assessment operational 2015 -16 • New online ACCESS for ELLs 2. 0 assessment operational 2015 -16 (in addition to paper) • Standard setting this summer • New standard setting July/August of this year

Evolving the Assessment of Listening and Speaking English language proficiency tests have been assessing the “four” domains for quite a while. We are focused on how this next generation of EL tests addresses Listening and Speaking Previously ◦ Listening via teacher read or recorded prompts that students responded to ◦ Standardization for read aloud ◦ Errors in advancing to the correct prompt ◦ Speaking was often 1 to 1 with student reading and teacher scoring response with rubrics then recording the student’s score ◦ Time consuming ◦ Reliability of scoring ◦ Standardization

Question: Can technology improve the assessment of Speaking & Listening? Provide a more standardized domain assessment , increase reliability and validity without increasing costs ◦ Removing the demands of 1 to 1 delivery, teacher training and scoring of Listening item ELPA 21 and WIDA worked to build items that could use an “intuitive” interface that students with little exposure to technology could use All grades K-12 use the same interface (WIDA 1 -12). All audio is professionally recorded (not TTS) Students get one (WIDA)/two (ELPA 21) opportunities to hear the prompts in Listening Students get one (WIDA)/two (ELPA 21) opportunities to record in Speaking

Speaking requires additional hardware Students must have headsets (headphone with microphone) ◦ Directional adjustable microphone the best ◦ Think about durability and sanitation The headset must be compatible with the testing device ◦ Dual RCA plugs, single RCA, USB ◦ Adapters (Y, USB to RCA) The headset must “fit” the user (ELPA 21 is testing Kindergarten students) Cost is an issue Headsets are “only” needed for the maximum number of students who will test simultaneously in one site ◦ Can easily be organized into kits and shared between testing sites

ELPA 21 “Controls” LISTENING SPEAKING Separate control for directions and question. Record, stop and play controls Darker background when “playing”

WIDA “Controls” / Listening 11

Speaking

ELPA 21 Demonstra tion

ELPA 21 Speaking and Listening ADMINISTRATION CONSIDERATIONS LESSONS LEARNED

English Language Proficiency (ELP) Assessments and English Language Arts (ELA) —What’s the diff? ELP ASSESSMENTS ELA ASSESSMENTS Required to assess and report on all 4 language domains Most assess reading skills and sometimes writing, rarely listening or speaking ◦ Reading, Writing, Listening, and Speaking Construct is the learning of a new language ◦ The interactive nature of language makes assessing listening and speaking challenging Construct usually centered reading comprehension, vocabulary, writing skills

Listening and Speaking Evolution: ELPA 21 and ELDA (Legacy Assessment) ELPA 21 (K-12) LEGACY TEST (ELDA 3 -12 *) Listening ◦ Students wear headsets and listen to audio prompts as they move independently through an online form Speaking ◦ May be group administered ◦ Students listen to prompts and speak into headsets where their responses are recorded as they move independently through an online form ◦ Scored at a scoring center ◦ Students listen to CDs while working through a booklet Speaking ◦ Individual administration ◦ Students listened to CDs/look at pictures in booklets ◦ Scored on the fly with rubrics by test administrator * K-2 --inventory of skills

Benefits: Online Assessment of Listening and Speaking May be group administered (with caveat for young students) Embedded audio eliminates need for additional playing equipment Students may re-record response Recorded responses allow for re-listening during scoring Eliminates need for training local test administrators on rubrics Trained scorers at a center ◦ Uniformity in training of scorers/more consistent ratings

Challenges: Online Assessment of Speaking and Listening Use of headsets ◦ Can be costly and may not be purchased with Title III funds ◦ May be shared as test does not need to be taken at the same time ◦ Headsets used with other software programs may work ◦ Need to fit student’s head ◦ Students need practice speaking into them ◦ Microphones pick up background noises/other students and teacher talk ◦ Teachers sometimes heard “coaching” students too much ◦ Consider adding physical space between students for speaking portion ◦ Recommend trying out several before place large order

Challenges Continued… Assessing young students Low student/test administrator ratio is recommended Test administrators should be very familiar with Directions for Administration prior to test time Practice with Interactive Demo (practice test) is critical ◦ Speaking record/playback/re-record features Headsets need to fit Test administrators need to understand when it is appropriate to help with technology/navigation and what constitutes too much assistance

Preparation and Practice = Success Advance coordination between school/district IT folks and vendor ◦ Technical support ◦ Ensure devices are supported Familiarity with directions and navigation by test administrators Understand allowed accessibility features and how they may be applied in the platform (embedded and non-embedded) Plenty of practice with interactive demo (practice test) and using the features prior to testing