CCSSO Conference Designing Items to Adapt to UserSupplied

CCSSO Conference Designing Items to Adapt to User-Supplied Hardware Debbie Pisacreta Principal Research Systems Specialist 6/21/13 Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved.

2012 Study i. Pad Focused on item types that may cause a performance difference when taking a test on an i. Pad vs. a laptop computer such as: – items that require typing – items that require editing text – items that require touching the screen to answer a question (e. g. drag and drop) 2 Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved. 1/22/2022

Using external keyboard vs. i. Pad soft keyboard External Keyboard • All keys visible • All content on test screen visible 3 i. Pad Soft Keyboard • Requires switching between 3 keyboards • Reduces amount of visible content on screen Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved. 1/22/2022

Results of Timed Typing Test Typing on the i. Pad keyboard is slower and less accurate for most participants Average Typing Time Average Number of Errors i. Pad Soft Keyboard 3: 16 5 External Keyboard 2: 18 3 Observations: 4 • switching between keyboards for numbers and symbols resulted in slower time on the i. Pad keyboard • some participants stated that it was easier to type the wrong key on the i. Pad because the surface was flat as opposed to the raised keys on the keyboard Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved. 1/22/2022

Keyboard Preference Most participants preferred the external keyboard for typing essays Type of Keyboard Which keyboard would you prefer for general use? Which keyboard would you prefer for typing essays? i. Pad 2 3 External 8 7 Reasons cited for preference: – – They were familiar with it Easier to touch type All symbols and numbers are visible You can see the entire screen Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved. 1/22/2022

Selecting text with cursor vs. pointing to select Using a Mouse • Allows precise cursor placement • Easier to select text • Faster to edit text using keyboard shortcuts 6 Using a Finger • Precise placement of cursor is difficult • More cumbersome to select text • Editing text is a 2 -step process Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved. 1/22/2022

Results of Text Editing Tasks Some participants had difficulty using i. Pad text editing features Able to highlight a word? Able to copy and paste a sentence? • Yes 6 No 4 7 3 Participants who were not familiar with i. Pad made common errors such as – dragging their finger through text they wanted to highlight – able to select a single word but didn’t understand how to select multiple words or to use the copy and paste buttons Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved. 1/22/2022

Items Requiring Touching the Screen to Answer a Question Most participants had no difficulty on items that required touching the screen Except for: • Items that required touching a small area (2 made errors) • Items that required precise cursor placement (3 had difficulty; especially between words) Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved. 1/22/2022

Items Requiring Minimal Typing Most participants had no difficulty on items that required minimal typing Except for: • • One participant who used the external keyboard found it cumbersome to move between the screen and keyboard One participant who used the i. Pad keyboard found it annoying to switch keyboards when typing a number Number of Big Cards Number of Small Cards b s 0 10 1 9 2 6 Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved. Total Time to Make all the Cards m (m = 12 b + 6 s) ü If m = 60 minutes 1/22/2022

Suggestions • Test takers should be provided with an external keyboard and stand • Test takers should be provided with a tutorial on i. Pad features, specifically on: – enlarging screen content – placing the cursor – switching between keyboards to type numbers and symbols – closing the keyboard after typing – selecting a word or phrase – copying and pasting text – scrolling text • Touch areas should follow Apple recommendation (minimum 44 x 44 pixels) Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved. 1/22/2022

Further Research • Explore more complex items • Explore timing differences for specific item types. For items that require touching and typing, particularly for numeric entry, it may take longer to complete this type of item on an i. Pad. 11 Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved. 1/22/2022

12 Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved. 1/22/2022

i. Pad Magnification Bubble 13 Confidential and Proprietary. Copyright © 2013 by Educational Testing Service. All rights reserved. 1/22/2022