C Wheadon Q He CEM CENTRE DURHAM UNIVERSITY

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK An Investigation of The Response Time for Maths Items in A Computer Adaptive Test Chris Wheadon and Qingping He (CEM Centre, Durham University, UK)

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Background • The CEM Centre’s Baseline Assessments on Primary and Secondary School Students • The CEM Baseline Tests for Secondary Schools (for students aged 11 -18: three separate projects) Paper based, Efficiency, Cost • Use of CTT and IRT in Computerised Testing and to Interpret Test Data • To Develop A Computerised Test to Replace the Existing Paper-based Baseline Tests for the Three Secondary Projects

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK CABT-The CEM Computer Adaptive Baseline Test • CABT comprises a calibrated item bank and a question display system • The adaptive test contains an adaptive maths test and an adaptive English vocabulary test • Testing is delivered through the Web or from the school’s local area network

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Establishing A Calibrated Item Bank • Testing of items by school students from different year groups through the administration of a series of tests containing common items • Calibration of items in each test using the Rasch model (items meeting the Rasch requirements are used) • Setting of the reference test and equating of different tests using the Rasch model and common items • Further new item calibration through the embedding of new items in the adaptive test

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Developing The Question Display System - Conducting Adaptive Testing • Realisation of adaptivity is achieved through the implementation in the question display system of the Rasch model for ability estimation (MLE method is used) and question selection • Rules for stopping test: minimum number of question; maximum number of questions; and convergence value • Variable starting difficulty for the first question for different year groups.

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Preliminary Results from the Adaptive Maths Test • There are over 500 items in the maths item bank • Effort has been made to make the items as curriculumfree as possible • Items are content-independent to each other • Item types include MCQ, short free text entry questions and interactive questions • Items cover a wide range of difficulties in order for the three secondary projects (for three different year groups) to use the same adaptive test as their baseline test

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Individual Student: The Distribution of Ability and the Distribution of Item Response Time

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Individual Student: The Distribution of Ability and the Distribution of Item Response Time

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK All Items: The Effect of Item Difficulty on Response Time

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK A Selection of Items: The Effect of Age of Test Takers on Item Response Time

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK A Selection of Items: The Relationship between Response Time for Correct Answers and Response Time for Incorrect Answers

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Individual Item: The Effect of Ability and Age on Response Time (Difficulty: -0. 6 logits) – Easy Item

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Ability Bands within a Year Group for the Specific Item (Year 7)

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Individual Item: The Effect of Ability and Age on Response Time (Difficulty: 1. 1 logits) – Mediumdifficulty Item

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Ability Bands within a Year Group for the Specific Item (Year 10)

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Individual Item: The Effect of Ability and Age on Response Time (Difficulty: 2. 9 logits) – Difficult Item

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Ability Bands within a Year Group for the Specific Item (Year 12)

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Conclusions and Further Work • Response time for all the maths items in the test generally increases with item difficulty but shows great variability • Item difficulty levels and the age and ability of test takers have significant influence on item response time • The information obtained in this study can be used for constructing more efficient tests • Further work will involve investigating the relationship between CABT results and students’ subsequent academic performance