ASSESSMENT IN EDUCATION Copyright Keith Morrison 2004 Reliability

ASSESSMENT IN EDUCATION Copyright Keith Morrison, 2004

Reliability Ü Consistency across markers, occasions, test items, test types, marking conventions, grading procedures, contexts. Inter-rater reliability Dependability Degree of confidence Transparency Ü Test-re-test, equivalent Copyright Keith Morrison, 2004 forms, internal consistency.

Errors in marking Threats to reliability from markers Inter-rater reliability Inconsistency in markers Variation at grade boundaries Copyright Keith Morrison, 2004 Halo effect

THREATS TO RELIABILITY FROM STUDENTS AND TEACHERS • • Motivation and interest in the task Relationship between assessor and assessee Test conditions/environment Hawthorne effect Distractions Teacher expectations Time of day/week Students are not always clear on what they think is being asked Copyright Keith Morrison, 2004

THREATS TO RELIABILITY FROM STUDENTS AND TEACHERS • Students may see two similar questions as dissimilar • Teachers teach to the test • Teachers and students practise test-like materials • Students can perform the skill in the test but not in real life, and vice versa • Cultural and ethnic background affect meaningfulness of the test • Unreliability of marking practices Copyright Keith Morrison, 2004

THREATS TO RELIABILITY FROM ASSESSMENT ITEMS • • Task may be multi-dimensional Validity of items Language of the assessment Readability levels Size and complexity of numbers Number and type of operations Form and presentation of questions Early error in a sequence may block later stages of the sequence • Questions using mechanical toys may favour boys over girls Copyright Keith Morrison, 2004

THREATS TO RELIABILITY FROM ASSESSMENT ITEMS • Questions requiring use of dolls or kitchen work may favour girls over boys • Essays favour boys if they concern impersonal topics and girls if they favour personal topics • Boys perform better than girls on multiple choice items • Girls perform better than boys in written work • Continuous assessment favours girls • Questions are culture-bound • The length of the test affects performance Copyright Keith Morrison, 2004

VALIDITY Face validity Content validity CONSTRUCT Consequential Copyright Keith Morrison, 2004 validity Predictive validity

KEY FEATURES OF ASSESSMENT (1) • Assessment to empower pupils as learners. • Classroom assessment impacts significantly on the pupils’ sense of self, expectations, motivation and confidence. • Assessment should provide guidance to both teachers and pupils about what needs to be learnt next. • Assessment should embody an approach to teaching and learning in which the development of long-term dispositions is more important than short-term performance. Copyright Keith Morrison, 2004

KEY FEATURES OF ASSESSMENT (2) • The purposes are to be diagnostic and formative, providing feedback and being educative. • Teaching should be adjusted in light of assessment evidence. • Assessment should promote, not damage, student motivation and self-esteem. • Assessment should be constructively critical and provide rich, positive feedback and feedforward. • The assessments should be criterion-referenced and the criteria should be public. • The assessments should lead to diagnostic teaching. • Assessment should promote student self-evaluation. Copyright Keith Morrison, 2004

KEY FEATURES OF ASSESSMENT (3) • The assessments should be built on evidence rather than on intuition. • Assessment data should be derived from everyday classroom activities. • Assessment opportunities should be sought in everyday classroom activities. • Semi-structured approaches to gathering data are recommended, generating words rather than numbers (measures). • Assessments should be linked to the student teacher’s and the student’s action planning and target setting. Copyright Keith Morrison, 2004

KEY FEATURES OF ASSESSMENT (4) • Involve the students in the assessment process. • Communicate the assessment criteria to students. • Demonstrate validity and reliability. • Demonstrate fitness for purpose in deciding the method(s) of gathering assessment data and setting assessment tasks. • Select assessment methods that accord strongly with everyday teaching and learning processes. Copyright Keith Morrison, 2004