Comprehensive Accountability Systems A Framework for Evaluation Kerry

Comprehensive Accountability Systems: A Framework for Evaluation Kerry Englert, Ph. D. Paper Presented at the Canadian Evaluation Society June 2, 2003 1

Accountability – Positives and Negatives n Positives ¡ ¡ Focusing teachers on helping all students learn Providing schools with data so that they can make improvements Providing educators and policymakers information on what educational programs are working Providing parents with data about how their child and child’s school is doing 2

Accountability – Positives and Negatives (cont’) n Negatives ¡ ¡ Narrowing the curriculum Too much time testing draws away the focus from teaching Focus on bringing students to proficient and not to advanced Increase retention rates, or place more students in special needs classes. 3

Purposes of Accountability Systems n In education, the purpose of an accountability system is generally ‘to improve student learning’ ¡ ¡ ¡ Informing students, parents, teachers Monitoring the learning process and holding students, schools, teachers, and states responsible for the learning outcomes Certifying teacher quality Evaluating school and program effectiveness Ensuring equal opportunities to learn for all students 4

Characteristics of Effective Accountability Systems n Clear standards and expectations ¡ ¡ The standards must be explicit so that teachers and student must know what must be achieved. High expectations for all students – in order to ensure equity, all students should be held to high standards. 5

Characteristics of Effective Accountability Systems n High-quality assessments must be: ¡ ¡ Aligned to standards – The degree to which an assessment mirrors the content standards. Reliable – Is the degree to which a test accurately measures the students’ true ability on a construct. Valid – Refers to the degree a test measures the intended construct and the appropriateness of the inferences made from the test scores (Messick, 1989). Fairness provisions – Reduce content irrelevant variance. 6

Characteristics of Effective Accountability Systems n Multiple measures ¡ ¡ Because error is present in all tests, examining homework, classroom assessments, and other standardized tests allow for a more complete picture of what students know and are able to do. By incorporating multiple measures, other important factors of effective schools can be accounted for such as drop out rates, school safety, expulsion rates, and attendance. 7

Characteristics of Effective Accountability Systems n n n Readily understandable to the public and have stakeholder support Diagnostic uses for data – The data should be easily usable for teachers. Sanctions and rewards linked to results Flexibility and fairness to allow for local differences and creativity Alignment of resources, support, and assistance for improvement 8

Moving Forward n n The primary focus of this phase of the research project was to summarize the existing literature The next phase will examine the how districts currently evaluate their accountability system and how they are using the information to monitor learning. 9

Contact Information n Kerry Englert 2550 South Parker Rd. Suite 500 Aurora, CO 80014 (303) 632 -5627 kenglert@mcrel. org n A copy of the complete document Comprehensive Accountability Systems: A Framework for Evaluation can be found at: http: //www. mcrel. org/topics/product. Detail. asp? topic s. ID=1&product. ID=128 10