New Directions in Pupil Learning Assessment Emerging Models

New Directions in Pupil Learning Assessment: Emerging Models from the Teachers for a New Era Initiative California State University, Northridge Beverly Cabello Evidence Co-Leader, Teachers for a New Era James David Ballard Evidence Research Team, TNE "Extending the Culture of Evidence: Teacher Work Samples and other Promising Practices" to be held in Portland, Oregon, on July 19 - 21. Cabello 2006 1

Evidence • Goals: – To locate the connections between teacher education and pupil learning – To create a permanent viable system of data gathering, data analysis, and data sharing. – To inform our programs and improve teacher education, pupil learning, and program efficacy – To develop and test a theory of teacher change and impact Cabello 2006 2

CSUN’s Conceptual Framework of TNE Evidence Components Teacher Indicators Other than CSUN: formal education outside of CSUN, personal experiences, personal characteristics, & teaching experiences (if Interns, or former paraprofessionals); knowledge & beliefs about teaching & learning. Contextual Factors: (including district policies, school context, peers Indicators Related to Teaching Quality of CSUN credential recipients: teaching practices & how these are affected by district policies as well as school policies & context. Teacher Indicators within CSUN Formal University Preparation: Undergraduate preparation (at CSUN or elsewhere), Pathway, credential coursework & fieldwork. Cabello 2006 Pupil learning & achievement (a subset of which is measured by test scores) Individual Pupil Characteristics: previous educational experiences, English language proficiency, disabilities, etc. ) 3

• What is involved: Audit availably of data, quality of data, location of data, and access to data: • College level, • University sources, • System wide sources, • and District sources. • Lessons learned from these data sources: • • • What is there and not there Aggregate possibilities Types of variables, fields, and definitions Identifiers and confidentiality Various data programs, dirty data, and data conflicts • Existing and needed data sharing agreements, Union issues, privacy, etc. Cabello 2006 4

CSUN Pilot Study Research Questions 1. Do significant differences exist between the reading and math achievement of pupils in grades K - 12 and the specific CSUN teacher preparation programs their teachers experienced? 2. Does a significant relationship exist between teacher preparation program indicators of candidates‘ knowledge, skills & dispositions (e. g. course grades) and pupil learning indicators? Cabello 2006 5

Quantitative Study Categories of Variables • Pupil – Personal demographic – Family demographic (parents) – English Language fluency – All test scores • School context – Participation in Title 1 – Participation in Meal Program – Class size Cabello 2006 6

• Teachers (district information) – Number of years teaching in district – Levels of education – Type of credential – Emergency or intern – Demographics • Teachers (CSUN Preparation Data) – Pathway – Credential awarded (Ed. Specialist, other) – Level (elementary, secondary) – Demographics – Grades in professional courses Cabello 2006 7

CSUN PATHWAYS Multiple Subjects Credential Programs Integrated Teacher Education Program – (ITEP) Freshman Integrated Teacher Education Program (ITEP) - Junior 1053 Candidates Multiple Subjects Intern Program recommended in Traditional Program 2004/5 Accelerated Collaborative Teacher (ACT) Preparation Program Single Subject Credential Programs Single Subject Internship Program Four Year Integrated in English or Mathematics Traditional 826 Candidates ACT recommended in 2004/5 Cabello 2006 8

Estimated Marginal Means of CST Augmented Math Total Score at Grade Level = yrs 1 & 2 PATH B-MS C-SS Estimated Marginal Means 60 40 20 1999 2000 2001 2002 File. Yr Cabello 2006 9

Estimated Marginal Means of CST Augmented Math Total Score at Grade Level = yrs 3 & 4 PATH B-MS C-MS Estimated Marginal Means 100 C-SS 80 60 40 20 1999 2000 2001 2002 File. Yr Cabello 2006 10

Estimated Marginal Means of CST Augmented Math Total Score at Grade Level = yrs 5 & 6 PATH B-MS C-SS Estimated Marginal Means 80 60 40 20 0 1999 2000 2001 2002 File. Yr Non-estimable means are not plotted Cabello 2006 11

Estimated Marginal Means of SAT 9 Math NCE at Grade Level = yrs 1 & 2 55. 0 PATH Estimated Marginal Means B-MS C-SS 50. 0 45. 0 40. 0 35. 0 1999 2000 2001 2002 2003 File. Yr Cabello 2006 12

Estimated Marginal Means of SAT 9 Math NCE at Grade Level = yrs 3 & 4 PATH Estimated Marginal Means 55. 0 B-MS C-SS 50. 0 45. 0 40. 0 35. 0 30. 0 25. 0 1999 2000 2001 2002 2003 File. Yr Cabello 2006 13

Estimated Marginal Means of SAT 9 Math NCE at Grade Level = yrs 5 & 6 PATH Estimated Marginal Means 60. 0 B-MS C-SS 55. 0 50. 0 45. 0 40. 0 35. 0 30. 0 25. 0 1999 2000 2001 2002 2003 File. Yr Non-estimable means are not plotted Cabello 2006 14