The University of Texas System Assessment of Teacher

The University of Texas System Assessment of Teacher Preparation Programs Funded by the Houston Endowment September, 2004 1/9/2022 1

Overview of Presentation • • • 1/9/2022 National Center for Educational Accountability Introduction to the study Research questions Indicators Data sources Data security and confidentiality Teacher Survey Data collection (Mary Lummus-Robinson) School district partnership (Holly Williams) Questions & discussion 2

National Center for Educational Accountability • Created in 2001 • Sponsoring organizations • Education Commission of the United States • Just for the Kids • The University of Texas at Austin • www. nc 4 ea. org 1/9/2022 3

Introduction to the study Purpose: To identify relatively high performing teacher certification programs in the UT System and to determine what makes these programs so successful and what staff from other programs can learn from them. 1/9/2022 4

Introduction to the study • UT System requested development of a system for evaluating teacher preparation programs • 4 -year study of graduates from 9 UT System components – 4 cohorts of graduates beginning in 2002 -2003 – teaching in Texas public schools • grades 4 -8 • grades 9 -11 on an exploratory basis 1/9/2022 5

Introduction to the study • Unique contributions of the study – Value-added assessment of student growth – Colleges of Education evaluated on the basis of the performance of the graduates’ students – Translation of evaluation results into improvement of teacher preparation programs 1/9/2022 6

Research Questions • 1/9/2022 Is teacher preparation program related to students’ growth on the Texas Assessment of Knowledge and Skills (TAKS) when controlling for teachers’ prior academic achievement and for school characteristics? 7

Research Questions • 1/9/2022 Is teacher preparation program related to graduates’ performance on teacher certification exams when controlling for graduates’ prior academic achievement? 8

Research Questions • 1/9/2022 Is teacher preparation program related to graduates’ selfreported teaching ability when controlling for graduates’ prior academic achievement and for school characteristics? 9

Research Questions • 1/9/2022 What are the characteristics and content of the most highly effective teacher preparation programs? 10

Indicators • Academic growth of graduates’ students as measured by the Texas Assessment of Knowledge and Skills (TAKS) • Graduates’ teacher certification exam scores (TEx. ES/Ex. CET) • Graduates’ self-reports of their teaching ability as indicated on the Teacher Survey • Qualitative data on the characteristics and content of each program 1/9/2022 11

Data Sources • School districts • State Board for Educator Certification (SBEC) • Graduates • UT System components • Texas Education Agency (TEA) 1/9/2022 12

Data Security and Confidentiality • Protect confidentiality of the data • Merge datasets – – 1/9/2022 Encryption algorithm developed at TEA Shared with UT System and SBEC School district data to TEA for encryption Files merged at NCEA after encryption 13

Teacher Survey • Interstate New Teacher Assessment and Support Consortium Core Standards – How well do you do the following? – How well did your teacher preparation program prepare you to do the following? – How well have your in-school training experiences prepared you to do the following? 1/9/2022 14

Teacher Survey • Interstate New Teacher Assessment and Support Consortium Core Standards (item examples) – Adjust plans to meet students’ needs – Foster relationships with colleagues – Communicate student progress to parents 1/9/2022 15

Teacher Survey • Rating of overall teaching ability • Attributions of teaching ability – Teacher preparation program – In-school training experiences – Natural ability – Other 1/9/2022 16

Data Collection Overview • Data sources – – – 1/9/2022 5, 297 Teacher Preparation Program Graduates 1, 675 Texas High School Principals 9 UT System academic components Texas Education Agency (TEA) State Board for Educator Certification (SBEC) 102 school districts 17

Teacher Survey • Administered online March-August 2004 • 5, 297 Graduates sampled for online survey • 15% Response Rate – 6 attempts made to reach target graduates • • 1/9/2022 Introductory letter Follow up letter Two email reminders Two mailings including a hard copy of the survey 18

Data Collection • UT Components – Graduates’ prior academic achievement • Transcript data • Aggregates for college – Specific data collected at each institution varies • State Board of Educator Certification – Certification type and level – Exam scores – Years of experience – Employment information 1/9/2022 19

Data Collection • Texas Education Agency – Teacher and student demographic data – Holding all district data submitted • Teacher Preparation Program Staff Survey – Specify unique attributes of each program • 100% response rate 1/9/2022 20

Data Collection School Districts • Districts procedures for submitting research request vary – 102 districts contacted • 8 had detailed procedures – Followed set procedures • 94 had no procedures – NCEA developed generic request 1/9/2022 21

Data Collection School Districts • Between 4 -12 contacts made with each district • Introduction • Specific Data Request • Follow up via Mail, Phone Calls, and Email • Follow up Data Request • Repeat 1/9/2022 22

Data Collection School Districts • 102 School Districts contacted • 51 Districts participated in study • 51 Districts declined – – – 1/9/2022 35 Districts had no response 6 Districts had problems extracting data 5 Districts declined citing strain on resources 4 Districts’ superintendents declined participation 1 District refuses all research unless state mandated 23

District Participation • Austin Independent School District – Participation in the NCEA Project will help lead to • Ongoing improvement in UT System teacher preparation programs • A more Highly Qualified applicant pool for our District • Academic growth for AISD students 1/9/2022 24

New Teachers • New teachers in AISD, 2004 -05 – 800 new teachers hired – 300 of these hold an alternative certification • Special interest in findings regarding student outcomes for teachers who are alternatively certified • Results and recommendations in this area will likely impact our future hiring practices 1/9/2022 25

Research in AISD • Due to an increase in requests by external groups to conduct research in AISD, in 1996, our Dept. formalized a Research Proposal Application and Review Process (similar to an IRB process) – Researcher submits application describing methodology of the study and the benefit to district (2003 -04 N=70) – Committee of staff with expertise in the appropriate program area(s) and staff with expertise in research methodology review application – Project is approved, approved with modifications, or declined 1/9/2022 26

NCEA Project • NCEA submitted application – Project was approved because of the clear benefit to AISD • Many of our teachers come through the UT system; • The content area of Teacher Preparation is of interest, particularly in light of NCLB requirements; • The design of the study and the evaluation system was well thought out and clearly articulated; and • The project did not infringe upon student or campus staff time. 1/9/2022 27

District Expectations • We look forward to reaping the benefits of this project – Expectations include • Best Practice findings will be used to make modifications, where recommended, to teacher pre-certification training in the UT system • Outcomes regarding alternatively certified teachers will help district staff make more knowledgeable hiring decisions • The evaluation system will be used for continuous improvement in UT system teacher preparation programs 1/9/2022 28

Wrap-up • Questions • Available information – Business cards – Higher Achievement for All Students (NCEA) – 9 Essential Elements of Statewide Data Collection Systems 1/9/2022 29