TQP A Longitudinal Study of Ohio Teacher Education

TQP: A Longitudinal Study of Ohio Teacher Education Spring 2007 Update OCTEO Crowne Plaza Dublin April 5, 2007

Factors and Forces • Higher Education Act of 1998 – Title II Report on Teacher Quality • Ohio Report Card Task Force (1999 -2001) – OCTEO Focus Groups • Political will formed between Ohio Department of Education, Ohio Board of Regents, Governor’s Office, Teacher Education Institutions (Private and Public), and School Districts • Factors in Ohio – Standards Based Curriculum – Value-Added Measurement (VAM) – SB 311 signed into law January 3, 2007

Factors and Forces • Confidentiality Promise Promotes Collaboration – Participants (all data coded) – Colleges and Universities • Preservice, Demographics, Inservice data • Ohio aggregated data reported publicly • Institutions get their own data to use in program review and accreditation reports – Districts • Inservice data • Ohio aggregated data reported publicly • District gets their own data to use in EY, PD review – FERPA ruling from OSU attorney

Stakeholders and Participants • All 50 Ohio Teacher Preparation Institutions – – Institutional Representatives (50) Principal Investigators (UC, OSU, UD) Research Design Teams (13 IHEs) Field Researchers (10 IHEs – changes as needed) • District Partners – Entry Year Coordinators – Administrators in field study districts • Ohio Advisory Board • External Audit Panel • Multiple federal, state, corporate, and foundation funders

TQP Research Questions 1. Do variables of teacher background, initial preparation, and on-going professional learning relate to teacher practices, student learning and achievement? 2. How do specific elements of teacher preparation and aspects of school contexts impact novice teachers’ development during their first three years of teaching? 3. Do high value-adding teachers (HVATs) have characteristics, instructional practices, and understandings that differ from other teachers along the value-added continuum? 4. What specific school contexts are associated with HVA novice and experienced teachers?

Benefit to Institutions • If 5 student teachers participate, a customized report is given to each institution. It is not made public by TQP. • Both pre-service and in-service surveys are done at TQP expense using valid and reliable instruments based on national studies. • Patterns identified on annual reports have been useful for internal review and reform. • An NCATE/TQP cross-walk has simplified using survey data in accreditation reports. • Comparison of local and state, sometimes national, data. • Ultimately intend to use “best practices” findings to improve preparation program.

A Public University’s Story • Facilitated discussion within the teacher education unit by presenting questions to consider and a common language for the discussion. • Assisted in responding to a need to cultivate a climate of assessment. • Unit developed a systematic approach to the organization and display of data • Identify trends within the teacher education unit and follow over time.

Story (con’t. ) • Compare teacher candidates’ demographics and program perceptions with those of other Ohio programs. • Compare perceptions and demographics among and between our main and regional campuses. • Trace and quantify changes in perceptions with regard to specific subsets within the data. • Access to longitudinal data about candidate perceptions of individual licensure programs which inform programmatic decisions.

Challenges to Institutions • • • Getting surveys administered Collecting and submitting demographic datafile Maintaining student confidentiality Securing and meeting IRB requirements Using the data to inform: – Program and policy decisions – Supply/demand issues • Tension between external regulatory bodies and internal program implementation and resources

Benefit to Districts • If 5 teachers participate, a customized report is given to the district. It is not made public by TQP. • The only available source of Praxis III data. • Comparison of local, state and national data on induction and mentorship.

A Large Urban District’s Story • TQP Partnership agreement: – Agreed to and signed by Superintendent and Union President, March 2005. – Administer TQP Inservice Teacher Survey annually to all novice teachers hired Fall 2005 – Spring 2009. – Help recruit novice teachers for three year follow-up case studies. – Help recruit experienced teachers for one year case studies. – Work with TQP to get teacher-level value-added scores into the TQP data set.

Expectations • Learn what the most effective teachers do to promote academic progress of students in their district. • Improve hiring practices by knowing what an effective district teacher looks like. • Improve teacher retention through enhanced induction and professional development programs. • Ultimately improve student learning in the district.

2005 – 2006 School Year • Informational meetings early September • Lots of communication from administration and union encouraging teachers to attend meetings • All novice teachers encouraged to complete survey. • A subset of novice and experienced teachers participated in the pilot.

Challenges in Large Urban Districts • Late summer/early fall teacher hires and placements • Concern about Praxis III and other beginning teacher issues • Entry Year Teachers (EYTs) are first to get RIF notice. They seek jobs in other districts rather than risk not being called back. • If re-hired, may not be in same grade/subject. • Pressure to meet assessment standards.

Longitudinal Graduate Follow-up Study • Survey student teachers ‘ 03 -’ 04 through ’ 08 -’ 09 – Anticipate 30, 000+ participants – Printed or on-line surveys administered by institutional liaisons – Expect to add value-added scores of graduates • Follow novice teachers each year through ’ 08 -’ 09 – Anticipate 5, 000+ participants – Printed or on-line surveys

Pre-service Subscales Report Example

• Sample mid-point findings from: – Demographic Database (submitted by Institutional Representatives at each IHE) – TQP Preservice Teacher Survey (completed during student teaching, usually in a seminar) – TQP Inservice Teacher Survey (completed each spring 2004 – 2009 hard copy or on-line)

Average GPA of 15, 248 teacher completers = 3. 48 * As High School Seniors ** National Percentiles N = 17, 127 teacher completers in Ohio from Academic Years 2004, 2005, and 2006.

N for Ohio TQP Teacher Completers = 17, 127

TQP Ohio 2005 and 2006 Entry Year Teacher's Satisfaction Level: Mean Score by Item 7. 00 6. 00 5. 59 5. 00 Mean 6. 18 5. 82 4. 73 4. 97 its t en m e 5. 21 5. 01 4. 00 3. 00 2. 00 1. 00 /f ry la Sa ge rin f ne be nc a dv a l na io s es s tie ni u rt o pp O sio s fe ro n o isi l/p of r rp fo a ch l na na so a on Scale: 1=Very Negative; 7=Very Positive N=2, 805 -2, 812 depending on the item ns tio di l ra G e en rk wo n co o (h la , c s ur u lo L e, ss n to i ss fe ro p f g m om la e ev in ak or th u a c de y/ r pe of l ve Le . ) tc ity e ng lle siz rk wo e d, oa g ea ue s ll l ns ith co th ns w tio c ra I e nt ts en r te In tio ac wi s d tu

New Ohio teachers report being observed little by their mentors in a typical week N=2, 305

Efficacy for Preservice (Cohorts I, III) and Inservice (Cohorts I, II)

Efficacy for Preservice (Cohorts I, III) and Inservice (Cohorts I, II)

Notes on the Data – Self-report, exams, interviews, documents, VAM – Reported for items and sub-scales – Surveys are too long, but… • What to cut? • Sub-scale reliability consistent throughout – Interpretation: • Implicit standard (4 on 1 -5 scale) • Relative score compared to other items, sub-scales • Confidence intervals – Ultimately, findings will be based on Structural Equation Modeling related to VAM

• Challenges of Large-Scale Collaboration – – – – – Ownership, Turf, and Trust Data Collection on Main and Regional Campuses Multiple Data Bases that Don’t Connect Well Communication Both Internal and External Resources vs. Costs Setting Priorities Follow-through on Tasks Recruitment of Participants Maintaining Political Will

Sustaining the Partnerships – Ability to share and use data – Common language used statewide – Some issues transcend local context; other issues vary according to resources and other local decisions – Respond to emerging needs • Revise protocol to collect needed data • Work with ODE/OBR/ESB to develop legislated reports • Enable on-line surveys, submission and retrival of data – Communication through email, meetings, newsletters

What questions or comments do you have ? For Additional Information: http: //www. tqpohio. org