Streamlined TELPAS Holistic Rating Training System for Spring

Streamlined TELPAS Holistic Rating Training System for Spring 2011 December 2010 Texas Assessment Conference TEA’s Student Assessment Division, TELPAS Team Cristina Vázquez, Manager Juanita Juárez, Program Specialist Pearson’s Online and Technology Projects, Laura Likon, Program Manager

Topics l l Streamlined holistic rating training system for spring New supplemental support training models, including the training of supplemental support providers Reports to monitor course completions and calibration activities Updates to the Texas Training. Center 2

Holistically Rated Components of TELPAS Grade Levels Language Domains K– 1 Observational assessments in listening, speaking, reading, and writing 2– 12 Observational assessments in listening and speaking; writing collections for writing Online multiple-choice reading tests are used to assess the reading domain for grades 2– 12. 3

Overview of the Rating Process l l Districts identify ELLs in grades K– 12 and designate raters to assess students Test administration procedures and holistic rating training conducted to prepare grades K– 12 raters to assess English language proficiency consistent with the holistic scoring rubrics, the Proficiency Level Descriptors (PLDs) Raters gather writing samples for ELLs in grades 2– 12 to include in each student’s writing collection Raters use PLDs to assign students a rating in each language domain assessed 4

Role of Testing Coordinators in Holistic Rating Training l l Holistic rating training no longer job of state-authorized teams of TELPAS trainers District and campus personnel have primary responsibility for supporting holistic rating training Vital for testing coordinators to have good working knowledge of holistic rating training As in past, testing coordinators responsible for ensuring l l l adequacy of training sufficient numbers of raters Collaboration with bilingual/ESL specialists encouraged, but testing coordinators ultimately responsible for TELPAS administration 5

Holistic Rating Training Requirements

Fall ELPS-TELPAS Professional Development (For new teachers and new prospective raters -- recommended) Spring TELPAS Administration Procedures Training Covers holistic rating training requirements and procedures for conducting TELPAS New Raters Returning Raters Online Basic Training Course Online Calibration Activities Sets 1 and 2: Set 2 required if not successful on Set 1 If not calibrated: Supplemental Holistic Rating Training Online Calibration Activities Sets 1 and 2: Set 2 required if not successful on Set 1 Final Online Calibration Activity Set 3 7

Key Features of Streamlined Training System l l l The online holistic rating training of new raters now occurs only in spring, rather than in both fall and spring. The training processes for K– 1 and 2– 12 raters are now aligned. New supplemental training model will be implemented to support the needs of raters who do not calibrate by end of second set. The Texas Training. Center website has been redesigned to make it easier to use. The Training. Center reports for coordinators to use to monitor training completion and performance have been revised and simplified to reflect the streamlined processes. 8

Foundational (Awareness) Training l Training resources: l Power. Point modules § § § l Making the ELPS-TELPAS Connection: K– 12 Introduction Introductory Training on the PLDs (separate modules for K– 1 and 2– 12) Grades 2– 12 Writing Collection Overview New Educator Guide to TELPAS l Notification about resources sent to ESCs and districts Aug 2 l Experienced TELPAS personnel may provide training Additional Resource: A Power. Point (2010– 2011 Holistic Rating Training Requirements) to help administrators understand the holistic rating training requirements is available on the ELL Assessment webpage at www. tea. state. tx. us/student. assessment/ell/. 9

Alignment of Training Requirements for K– 1 and 2– 12 l l New raters complete an online basic training course and online calibration activities Returning raters complete only online calibration activities Important Note: Individuals are not authorized by TEA to serve as raters unless they complete the state-required training and calibration activities. Raters who complete all requirements but do not successfully calibrate may be authorized to serve as raters at the discretion of the district, but districts are required to provide rating support in all domains for these raters. 10

General TEA Definitions of New and Returning Raters who have completed K– 1 training but not 2– 12 training will be new raters if they switch to 2– 12 training, and vice versa. 11

General TEA Definitions of New and Returning Raters l A district may, at its discretion, require a returning rater to repeat new rater training. l l This may be advisable if testing personnel believe that a returning rater who was unable to meet rating accuracy requirements in the past would benefit from retaking the online basic training course prior to calibration. Additionally, TEA recommends that districts consider requiring this if they have not completed TELPAS training since the 2007– 2008 school year. 12

Spring 2011 Online Training Components l Online Basic Training Course (Required for New Raters) l l Online Calibration (Required for New and Returning Raters) l l One for raters of K– 1 and one for raters of grades 2 and above Separate calibration activities for grades K– 1, 2, 3– 5, 6– 8, and 9– 12 Assembling and Verifying Grades 2– 12 Writing Collections l Optional course that may be used to supplement test administrator procedures training 13

Online Calibration Activities l l There are three sets of online calibration activities. Each calibration set contains 10 students to rate, and all holistically assessed language domains are represented in each set. l l l For K– 1: listening, speaking, reading, and writing For 2– 12: listening, speaking, and writing A rater needs to rate at least 70 percent of the students correctly to demonstrate sufficient calibration. 14

Online Calibration Activities l Raters complete only as many sets as it takes to calibrate. l l Many raters will be able to calibrate on the first set of activities, and many others will be able to calibrate by the end of the second set. With supplemental support, very few raters should have difficulty calibrating by the end of the third and final set. 15

Grade Clusters of Training l l l K– 1 training components are separate from those of grades 2 and above It is important that raters know which grade cluster of training to complete For rater with students in multiple clusters in grades 2– 12 l l l Rater required to train in the cluster with most ELLs As a best practice, rater should review some online basic training course practice activities in additional cluster(s) For rater with students in multiple clusters including K– 1 l l l Rater required to train in K– 1 and one other cluster in grades 2– 12 Rater of K– 1 who will rate more than one additional cluster within grades 2– 12 should train in the cluster (2, 3– 5, 6– 8, 9– 12) with the most ELLs As a best practice, rater should review some online basic training course practice activities in other cluster(s) 16

Key Dates for Holistically Rated Components Date Activity 1/3 Launch of Assembling and Verifying Grades 2– 12 Writing Collections course 1/3– 7 TELPAS Manual due in districts (available online in December) 1/7 End date for district coordinator training—all TELPAS components 1/10 Launch of online basic training courses for new K– 1 and 2– 12 raters (course should not be taken until after rater administration procedures training) 1/21 End date for campus coordinator training—holistically rated components 1/18– 2/1 TEA conducts web-based supplemental support TOTs 1/31 Calibration window opens for new and returning raters—first 2 sets 2/1 End date for administration procedures training – holistically rated components 2/1 Earliest eligibility date for TELPAS writing samples 2/16 3 rd and final calibration set available (supplemental training window starts) 3/7– 4/8 TELPAS assessment window 17

Supplemental Support After Second Calibration Set for Raters l l l Raters not successful after calibration set 2 to get supplemental support and then complete set 3 before start of testing window Raters need access code to complete set 3 Supplemental training resources to be posted in Resources area of Training. Center website (in section requiring coordinator access) Districts, not ESCs, have primary responsibility for supplemental training of raters. 18

Training of Supplemental Support Providers l TEA conducting several web-based training-of-trainer (TOT) sessions January 18–February 1. l l l Registration period ends December 17, 2010 Helps districts prepare to meet the needs of any raters who have difficulty calibrating by end of set 2 Enables districts to provide raters with supplemental support locally ESC and district staff who attend web-based training may train more support providers afterward as needed Districts should train a sufficient number of district and campus support providers to meet their needs. l l Encouraged to train at least one supplemental support provider for each of the following grade spans: K– 1, 2– 5, and 6– 12 Districts with substantial ELL populations may find it beneficial to have a supplemental support provider on each campus 19

Criteria for Supplemental Support Providers l A supplemental support provider must be a 2009– 2010 l l l state-authorized trainer, or fully trained and refreshed K– 1 rater, or fully trained, qualified, and refreshed 2– 12 rater Individual must attend a TEA web-based training or turnaround training session Individual must calibrate on either set 1 or 2 20

Supplemental Support Provider TOT Sessions Web-Based Training Dates Date Time (CT) January 18 1: 00– 4: 30 January 20 8: 30– 12: 00 January 24 1: 00– 4: 30 January 26 8: 30– 12: 00 January 31 1: 00– 4: 30 February 1 8: 30– 12: 00 21

Two Basic Models for Providing Supplemental Support l Individual or group sessions l l Raters attend a one-on-one or group session with a supplemental support provider before completing their third and final calibration set. Brief meeting followed by additional consultation at rater’s request l l Rater meets briefly with the supplemental support provider for instructions on resources to review. Rater schedules time with the support provider to get clarification, if needed, before completing the third and final calibration set. Note: The models are described in detail in TELPAS Campus Coordinator Activity 6 of the Coordinator Manual. 22

End Dates for Holistic Rating Training ● TEA recommends end date of March 7, first day of TELPAS assessment window ● Raters are directed, however, to complete training by the dates established by their testing coordinator and that they must complete training requirements before rating their students ● Basic training courses and calibration activities will be open throughout TELPAS assessment window to l l allow districts to handle extenuating circumstances allow raters to refer back to course information and practice activities as they rate their students 23

Training Certificates l Training certificates are generated for raters who complete training courses and calibration activities. l l A certificate of completion is issued when a rater completes the online basic training course. A certificate of successful completion is issued when a rater demonstrates sufficient calibration (70 percent or better) on a calibration set. 24

Updates to Training Certificates l l Professional development hours will now only be indicated in the rater’s training history within the LMS, not on the training certificate. l Four hours are awarded for the K– 1 basic training course, l Five hours for the 2– 12 basic training course, and l Two hours are awarded (in all) for calibration activities In either case, rater prints certificates from Training History and turns them in to campus coordinator. 25

Responsibilities of Testing Coordinators for Holistic Rating Training l l Ensuring raters understand the importance of being properly trained on the holistic rating process Ensuring new and returning raters understand which training to take and for which grade clusters Monitoring that all raters complete their training requirements and receive supplemental training support if needed Ensuring that any individual who serves as a rater but is not successful on the calibration portion of training is provided rater support during the TELPAS administration 26

Monitoring Course Completions and Performance on Calibration Activities

Available Reports for Online Training and Calibration l TELPAS Returning Rater Planning Roster l l TELPAS Confidential Course Completion Roster l l Lists online basic training course completion and in-progress status and performance by user for the current year TELPAS At-a-Glance Training & Calibration Report l l Uses LMS records from the 2008– 2009 and 2009– 2010 school years to generate a list of recently trained raters based on TEA’s recommendation Provides a comprehensive list showing basic training course and calibration completions for all registered users for the current year TELPAS Confidential Calibration Summary Report l Provides calibration summary information by grade cluster (K– 1, 2, 3 – 5, 6– 8, 9– 12) and for grade clusters combined 28

Who Can Access Reports from the Texas Training. Center l l l Region and district testing coordinators are given administrative access by Pearson Region and district testing coordinators may choose to give administrative access to individuals as assistants at the region or district level Campus testing coordinators receive administrative access upon approval by district test coordinators or district coordinator assistants For more information about how to get administrative access, refer to the TELPAS Coordinator’s User Guide for Online Holistic Rating Training in Appendix F of Coordinator Manual. 29

TELPAS Returning Rater Planning Roster 30

TELPAS Confidential Course Completion Roster 31

TELPAS At-a-Glance Training & Calibration Report 32

TELPAS Confidential Calibration Summary Report 33

Important Notes about Reports l l l Personnel with administrative authorization are able to access reports on demand to monitor TELPAS online training and calibration. The reports are updated nightly. In order for users to appear in the correct reports, their location (region, district, campus) must be up to date in the My Info section of the Training. Center. Users who have registered in the Training. Center and have started but not completed a calibration set are not included in the summary reports. Only completed sets are presented in the reports. However, users who have started a training course will appear in the course roster as being “in progress. ” 34

Updates to the Texas Training. Center 35

TELPAS Training. Center Updates l Improvements made for easier navigation to the correct training component 36

TELPAS Training. Center Updates 37

TELPAS Training. Center Updates 38

TELPAS Training. Center Updates 39

TELPAS Training. Center Updates 40

Resources l 2011 District and Campus Coordinator Manual l l 2011 TELPAS Manual for Raters and Test Administrators l l l Includes the Fall 2010–Spring 2010 Rater’s User Guide for Online Holistic Rating Training 2011 Educator Guide to TELPAS Grades K– 12 Assembling and Verifying Grades 2– 12 Writing Collections online course l l Includes the Fall 2010–Spring 2011 Coordinator’s User Guide for Online Holistic Rating Training Found on the Texas Training. Center Power. Point training modules available on TEA’s Student Assessment Division website on the ELL Assessments page: l l Making the ELPS-TELPAS Connection, Grades K– 12 Introduction Introductory Training to PLDs (for grades K– 1 and 2– 12) Grades 2– 12 Writing Collection Overview 2010– 2011 Holistic Rating Training Requirements 41

Holistic Rating Training TETN l When: January 7, 2011 l Event#: 8302 l Summary: Session for coordinators to review spring online holistic rating training and calibration procedures. There will also be a review of the Texas Training. Center website updates and reports. 42

Contact Information l TEA’s Student Assessment Division l l l E-mail address: ELL. tests@tea. state. tx. us Phone: 512 -463 -9536 Pearson’s Austin Operations Center l l l E-mail address: TELPAS. techhelp@support. pearson. com Phone for coordinators: 800 -252 -9186 Phone for raters: 800 -627 -0225 43