TELPAS 2017 Holistic Rater Training for Teacher Raters

TELPAS 2017 Holistic Rater Training for Teacher Raters & Verifiers Date: CTC:

DISCLAIMER This training Power. Point does not take the place of reading the appropriate test administrator manuals. 2017 TELPAS

TELPAS Basics

Definitions • ELLs o English Language Learners • ELPS o ELPS = English Language Proficiency Standards o Federally required instructional standards designed to ensure that ELLs are taught the academic English they need for school purposes • TELPAS o TELPAS = Texas English Language Proficiency Assessment System o A federally required assessment program designed to measure the annual progress that ELLs make in learning academic English • TELPAS Raters o Teachers trained to assess ELLs for TELPAS 2017 TELPAS

TELPAS Assessment • Grades K– 1 – Holistically rated observational assessments of listening, speaking, reading, and writing • Grades 2– 12 – Holistically rated observational assessments of listening and speaking – Holistically rated writing collections – Multiple-choice online reading tests in six different grade clusters: 2, 3, 4– 5, 6– 7, 8– 9, 10– 12 2017 TELPAS

TELPAS provides proficiency level ratings for each language domain, plus an overall, composite rating. 2017 TELPAS

Summative TELPAS Assessment For each domain (listening, speaking, reading, and writing), TELPAS answers the following question: How well is the student currently able to understand use English during grade-level instruction? 2017 TELPAS

TELPAS Results TELPAS results are used to — • set learning goals for ELLs • keep parents and students aware of annual progress in learning English • inform instructional planning and bilingual/ESL program exit decisions • report performance to the public • evaluate programs, resources, and staffing patterns • evaluate districts and campuses in federal and state accountability and monitoring indicators 2017 TELPAS

Who Takes TELPAS? All ELLs in grades K– 12 are assessed annually, including those whose parents decline bilingual/ESL program services Do NOT test M 1 or M 2 students – They are NOT LEP and have been exited. • In rare cases, an ELL served by special education may be exempted from TELPAS by the ARD committee in conjunction with the LPAC • ARD exemption is on a domain-by-domain basis as a result of a severe disabling condition and is addressed by the LPAC and ARD committees and documented in the student’s IEP 2017 TELPAS

Test Security & Confidentiality

Test Security and Confidentiality (Rater Manual, for complete list refer to pages 3 -7) � Testing Irregularities � Serious Violations include: � Directly or indirectly assisting students with responses to test questions or writing samples � Tampering with student responses or writing samples � Falsifying TELPAS holistic ratings � Using answer sheets or notes that contain answers to complete TELPAS calibration activities � Viewing secure test content during or after an assessment without permission � Discussing secure test content, student responses, or student performances � Scoring student reading tests, either formally or informally � Duplicating, recording, or capturing electronically secure test content without permission from TEA � Cheating on Calibration Test or taking Calibration Test “not” in 2017 TELPAS 11 monitored setting

Test Security and Confidentiality (Rater Manual, pages 3 -7) � Procedural Errors � A test administrator failed to issue the correct materials, or students were provided nonallowable materials. � Testing personnel who were not properly trained were allowed to rate students or administer tests. � A test administrator failed to use the test administrator manual or failed to read aloud the bolded test administration directions as outlined in the manual. � A test administrator failed to remove or cover all instructional displays that might aid students by providing direct answers during testing. � A TELPAS writing collection was not submitted in accordance with required assembly criteria. � A test administrator failed to provide a student with the correct student authorization to access the TELPAS reading test. 2017 TELPAS 12

Test Security and Confidentiality (Rater Manual, pages 3 -7) � Eligibility Error �Eligible students were not rated in one or more domains. �Eligible students were not administered the grades 2 -12 reading test. �Ineligible students were assessed. � IEP Issue �A student receiving special education services was provided an unapproved or undocumented accommodation or was not provided a prescribed accommodation. �A student receiving special education services was administered the wrong test. � Improper Accounting for Secure Materials �A rater, test administrator, campus coordinator, or district coordinator lost or misplaced writing collections or other secure materials. 2017 TELPAS 13

Test Security and Confidentiality (Rater Manual, pages 3 -7) � Monitoring Errors – for Online TELPAS Reading Testing �A campus coordinator did not require raters to return confidential materials �A test administrator left a room unattended when students or secure materials were present or when secure online tests were open and visible �Secure online tests were left open and visible during a lunch break, a short break taken in the test room or restroom breaks �Testing personnel did not monitor students during a break �A test administrator did not ensure that students worked independently during testing �Students were not prevented from using cell phones or any other electronic device to take pictures or send messages �A student was allowed to remove secure materials from the testing area. 2017 TELPAS 14

Security Oaths • Any person who has more than one testing role (reading test administrator and rater) must receive appropriate training and sign a security oath for EACH role. • All Oaths found in Appendix A: – TELPAS Raters – TELPAS Writing Collection Verifiers – TELPAS Test Administrators – TELPAS Technology Staff 2017 TELPAS 15

Holistic Ratings The Basics

Language Domain Definitions 2017 TELPAS

Proficiency Level Definitions For all grade levels and domains, ratings are Beginning, Intermediate, Advanced, or Advanced High 2017 TELPAS

Holistic Rating Process & Participants

Overview of the Rating Process • Campuses identify ELLs in grades K– 12 and designate Raters to assess students • Test administration procedures and Holistic Rating training conducted to prepare grades K– 12 Raters to assess English language proficiency consistent with the holistic scoring rubrics, the Proficiency Level Descriptors (PLDs) • Raters gather writing samples for ELLs in grades 2– 12 to include in each student’s writing collection • Raters use PLDs to assign students a rating in each language domain assessed (PLDs – Rater Manual pages 29 -35) 2017 TELPAS

Who is Involved in the Rating Process? • Campus Testing Coordinator • Verifier (assists with Writing Collection) • LPAC Administrator (assist with providing a list of all ELL students) • Raters (certified personnel) 2017 TELPAS

Responsibilities of TELPAS Raters • Rater must have the student in class. • Rater must rate the student in all domains for which the student is eligible. A student is not permitted to have one rater for some domains and another rater for another domains. • Assist as directed by the CTC/LPAC administrator in collecting writing samples. Follow all timelines as directed. • Study and pass the TELPAS Calibration Test. • Review the Basic Training Course as needed to pass the Calibration Test. • New Raters and Raters with a 3 year lapse: Required to complete the basic training course before attending monitored Calibration Test • Lost certificate = Complete Basic Training Course • Complete Student Rating Roster and submit by deadline. 2017 TELPAS 22

TELPAS Verifier Responsibilities • Must be certified individuals - they can be Teachers, Librarians, Counselors, or Administrators • Verifiers cannot be paraprofessionals or individuals who are also Raters • Complete the online “Assembling and Verifying Grades 2 -12 Writing Collections” training course • Manage Writing Collections • Provide each Writing Collection with a “Writing Collection Cover Sheet” and “Writing Collection Verification Checklist” • Campuses must have at least one Verifier, but it is suggested to have more than one, or at least train a back-up Verifier 2017 TELPAS

TELPAS Student Rating Roster ALL ELL students are listed on a Rating Roster including: • Students receiving Special Education Services who will be exempted from one or more domains • Students whose parents have waived services 2017 TELPAS

Rater Credentials Each teacher selected to rate an ELL must 1) have the student in class 2) be knowledgeable about the student’s ability to use English in instructional and informal settings 3) hold valid education credentials such as a teacher certificate or permit 4) be appropriately trained, as required by TEA 5) rate the student in all eligible language domains 2017 TELPAS

Writing Samples

Writing Sample Overview • • Feb. 13 th – First day you can collect writing samples. Collect at least 5 total samples for the writing collection. All assignments must include student’s name and date Copies of students writing assignments are allowed if the copied pages are clear, legible, and include no teacher edits • Samples may be typed if spell check and grammar check are disabled • Collections should contain samples from a student’s “comfort zone” - English they know and feel comfortable using AND samples which stretch students beyond the comfort zone 2017 TELPAS 27

Writing Sample Requirements • Writing Collections must include: – at least 1 narrative about a past event – at least 2 academic writing samples from math, science, or social studies • Suggestions for other types of writing assignments (see TELPAS Rater Manual, pp. 16 -20): – Basic descriptive writing on a personal/familiar topic – Writing about a familiar process – Personal narratives and reflective pieces – Expository and other extended writing from language arts classes 2017 TELPAS 28

Writing Samples NOT Permitted • Papers containing copied language • Papers in which student relies heavily on resources (dictionary, thesaurus, etc. ) • Papers showing teacher comments and corrections • Worksheets and question-answer assignments • Papers that are polished with help from peers or teachers • Papers written primarily in student’s native language • Papers that are brief, incomplete, or rushed 2017 TELPAS 29

Important • IMPORTANT: Raters proofread each sample to ensure it meets the criteria - not the Verifiers confirm that each sample is eligible to be included in the writing collection. • The Verifier must provide a Writing Collection Cover Sheet and a Verification Checklist for each collection of writing samples (Rater Manual Appendix C) • The writing collection Verifier(s) must complete the Verification Checklist to make sure each sample meets criteria. • Collections that do not meet the requirements will be returned to the Rater. • Once the collection meets all requirements on the Verification Checklist, the Verifier initials and signs the Verification Checklist. • After the collection is verified, the collection is given back to the Rater for Rating. 2017 TELPAS 30

Date to Collect Samples • On this slide you can list the dates that the teachers are to collect the samples from the students or provide them with the calendars I gave you. Set your dates and be firm with them. • TIPS you may include: – Have ALL students write, not just the LEP students – Use colored paper to identify writing types: yellow(past event), blue(math), green (science) – Don’t announce this is TELPAS writing, just have students write like they do in class 2017 TELPAS 31

Holistic Rating Training Basics

Training 2017 TELPAS

New and Returning Rater Definitions for K– 1 and 2– 12 • CTCs must obtain the Basic Training Certificate from teachers and place a copy in their CTC binder. If the teacher can not produce a copy, the teacher must take the Basic Training Course. • Returning raters that have not completed calibration activities within the last three school years are required to complete the online basic training course 2017 TELPAS before calibrating.

Key Dates 2017 TELPAS

Two Types of Training • Online basic training course This course is for new raters and raters that have not completed calibration in the last three school years. It provides instruction on using the PLD rubrics and gives raters practice rating students in each language domain. There are separate courses for K– 1 and 2– 12. • Online calibration This is for all raters. Raters use the PLDs to rate students in each language domain. Raters have two opportunities to calibrate on assigned grade cluster. 2017 TELPAS

Assembling & Verifying Training • Assembling and Verifying Grades 2– 12 Writing Collections o Required course for Verifiers. This is used to supplement Test Administrator procedures training o Optional course for Raters and Coordinators 2017 TELPAS

Training Courses Separated by Grade Clusters • Coordinators must be certain that Raters know which assigned grade cluster(s) they must complete before starting the online basic training Grade Clusters Grades K– 1 Grade 2 Grades 3 -5 Grades 6– 8 Grades 9– 12 2017 TELPAS

Raters with Students in Multiple Grade Clusters within Grades 2– 12 – Raters are required to train and calibrate in the cluster in which they have the most ELLs. – As a best practice, these raters should also review online basic training course practice activities in the additional cluster(s). Example: A grade 2– 5 ESL teacher has most of her ELLs in grade 3. She must complete training course and calibration in grades 3– 5. As a best practice, she should review the online practice activities for grade 2 to check her readiness to apply the rubrics appropriately. 2017 TELPAS

Raters with Students in Multiple Grade Clusters that Include Grades K– 1 – Because of differences in the instructional content and rating rubrics, these raters must complete training and calibration for K– 1 and at least one other cluster in 2– 12. – Raters with more than one additional cluster should train in the cluster in which they have the most ELLs. – As a best practice, these raters should also review online basic training course practice activities in the additional cluster(s). Example: A grade 1– 3 ESL teacher has most of his ELLs in grade 2. He must complete training course and calibration in grades K– 1 and 2. As a best practice, he should review the online practice activities for grades 3– 5 to check his readiness to apply the rubrics appropriately for his 3 rd grade students. 2017 TELPAS

When and where do raters take the online basic training course? • Depending on campus arrangements, raters may either complete the online basic training course during school hours, after school, or on weekends. • The online training system allows raters to access the training from any computer that meets the minimum system requirements. It is a violation of state assessment procedures to record, discuss, or share answers to the rating practice and calibration activities. Prior to creating a TELPAS Online Training Center account, raters are required to read an online statement and affirm that they will complete the rating activities independently. 2017 TELPAS

When and where do raters complete calibration? • Both new and returning raters must complete calibration in a monitored session that has been established by the campus. • Our Monitored Sessions will be: – 1 st session: __________ (all raters) – 2 nd session: __________ (for those that do not pass the 1 st time) It is a violation of state assessment procedures to record, discuss, or share answers to the rating practice and calibration activities. Prior to creating a TELPAS Online Training Center account, raters are required to read an online statement and affirm that they will complete the rating activities independently. 2017 TELPAS

IMPORTANT • All raters both NEW and RETURNING, must create an account in the New TELPAS Online Training Center. • Each year the training center closes. NOTHING is saved. Raters MUST retain a copy of their Basic Training Certificates. 2017 TELPAS

TELPAS Online Training Center https: //tx. scoring. pearsonassessments. com/scoring / 2017 TELPAS

TELPAS Online Training Center https: //tx. scoring. pearsonassessments. com/scoring/# training 2017 TELPAS

2017 TELPAS

2017 TELPAS

Holistic Rating Calibration Activities

Preparing for Calibration Sets • New raters (and those who have not completed calibration in the last three school years) must complete the online basic training course before attending a calibration session. • Raters that can not produce a Basic Training Certificate must complete the online basic training course. • All raters have the option to review the online basic training course, which includes practice rating activities, before attending a monitored calibration session. 2017 TELPAS

Monitored Calibration Sessions • All raters must receive administration procedures training prior to attending a calibration session and must have a signed oath on file. A roster of participants in each session will be maintained. • Raters will be given a passcode by the session proctor in order to access calibration. • Raters may use their TELPAS Rater Manual or a copy of the PLDs and scratch paper to make notes during calibration. All notes must be turned in to the proctor and destroyed at the end of the session. • Raters will need to complete calibration activities individually within the monitored group session. Therefore, the use of headphones is required. 2017 TELPAS

Monitored Calibration Sessions • Raters must begin with calibration set 1. If successful, a certificate will be available in the scoring summary tab of the TELPAS Online Training Center. The certificate should be printed and turned in to the proctor. • If raters are unsuccessful on set 1, they will be notified by testing coordinator when monitored session for set 2 will be held. • After completing a calibration set, raters immediately see results and annotations explaining the correct ratings. • Raters will need 1 -2 hours to complete a calibration set. Raters must finish a calibration set once it is started. They will not be able to save their progress and return later to complete the set. 2017 TELPAS

Will raters be able to refer to any resources during calibration activities? Yes, raters should use their rating rubrics (PLDs), TELPAS Rater Manual, notes, as needed. Any notes taken during calibration should be done on the scratch paper provided and not in the TELPAS manual. At the end of each calibration session, all notes taken by raters must be turned into the proctor and destroyed. 2017 TELPAS

How many students must be rated successfully? • For K– 1, each set contains 10 students to be rated. You must rate at least 70 percent of the students correctly within a set to demonstrate sufficient calibration. • For 2– 12 grade clusters, each set contains 12 students to be rated. You must rate at least 67 percent of the students correctly within a set to demonstrate sufficient calibration. 2017 TELPAS

Recap of Calibration Process • • • Calibration is conducted in monitored sessions. A passcode is required to access calibration. There are 2 calibration sets. Raters who calibrate on set 1 are done. Raters who don’t calibrate on set 1 must PAS complete set 2. • Certificates must be printed and turned in to the proctor. 2017 TELPAS

What happens if a rater is not successful? – The teacher will still be a rater with a “support provider” assigned. The CTC in conjunction with the Bilingual Coordinator will select a trained rater to “oversee” and “ 2 nd rate” that individuals list of students. This support will ensure that the rater’s students are evaluated consistent with the rating rubrics. Individuals are not authorized to serve as raters unless they complete all required training components. 2017 TELPAS 22

Course and Calibration Certificates • After completing all modules in the 2 -12 online basic training course, your campus coordinator will be able to provide you with a certificate of completion. • For the K-1 & Verifier Course, the rater will print their own certificate after completion. 2017 TELPAS

Rater will print Calibration Certificate! • After the rater completes the monitored calibrations successfully, the rater will print their “own” calibration certificate. After successfully calibrating, Raters go to the “Scoring Summary” tab of the TELPAS online Training Center to access and print a certificate. The certificate must be provided to the session Proctor. Copies of calibration certificates must be kept by CTCs – return originals to Raters after copies are made. PDFs are also provided which can be emailed to CTCs. 2017 TELPAS

Campus Specific Information 2017 TELPAS 58

Please sign your Rater or Verifier Oath and Return it to me before you leave.