RI Next Generation Science Assessment RI NGSA Test

RI Next Generation Science Assessment (RI NGSA) . Test Coordinator Training January 29 -31, 2018

Presenters • RIDE - Phyllis Lynch – Director, State Assessments - Pamela Denton – Coordinator, State Assessments - Simone Palmer – Science and Technology Specialist - Heather Heineke – Assessment Specialist • AIR - Liz Mortimer – Program Manager - Cathy Cameron – Senior Program Manager - Matt Steele – Project Assistant 2

Presentation Overview § Overview of Test Design § AIR Systems § Accommodations and Accessibility Tools 3

Design Challenge for State Science Assessment § Criteria • Aligned to three-dimensional standards • Reflective of classroom instruction • Equitable for all students • Informative for instruction • Valid and reliable § Constraints • Time for development, administration, reporting • Cost of development, administration, scoring, reporting • Logistics (scheduling, administration, technology)

Different Standards= Different Design RI Grade Span Expectation NGSS Performance Expectation PS 3 (5 -8) INQ+ POC – 8 Use data to determine or predict the overall (net effect of multiple forces (e. g. , friction, gravitational, magnetic) on the position, speed, and direction of motion of objects Students demonstrate an understanding of force (e. g. , friction, gravitational, magnetic) by… 8 d making and testing predictions on how unbalanced forces acting on objects change speed or direction of motion, or both. 8 e describing or graphically representing that the acceleration of an object is proportional to the force on the object and inversely proportional to the object’s mass. 5

Developing Assessments for the NGSS (BOTA Report): CONCLUSION 2 -1 • Measuring the three-dimensional science learning called for in the framework and the Next Generation Science Standards requires assessment tasks that examine students’ performance of scientific and engineering practices in the context of crosscutting concepts and disciplinary core ideas. • To adequately cover the three dimensions, assessment tasks will generally need to contain multiple components (e. g. , a set of interrelated questions). • It may be useful to focus on individual practices, core ideas, or crosscutting concepts in the various components of an assessment task, but, together, the components need to support inferences about students’ threedimensional science learning as described in a given performance expectation. *Pellegrino, J. (n. d. ). Developing assessments for the next generation science

RI Educator Feedback on Science Test Design District leaders and teacher feedback for design of new assessment showed that majority of educators preferred: § § § Testing in grades 5, 8, 11 Shorter testing time -two 90 -minute sessions Administration during May/June Computer-based; some preferred computer and paper options Shorter turnaround time for results Combination of response types • 83% of both teachers and district leaders supporting inclusion of performance tasks • 67% of teachers/83% of district leaders preferring technologyenhanced items

NECAP Inquiry Tasks vs. RI NGSA Tasks § Similarities • Performance-based • Phenomenon- or Scenariobased • Assess both knowledge and skills • Multiple components • Require student-generated responses § Differences: • Computer-based • Shorter • No multiple choice except in context of clusters/standalone items • May or may not have constructed responses

RI Next Generation Science Assessment: Test Design § Standards: NGSS Performance Expectations (Grades 3 -5, MS, HS) § Content: Block matrix—students receive different sets of clusters/standalone items with coverage of all PEs across grade band (~7 clusters, 12 standalones per student) § Item Types: • Clusters—phenomenon/scenario-based stimulus with a set of interrelated items divided into parts (4 -6) requiring multiple interactions (8 -10) • Standalone—stimulus with limited number of items and interactions • Types of interactions (drag-and-drop, graphing, virtual simulations, etc. ) • Multiple choice/select only within cluster of items • Short constructed response on field test in future TBD 9

RI NGSA Practice Items AIR Science Practice Test Items Elementary, Middle and High School https: //airscience. tds. airast. org/student 10

RI Next Generation Science Assessment: Administration Timeline and Duration § Assessment Administration Timeline: • Spring 2018 Census Field Test (no results reported) • Spring 2019 Operational Assessment (score reporting TBD) § Test Window: May 14 – June 1, 2018 § Grades: 5, 8, and 11 § Mode of Administration: Computer-based • Paper-based available as an accommodation § Duration: Two 60 -minute sessions (testing time only) • Suggested time limit but students can remain working as long as they are doing so in a productive manner. 11

AIR Systems Overview Copyright © 2014 American Institutes for Research. All rights reserved.

Overview • • • Portal Secure Browser Test Information Distribution Engine (TIDE) Practice Test TA Certification Course Test Delivery System (TDS) 13

Portal Copyright © 2014 American Institutes for Research. All rights reserved.

RI Next Generation Science Assessment Portal § Where you will access all things Science • • Secure Browser Practice Test AIR Systems Resources 15

Portal – http: //ri. portal. airast. org 16

Resources 17

Register for Email Updates 18

Secure Browser Copyright © 2014 American Institutes for Research. All rights reserved.

What is the Secure Browser? The secure browser is designed to ensure test security by prohibiting students from accessing any other programs or websites during testing. 20

Secure Browser § Must be used to take the NGSS assessment § Can be used for the practice test to make sure it is installed properly § A new desktop version is released once a year § i. Pad and Chromebook app versions are updated as necessary, normally once a year 21

Secure Browser Download 22

Test Information Distribution Engine (TIDE) Copyright © 2014 American Institutes for Research. All rights reserved.

How RI Data Enters TIDE § Student Data • Nightly student upload from e. RIDE – To change student information, update information in the Student Information System – The student information will update in TIDE overnight • Accommodations and designated supports updated in TIDE § User Data • RIDE will upload district test coordinators and data managers based on contact information in e. RIDE district profiles • District users will upload school test coordinators • School test coordinators will upload school users 24

User Roles § TIDE is used to provision all user roles for AIR systems Task District Administrator (DA) School Coordinator (SC) Teacher (TE) View Student Settings Edit Student Settings Add Rosters Add User Accounts Create Appeals Administer Assessments 25

Test Information Distribution Engine (TIDE) 26

Activating TIDE Account 27

Logging in to TIDE 28

Accessing AIR Systems 29

Overview of TIDE Tasks Preparing for Testing 30

Preparing for Testing: Add Users 31

Preparing for Testing: View/Edit/Export User 32

Preparing for Testing: Upload Users 33

Preparing for Testing: View/Edit/Export Students 34

Preparing for Testing: View/Edit/Export Students § Students are entered into TIDE through a nightly upload from e. RIDE. 35

Preparing for Testing: View/Edit Test Settings and Tools 36

Preparing for Testing: View/Edit Test Settings 37

Preparing for Testing: Upload Student Settings 38

Preparing for Testing: Uploading Student Settings 39

Preparing for Testing: Upload Student Settings Template 40

Preparing for Testing: Add Rosters 41

What are Rosters Used For? § Rosters are not required for testing purposes § Rosters are used for organizational purposes § Rosters are used to analyze students’ scores during reporting 42

Preparing for Testing: View/Edit Rosters 43

Preparing for Testing: Uploading Rosters 44

Overview of TIDE Tasks: Administering Tests 45

Administering Tests: Print Testing Tickets 46

Administering Tests: Create Appeals § District Administrators and School Coordinators can create appeals § Only the State has power to approve appeals 47

Administering Tests: Types of Appeals Appeal Type Description Reset a Test • • • Removes the test, and scores associated with the test, from the system Enables student to start a new test Requested due to accommodations not being set properly Reopen a Test • Allows for a test that has already been submitted or has expired to be reopened Grace Period Extension • • Extends the grace period if a test session is unexpectedly interrupted Gives access to all previous answers Restore a Test That Has Been Reset • • • Return a test from the Reset status to its prior status Only allowed on tests that have been reset A test can be restored if it was reset in error Invalidate a Test • • • Eliminates the test Student loses that test and does not receive a score Requested due to test security incidences 48

Administering Tests: View Appeals 49

Administering Tests: View Retrieved Appeals 50

Administering Tests: Monitoring Test Progress 51

Administering Tests: Plan and Manage Testing – Participation Reports 52

Administering Tests: Test Status Code Report 53

Administering Tests: Test Completion Rates 54

Overview of TIDE Tasks After Testing 55

After Testing: Non-Participation Codes § A non-participation event occurs when a student does not take a test as scheduled § You assign a code to explain the non-participation § Non-participation codes persist until they are changed § Are not reflected in student’s TIDE profile 56

After Testing: Non-Participation Codes 57

TA Certification Course Copyright © 2014 American Institutes for Research. All rights reserved.

TA Certification Course § All users must login to complete the course § Takes about 30 minutes to complete 59

TA Certification Course What is the TA Certification Course? • An online course that walks users through: – Navigating the TA Interface – Approving students to test – Pausing and stopping a test session – Logging into the test as a student • The TA Certification Course is one part of TA training. It supplements other training resources (user guides, the Test Administrator Manual) and necessary training on test administration, test security, and accommodations. 60

TA Certification Course This online course includes the following: § § § Contains step-by-step instructions for Test Administrators Contains interactive portion Contains quiz questions Multiple viewing opportunities Certificate of completion ALL TAs MUST TAKE THIS COURSE BEFORE PROCTORING A TEST! 61

Test Delivery System (TDS) Copyright © 2014 American Institutes for Research. All rights reserved.

Test Delivery System (TDS) Overview § Two interfaces: • Student • Test Administrator § Two sites • Operational • Practice Test 63

Practice Test Month 20 XX Copyright © 20 XX American Institutes for Research. All rights reserved.

Practice Test 65

Practice Test § Allows administrators, students, and guest users to become familiar with the online testing environment. § Provides a preview of question types that may appear on the operational assessment. § Provides students with the opportunity to test accessibility tools and assistive technology devices with the Test Delivery System (TDS). § Allows test taker to score their answers. 66

Student Interface Month 20 XX Copyright © 20 XX American Institutes for Research. All rights reserved.

Student Interface 68

Student Test Selection 69

Student Interface 70

End of Test 71

Test Administrator (TA) Interface Month 20 XX Copyright © 20 XX American Institutes for Research. All rights reserved.

TA Interface: Test Tree 73

TA Interface: Session ID 74

TA Interface: Approving Students 75

TA Interface: Approving Students 76

TA Interface: Save and Edit Details 77

TA Interface: Monitoring Student Progress 78

TA Interface: Pausing and Stopping Sessions 79

Test Accessibility Copyright © 2014 American Institutes for Research. All rights reserved.

Paper Test Booklets § 3 forms of Paper tests will be available § UEB Braille § Large Print § Standard § Paper tests will be ordered directly through AIR § Student response will be entered in the Data Entry Interface (DEI) § More information regarding paper assessments will be available in the coming months 81

Data Entry Interface • For students who take the paper Braille and Large Print Smarter Balanced and NGSS Field Test, the teacher must enter the student responses into the DEI. • The DEI does not require the secure browser. Users can access the DEI using most internet browsers (Chrome, Firefox, Internet Explorer, etc. ) 82

Accessibility Tool Designations § Accessibility Features § Designated Supports § Accommodations 83

Context Menus allow students to: § Mark items for review § Make notes in Student Comments § View item tutorials § Send print requests to the TA (if available) § Access additional features depending on test settings and item types 84

Accessibility Features RI NGSA Accessibility Features and Accommodations Manual www. ride. ri. gov/NGSA Copyright © 2014 American Institutes for Research. All rights reserved.

What are Accessibility Features? § Accessibility Features: Many supports are now available to all students, either on the computer-based tests or on the paper-based equivalent. Accessibility Features may be used by any student. 86

List of Accessibility Features for All Students § 100 s Number Table § Multiplication Table § Abacus § Calculator § Color Contrast § Expandable Passages § Highlight § Mark for review § Line Reader § Student Comments § Strikethrough § Masking § Periodic Table § Streamline Mode § Zoom 87

Accessibility Feature: Strikethrough § Allows students to cross out answer options 88

Accessibility Feature: Highlighter § Marks desired text, item questions, item answers, or parts of these with a color 89

Accessibility Feature: Mark for Review § Allows students to flag items for future review 90

Accessibility Feature: Zoom Button § Changes the size of test and graphics on the screen 91

Accessibility Feature: Line Reader § Assists students in reading by raising and lowering the tool for each line of the text on the screen 92

Accessibility Feature: Masking § Blocks off content that is not of immediate need or that may be distracting 93

Accessibility Feature: Color Contrast § Black on Rose § Black on White (default setting) § Medium Gray on Light Gray § Reverse Contrast § Yellow on Blue 94

Accessibility Feature: Streamline Mode § Rearranges how the item is presented to the student § Item stimuli are displayed above the item stem and response options 95

Designated Supports Copyright © 2014 American Institutes for Research. All rights reserved.

What are Designated Supports? § Designated Supports: Flexible test administration procedures may be used with any student, at the discretion of the principal (or designee). These include changes in the location of testing rooms, group size, seating of students, and scheduling of test administrations. 97

List of Designated Supports § Small group § Individual, one-on-one § Frequent, supervised breaks § Separate location § Seating § Adaptive furniture § Noise buffers § Familiar test administrator § Student reads aloud to self § Specific time of day § Stop Testing Policy § Text-to-Speech or Human Read Aloud § Read Aloud Selected Words 98

Read Aloud Options Entire Test: • Embedded in TDS and • human read aloud Selected Words: • Test administrator reads aloud, or signs, selected words on the science tests, as requested by the student. The student will point to the word or phrase that he or she needs read aloud or signed. Test administrator quietly reads aloud or signs the selected word to the student. Students using this feature may be tested alongside other students in groups of any size. 99

Considerations when making decisions about Read Alouds § Is the student a struggling reader? § Is the student learning Braille? § Test questions are designed to be read independently § Read Alouds do not take the place of rich, rigorous science instruction aligned to the NGSS. § Read Aloud Selected Words may be more helpful to students. 100

Designated Support: Text-to. Speech § Text is read aloud to the student via embedded Text-to. Speech technology 101

Designated Support: Mouse Pointers § Allows the size and color of the mouse pointer to be set in TIDE § Found in Zoom Tool 102

Accommodations Copyright © 2014 American Institutes for Research. All rights reserved.

What are Accommodations? § Accommodations differ from accessibility features or designated supports. Accommodations are specifically designed for students with disabilities and are changes in procedures or materials that increase the student’s ability to access the assessment. Accommodations and are available to only those students with document of the need through a formal plan or students who are English learners. 104

List of Accommodations § Amplification § Simplified test directions § AT/AAC § ASL § Braille (UEB) § Magnification § Print Size § Paper Edition of Test in English § Spanish § Word Prediction § Scribe § Speech-to-Text § Bilingual Dictionary and/or Glossary 105

Accommodation: Print on Demand § Paper copies of stimuli and/or items are printed for students § Must be set in by District Administrator (DA) 106

Accommodation: Print Size § Allows the font size viewed by the student in TDS to be set in TIDE for the entire test 107

Accommodation: Speech-to. Text § Speech-to-Text software such as Dragon Naturally Speaking, will work with AIR’s Test Delivery System (TDS). § Software can be used on the same computer the student is using for testing. § Student will need Permissive Mode set in TIDE § NOTE: Google Speech-to-Text will not work with the TDS. 108

Accommodation: Permissive Mode § Permissive Mode allows assistive technology software and devices to work in conjunction with the AIR Secure Browser § Must be set in TIDE 109

Training Resources § TA Certification Course § User Guides and Manuals • • TIDE User Guide Test Administration Manual (TAM) Test Coordinator Manual (TCM) TA User Guide § Upcoming Webinars § Practice Test 110

Proposed Webinar Subjects § Test Administration Policies and Procedures § Test Administrator and Student Interface § Accommodations § Open Office Hours 111

Contact Us RI Next Generation Science Assessment Help Desk 1. 866. 757. 9437 rihelpdesk@air. org RI Department of Education Simone Palmer, Science and Technology Specialist simone. palmer@ride. ri. gov Pam Denton, Assessment Coordinator pamela. denton@ride. ri. gov Heather Heineke, Assessment Specialist heather. heineke@ride. ri. gov 112