Transition to New Alternate Assessments A Paradigm Shift

Transition to New Alternate Assessments: A Paradigm Shift for Assessing Students with Significant Cognitive Disabilities California Educational Research Association December 3− 5, 2014 Lily Roberts, Ph. D. , and Sheila Self, M. Ed. English Language Proficiency and Alternate Assessments Office CALIFORNIA DEPARTMENT OF EDUCATION Tom Torlakson, State Superintendent of Public Instruction

Overview TOM TORLAKSON State Superintendent of Public Instruction • • • Background Theory of Action Core Content Connectors Communicative Competence Paradigm Shift National Center and State Collaborative (NCSC) Pilot Administration Model • Phase I Pilot Results • Phase II Pilot Results • New Alternate Assessments 2

Background TOM TORLAKSON State Superintendent of Public Instruction • In July 2014, the State Board of Education provided the following guidance: – Discontinue the administration of the California Alternate Performance Assessment (CAPA) for English−language arts (ELA) and mathematics. – Administer CAPA science in grades five, eight, and ten until there is a successor. – Move forward with the Phase II Pilot to include California students in data analyses. – Seek a waiver from double testing. – Explore alternate assessment options. 3

Theory of Action TOM TORLAKSON State Superintendent of Public Instruction 4

Core Content Connectors • Core Content Connectors (CCCs) are: TOM TORLAKSON State Superintendent of Public Instruction – Connected to a pathway of knowledge and skills in a subject area – Linked to the Common Core State Standards (CCSS) – Less complex than the CCSS – Focused on the main academic content in each subject and grade, to deepen students understanding in later grades – Used to plan instruction and assessment for students who will take the alternate assessment 5

Core Content Connectors (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Students typically follow a pathway of knowledge as they move upward through the grade’s “learning progression. ” • Adjusted alternate achievement standards called “essential understandings” are content-based and are entry points for students beginning to interact with grade-level curriculum. 6

Core Content Connectors (cont. ) TOM TORLAKSON State Superintendent of Public Instruction Progress Indicator: E. RL. k identifying central ideas and key details to derive author’s purpose, message, or theme. Common Core State Standard Core Content Connectors: 4 Essential Understanding 4. RL 2 Determine a theme of a story, drama or poem from details in the text; summarize the text. 4. RL. k 2 Determine theme of a story, drama, or poem; refer to text to support answer. Determine the topic of a story or poem. 7

Communicative Competence TOM TORLAKSON State Superintendent of Public Instruction • The foundation of the NCSC model for achieving college and career readiness is communicative competence. – Communicative competence is the basis for learning and is necessary for active community participation. – Students must develop expressive communication skills through direct instruction in order to have an effective, individualized system of communication at home, at school, and in the community. 8

Communicative Competence (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Communication at some level is possible and identifiable for all students regardless of functional “level. ” – For students who have not yet developed communicative competence, this must be an immediate objective. – Communication needs must be addressed during the individualized education program (IEP) process to ensure routine use of the appropriate communication system during instruction and assessment. 9

Paradigm Shift TOM TORLAKSON State Superintendent of Public Instruction “Some people believe that students with significant cognitive disabilities cannot successfully learn academic content. Studies show that this is a misperception. The education system should start with the assumption that every child can learn. This is called ‘the least dangerous assumption’ because exposing students to learning is not harmful, but keeping them from it is. ” National Center and State Collaborative 10

Common Care State Standards and Alternate Assessments TOM TORLAKSON State Superintendent of Public Instruction • Alternate assessments must be linked to the Common Core State Standards. • The alternate assessment is derived from the CCSS, using links called core content connectors. • Not all CCSS are measured, only the essential elements or “big ideas. ” • The benefit of aligning the alternate assessment with CCSS is that all educators can benefit from CCSS professional development and can collaborate on “how to” teach the standards for all students. 11

NCSC Administration Model TOM TORLAKSON State Superintendent of Public Instruction • NCSC is a consortium of 26 states, including California, and five national organizations. NCSC’s purposes are to: – Build an alternate assessment based on alternate achievement standards aligned with the CCSS through CCCs. – Assess students with SCD in grades three through eight and grade eleven. – Create grade-level content for ELA and mathematics that has four tiers with increasing levels of complexity. – Use computer-based technology for test administration. 12

NCSC Administration Model (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Student eligibility requirements: – Student has significant cognitive disabilities – Student learning content is linked to CCSS – Student requires extensive direct individualized instruction and substantial supports to achieve measureable gains in grade and age-level curriculum. 13

NCSC Administration Model (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • The pilot administration’s features included: – One-to-one testing – Test forms for grades three through eight and grade eleven for ELA and mathematics – Scripted directions for administration – Whole passages re-read if the student requested or would benefit from repetition – Some questions that referenced specific passage parts to be read to the student – Items at all levels of difficulty • Hand over hand was not permitted 14

NCSC Administration Model (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • The Learner Characteristics Inventory (LCI) provided student-level data about vision, hearing, and classroom setting. • The Student Response Check (SRC) was used before testing to observe the student’s communication ability by demonstrating a consistent response to three content neutral test questions by using a mouse, verbalizing answer, gesturing or pointing, using an eye gaze chart, circling or marking answers on a 15 paper copy.

NCSC Administration Model (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • If the student did not display a consistent and observable response: – Administer the first four items of the test; if you observe one response with any of the first four items, administer the test. – Administer the first four items of the test; if you do not observe a response, terminate the test. • The test has the ability to pause and resume as many times as necessary during the test window. 16

NCSC Administration Model (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Accommodations included: – Assistive technology for presentation of items – Assistive technology for student responses to questions (e. g. , alternate keyboard, switches and hub, or head mouse) – Print version of items – Scribe to complete graphic organizers or response template for writing product – Sign language interpretation 17

NCSC Administration Model (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Item types included: – Selected response – Constructed response (mathematics) – Constructed response (writing) – Open response (reading foundational items in grades four and five) 18

Sample NCSC Item TOM TORLAKSON State Superintendent of Public Instruction Selected response sample ELA reading passage for grade four: “We are going to read a story about a boy named Ben. After we read, you will be asked a question about theme. Listen for the message of the story. ” 19

TOM TORLAKSON State Superintendent of Public Instruction 20

TOM TORLAKSON State Superintendent of Public Instruction 21

TOM TORLAKSON State Superintendent of Public Instruction 22

TOM TORLAKSON State Superintendent of Public Instruction 23

Remember, theme is the message of the story. What is theme of this story? TOM TORLAKSON State Superintendent of Public Instruction having a lot of friends showing you can take care of a pet asking your parents for something 24

Phase I Pilot Results TOM TORLAKSON State Superintendent of Public Instruction • The target sample size for Pilot I was 16, 000 for the consortium and 7, 700 for California. • The consortium had a total of 5, 161 students complete the test, and California had 1, 651 students complete the test (32 percent). • The participation rate was significantly lower for all states because of the test window being changed from February to April. (April 15 through May 24) – Overlapped with Smarter Balanced and CAPA testing 25

Phase I Pilot Results (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • A post-survey of 178 coordinators and administrators indicated that: – Training was effective (77 percent). – Sample questions were used prior to administration (81 percent). – Technology infrastructure was sufficient (81 percent), but technology issues were experienced during testing (79 percent). – Directions for administration were easy to follow (68 percent). – Students were engaged in testing (58 26 percent).

Pilot I Pilot Results (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • NCSC Technical Advisory Committee (TAC) recommendations: – Conduct various studies of Pilot I data (e. g. , sample representativeness, scrolling issues). – Further investigate tiers. – Maintain use of tiers in operational assessment. – Develop stopping rule for students who respond without intentionality. – Adjust flagging criteria for item data review. 27

Phase I Pilot Results (cont. ) • TAC recommendations (cont. ): TOM TORLAKSON State Superintendent of Public Instruction – Work with special education experts to evaluate length and difficulty of the Phase II Pilot. – Use linear form and consider form length for the Phase II Pilot. – Postpone stage-adaptive assessments until after the grant ends. – Explore tension between reliability and student/teacher interest/fatigue. 28

Phase II Pilot Results (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • NCSC and states conducted a rigorous sampling effort to ensure appropriate number and range of students for Pilot II. • Pilot II focused on scoring for foundational, multipart, writing, and passage items. • Results will be available next year. 29

Phase II Pilot Results TOM TORLAKSON State Superintendent of Public Instruction • Testing window was October 20 through November 21, 2014. • Approximately 1, 500 California students completed testing. • 21 test forms were assigned in either reading and writing or reading and mathematics across grades three through eight and grade eleven. • 25− 30 items per content area were administered. 30

Phase II Pilot Results (cont. ) TOM TORLAKSON State Superintendent of Public Instruction 31

New Alternate Assessment TOM TORLAKSON State Superintendent of Public Instruction • Spring 2015 alternate assessment field test for all eligible students in grades three through eight and grade eleven • New computer-based items for ELA and mathematics: – Items aligned with the CCSS through CCCs – Approximately 15 items per content area • The spring field test: April 15 through June 10, 2014 32

New Alternate Assessment (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • To assist with the development of the 2015 field test: – An educator focus group was convened to discuss test item characteristics. – Educators provided guidance on effective item types for students with SCD. – A two day item development training occurred to guide educators in writing items to content standards. 33

New Alternate Assessment (cont. ) TOM TORLAKSON State Superintendent of Public Instruction − The California Department of Education (CDE) examined other state item types and processes (e. g. , Oregon). − CDE staff will participate in the item review process in December and February. 34

New Alternate Assessment (cont. ) TOM TORLAKSON State Superintendent of Public Instruction • The development of the 2016 operational alternate assessment will take into consideration outcomes and results from: – NCSC Phase I and II Pilots and operational assessment – California spring 2015 field test outcomes – Any other available sources 35

Contact Information TOM TORLAKSON State Superintendent of Public Instruction English Language Proficiency and Alternate Assessments Office Phone: 916 -319 -0784 Alternate Assessments e-mail: alternateassessment@cde. ca. gov Alternate Assessments Web page at: http: //www. cde. ca. gov/ta/tg/ca/altassessment. asp 36