Preparing Tennessee for PARCC Preparing Educators Guiding Students

Preparing Tennessee for PARCC: Preparing Educators, Guiding Students, and Transitioning the TCAP Writing Assessment to the Common Core State Standards CCSSO National Conference on Student Assessment June 20, 2013

The transition to Common Core Standards is central to strengthening Tennessee’s competitiveness Source: “Projections of Jobs and Education Requirements Through 2018” (The Georgetown University Center on Education and the Workforce), 2011 NCES NAEP data, ACT 1

Our goal is to be the fastest improving state This will be measured by our results on: • NAEP • ACT • PARCC TN currently ranks 19 th among PARCC states (on NAEP. ) Our goal, during the first year of PARCC, is to come in 12 th. 2

We believe there are three legs to the stool in our implementation plan Effective communication about the standards, importance, and potential Assessment alignment and transparency Student achievement Quality training & meaningful support Instructional materials and curriculum Alignment of accountability structure for LEA’s, schools, teachers, and vendors 3

Assessment Transition Approach 2011 -2012 -2013 -2014 -2015 Expand no-stakes assessments reflecting PARCC Narrow official assessments 4

PARCC is coming… PB A 3 Parts ELA & Literacy • Research Simulation • Literary analysis • Narrative writing 2 Parts Math • Short and extended response questions • Focus on conceptual knowledge and skill and modeling and reasoning standards EOY 2 Parts ELA & Literacy • 4 -5 texts (literary and informational including social science, scientific, and technical texts at grades 6 -11) • Short-answer comprehension and vocabulary questions 2 Parts Math • short-answer focusing on conceptual knowledge, skills, and understandings 5

Writing Assessment Previous Writing Exams 5 th, 8 th, 11 th Open Ended Prompts Narrative, Expository, Persuasive Paper Based 2012 -2013 Writing 2013 -2014 Writing Still 5 th, 8 th and 11 th Text Based Prompts Same genre structure 8 th and 11 th Grade Online, 5 th Grade optional online Offered as district option to all grades 311 Text Based Prompts Could be any genre, any grade 6

3 directives 1. Move closer to the format and expectations of PARCC, to the best of our knowledge of it 2. Get examples into the field ASAP to inform instruction 3. Where possible, involve teacher scoring to help educators understand the expectations of the standards 7

Assessment Development Paradigm Shift • Old Development: – Instruction drives assessment • New Development – Assessment drives instruction • All Development – Development best practices 8

Assessment Development Two driving documents: • Common Core State Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects from the “Common Core State Standards Initiative: Preparing America’s Students for College and Career” • PARCC “Draft Model Content Frameworks for English Language Arts/Literacy” 9

Assessment Development: Process Identify appropriate stimulus material • Grade level • Readability • Bias • Complexity/richness • Focus on Informational Texts 10

Assessment Development: Process (cont’d) Stimulus material chosen, reviewed by TN Educators, and prompts developed by MI. Review of prompts/stimulus material by TN educators. Layout of test forms for pilot test. 2012 TCAP/WA Pilot Test administration, scoring, and analysis 11

Assessment Development: Process (cont’d) 2013 Operational Test prompt selection, one per grade 2013 Operational Test administration 12

Assessment Development: Lessons Learned • Passage selection – Copyright vs. public domain – Text appropriateness and complexity – Informational vs. Literary • Prompt development – KISS – CCSS alignment • Mode mixture – Narrative, Informational/Explanatory, and Opinion/Argument 13

Assessment Evolution: 2012 TCAP/WA Prompt 14

Assessment Evolution: 2013 TCAP/WA Text 15

Assessment Evolution: 2013 TCAP/WA Prompt 16

Rubric Development Inclusion of trait expectations and language present in the CCSS for Writing Review of traitbased rubrics used on other assessments Development of three modespecific rubrics 17

Rubric Evolution: 2012 TCAP/WA Rubric 18

Rubric Evolution: 2013 TCAP/WA Informational/Explanatory Rubric 19

Online Administration Logistics • Technology Readiness Surveys performed on all online testing schools months prior to test dates. Potential capacity issues communicated to online-testing schools. – 2012 Pilot Test & 2013 Operational Test – Bandwidth, hardware, software, networking, etc. • Online testing platform downloaded by approved testing schools one week prior to test dates • 8: 30 -5: 00 technical support available via e-mail and phone before, during, and after testing window 20

Online Administration Success • Perception: – Early predictions of negative outcomes for systems, schools, and students • Reality – All but one school was approved for online testing for the operational test. – Very few technical issues (loss of connectivity/student work, inability to use testing platform, etc. ) – Many reports of increased student engagement during testing 21

The 3 -12 ELA CCSS Pilot • Context: districts eager to jump the gun on CCSS for ELA before full statewide implementation in 2013 -14 • Purpose: – To convene districts interested in early implementation of CCSS for ELA and facilitate sharing of promising practices – To encourage districts to incorporate instructional shifts in advance of full standards implementation – Expose teachers and students to shifted assessments in a low-stakes manner – Learn from leading districts through conversations and data collection to inform upcoming TDOE training and content offerings 22

The 3 -12 ELA CCSS Pilot Four key elements: 1. Quarterly Grand Division consortium meetings 2. Teacher completion of professional learning modules 3. Administration and scoring of four practice writing prompts 4. Periodic feedback to the TDOE through surveys and scoring data 23

ELA Pilot Scoring Material Development 2012 ELA Pilot form selection and administration. Rangefinding with TDOE and TN stakeholders. Training material and video development. Scoring email helpline implementation. 24

ELA Pilot Scoring Material Development (cont’d) Lessons Learned • Video training differs from in-person training – Full-blown scripts are essential in video training creation – Scoring staff creating scripts must be comfortable enough with technology to communicate with technical staff and ensure content and form work together as expected – KISS: viewers cannot ask questions during training • Video creation, editing, and production takes a long time • Teachers want anchor papers and scoring resources early 25

Evaluating the 3 -12 ELA Pilot • Surveys given at all four consortium meetings • Two teacher surveys and one principal survey • Data used to determine effectiveness of pilot and inform content of summer training 26

ELA pilot district leadership team survey results from final meeting % A (6 s) The practice scoring exercise at the beginning of this meeting increased my understanding of the expectations of the rubric. The anchor papers for each prompt and score point are helpful in establishing a common understanding of the rubric expectations. My school and/or district’s participation in the ELA Pilot allowed our teachers to better prepare students for the new format of the TCAP Writing Assessment given in February. % SA (7 s) % A/SA (6 s and 7 s) 54. 3 31 85. 3 50 34. 5 84. 5 39. 1 44. 4 83. 5 27

ELA Pilot teacher survey results from April 2013 • Of the scoring resources we provided, teachers found the scoring guides (with anchor papers) to be the most helpful, followed by scoring practice training sets and, last, videos explaining the scoring guides 28

2013 TCAP Writing Assessment results Average Scores Per Grade and Trait Grade 5 8 11 F/O 2. 44 2. 46 2. 74 S/E 2. 46 2. 09 2. 02 L/S 2. 63 2. 57 2. 88 C 2. 59 2. 69 2. 96 29

2013 trait progressions by grade 4 3. 5 3 C L/S F/O S/E 2. 5 2 1. 5 1 0. 5 0 5 th 8 th 11 th 30

2013 TCAP Writing Assessment results Key observations: • Only 1. 4% of 8 th Graders and 0. 9% of 11 th Graders received a score of 4 in Support/Elaboration. • Significant increase in condition code scores between 2012 and 2013: Grade 2012* 2013** 5 84 2, 683 8 30 4, 660 11 33 219 TOTAL 147 7, 562 *2012: 0 B = Insufficient to score or illegible **2013: 0 D = Too limited to evaluate 31

Writing Assessment Previous Writing Exams 5 th, 8 th, 11 th Open Ended Prompts Narrative, Expository, Persuasive Paper Based 2012 -2013 Writing 2013 -2014 Writing Still 5 th, 8 th and 11 th Text Based Prompts Same genre structure 8 th and 11 th Grade Online, 5 th Grade optional online Offered as district option to all grades 311 Text Based Prompts Could be any genre, any grade 32

Next steps • Available to all grades 3 -11, optional beyond 5, 8, 11 • Online only • 3 passages – more closely reflect RST • For more information, visit www. tncore. org 33

Questions 34

Thank you! Emily Barton Assistant Commissioner of Curriculum and Instruction Tennessee Department of Education lior. klirs@tn. gov Lior Klirs Coordinator of English Language Arts Content and Resources Tennessee Department of Education lior. klirs@tn. gov Katie Gerhardt Program Manager: TCAP Writing and Constructed Response Assessments Measurement Incorporated kgerhardt@measinc. com