Michigan Science MSTEP Opportunity to Learn for Teachers

Michigan Science M-STEP

Opportunity to Learn for Teachers Opportunity to Learn for Students “The idea of an assessment system begins with a commonsense point: no one assessment – or assessment occasion – can meet all the needs for information about what students know and can do in science” (p. 21 NASEM, 2017).

PE Bundle – Topic Bundles • Bundles should mirror how PEs are presented and taught in classrooms. • Bundled PEs should reflect the structure of the adopted standards. • Bundles should enable assessment via a single natural phenomenon presented within a stimulus.

PE Bundles High School Topic Bundle Domain Structure and Function Life Science Matter and Energy in Organisms and Ecosystems Life Science No. PEs 3 6 Interdependent Relationships in Ecosystems Natural Selection and Evolution Inheritance and Variation of Traits Space Systems History of Earth’s Systems Weather and Climate Human Sustainability 6 5 4 4 3 5 2 5 Life Science Earth Science Earth Science

PE Bundles High School Continued Topic Bundle Structure and Properties of Matter Chemical Reactions Forces and Interactions Energy Waves and Radiation Engineering Design Total Topic Bundles Domain Physical Science Physical Science Engineering 16 No. PEs 4 5 5 4 71

Example Item Cluster Map

Released Item Clusters https: //wbte. drcedirect. com/MI/portals/mi/

How to access the released item clusters �Open Chrome and go to https: //wbte. drcedirect. com/MI/portals/mi/ Step 1 Step 2

How to access the released item clusters Step 4 Step 3

Item Cluster Annotations https: //www. michigan. gov/documents/mde/2018_MSTEP_Annotated_Sample_Items_test_614886_7. pdf

Data Insights

Content & Equity Data Review Committees Recommend to accept, reject, or revise Trained to examine data while reviewing clusters Accepted clusters move to operational. Rejected or revise clusters move back for revisions.

Insights from Data Review �Opportunity to learn is still an issue Grade 5 – in general Grade 8 – Physical Science (Waves) Grade 11 – Earth Science �Life science clusters had the best data �Timing of the high school test may cause problems And the lack of incentive for doing well

Sample Blueprint Form 1 Form 2 Form 3 Form 4 Earth Science 2 Earth Science 3 Earth Science 4 Earth Science 5 Life Science 1 Physical Science 2 Physical Science 3 Physical Science 4 Physical Science 5 Field Test Earth Science 1 Life Science 2 Life Science 3 Life Science 4 Life Science 5 Physical Science 1 Op items = 44 Op items = 42 Op items = 43 Op items = 44

Reporting 2020

Individual Student Reporting will include: • Overall Scaled Score • Scaled Sub Score – Physical Science, Life Science, Earth Science • Performance Level Descriptors Reporting will not include: • Item level information • Standard distribution Preliminary reports will not be available in 2020. Data will be available later in the summer

District/School Level Reports �In Development Will provide Topic Bundle information to assist with programmatic decisions regarding curriculum, assessments, staffing, etc. �Will likely be available in October.

MDE Resource for Reports https: //www. michigan. gov/mde/0, 4615, 7 -140 -22709_70117 ---, 00. html

Accountability

Accountability �Science test participation continues to be included in Parent Dashboard and School Index results on www. MISchool. Data. org as it has been throughout the years of science field testing. �Science test proficiency will resume being reported on MISchool. Data as part of the Parent Dashboard and under the student assessment reporting section. �For the School Index, the MDE will need to seek a waiver from the US Department of Education in order to include science proficiency in School Index calculations for accountability. mde-accountability@michigan. gov

Ongoing Work

Cognitive Complexity – Item Level

Cognitive Complexity – Item Cluster Level & d e k a N raid High guided f A integration Level Doing science r o v i v Rating • Student driven with limited to no scaffolding • The three dimensions are used together to engage in sensemaking • • Multidimensional with at least 2 being used (foregrounded) equally. Minimal scaffolding embedded in the task items. While not all, multiple items require a higher degree of sensemaking. Students use reasoning to discriminate meaningful (valid, accurate, causal, etc. ) from other information. r u S c i t s u R g n i p m a C g n i p m a Gl Low guided integration • Multidimensional but in general only one dimension is heavily foregrounded. • Moderate to high degree of scaffolding is embedded in the task items. • Items, overall, require a low degree of sensemaking. Scripted integration • Scripted: a provided, well-defined set of actions or procedures are used to complete a given task • Reasoning is heavily supported by the task • Focused on obtaining an answer, not sense making in an effort to understand/explain a phenomenon.

Plan for Michigan Cognitive Complexity Framework for Science Assessments Design Tools for Classroom Use Design Communication Guide Design Applied for Item Cluster Writers Shareholder’s Design Complexity Framework

Multi-Point Scoring

Multi-Point Scoring • Currently, all items on M-STEM are worth one point – even with multiple parts • If we apply Multi-Point Scoring Rules how will it affect • 3 -D Argument • Student Scores • Data back to MDE • Item Cluster Writing • Assessment development processes

Tools to Support MSS/NGSS Teaching and Learning

Task Annotation Project in Science

Task Annotation Project in Science

Michigan Math & Science Leadership Network Professional Learning Opportunities! https: //docs. google. com/docum ent/d/1 v. ZVV_Ds. Cwf. OJ 00 KGk 1_L 1 kgx. AO Oo. EFHKyd 70 y. Ze. Q 8 U/edit

Short Course: tinyurl. com/3 Dassessmentdevelopment

What Can You Do?

Ways to Shift Thinking Around Assessment in Science 1. Find opportunities to learn about assessment literacy 2. Challenge yourself & your colleagues to change just 5% of your assessments this year to be more NGSS-like. 3. Take the sample item clusters like a student would. Understand their experience and what might cause them to trip up. How is this testing experience different from when you were in school? How might we need to address test taking strategies differently than before? What affordances does this test design offer your students?

Questions from the Field What kind of Test-Prep should I provide for my students to get them ready for the science MSTEP? �Provide students continual opportunities to learn science in phenomena-rich environments where sense-making using the three-dimensions is the primary goal. �Encourage students to pay attention to data, infographics, and text when taking the assessment.

Opportunities for State-Lead Work Committee Date Develop/Review PLDs Spring 2020 Content Review Tentative May 18 -20 Bias/Sensitivity Review Tentative May 21 -22 Item Cluster Development 1 July 13 -17 Item Cluster Development 2 July 27 -31 Content Data Review Aug 10 -12 Bias/Sensitivity Review Aug 13 -14

Thank you! Let me know how I can help you! Tamara (TJ) Smolek Science Education Research Consultant Michigan State Science Supervisor Office of Educational Assessment and Accountability smolekt@michigan. gov Cell: 517 -706 -9130