Building a Comprehensive Assessment System Edward Roeber Office

Building a Comprehensive Assessment System Edward Roeber Office of Educational Assessment and Accountability 11/4/2020 1

This is the time of assessment…. This is the era of standards-based reform, based on student assessment information Federal laws require much assessment at the state level District response … more assessment! Using assessment information is now “high stakes” - school survival now depends on it Yet, educators are not always prepared to develop assessments nor use the information provided by them 11/4/2020 2

Added Assessments New assessments will add even more external pressures Michigan Merit Examination End-of-course examinations With added state level assessment, as well as district-level assessments, our assessment systems may be out of balance - too much external testing, too little classroom-based assessment 11/4/2020 3

What’s Needed We need a balanced assessment system - one that honors the work in classroom assessment and is aligned to the state and district assessments A variety of approaches to assessment are needed - perhaps more assessment, but of different kinds All assessments needs to work in the same direction - a coordinated system of assessment 11/4/2020 4

What is a Balanced Assessment System? Formative Supports on-going learning in the classroom Interim Provides periodic snapshots of student achievement at the district level Summative Assesses student learning at the conclusion of a phase of learning All parts work together and are equally important 11/4/2020 5

Balanced Assessment Systems Are all elements of the assessment system coordinated? Are all three types present? Formative Interim Summative Are the content and skills measured compatible? Are standards, curriculum, instruction, learning, assessment and professional development aligned 11/4/2020 6

Balanced Assessment Systems Is assessment continuous - do the assessments measure student progress over time (days, weeks, months, school years, and school careers)? Are a variety of formal and less formal assessment methods used on an on-going basis? Is assessment integrated - does the assessment system fit in the larger educational system? Is assessment seamlessly integrated into instruction? 11/4/2020 7

Balanced Assessment Systems Are a variety of high quality assessments being used? Do all assessments meet professional standards (reliability, validity, and so forth)? Are the assessment purposes clearly spelled out and are assessments used that can achieve these purposes? Do all students participate in assessments appropriate for them? Do assessments have positive consequences - for students, educators, parents, and others? 11/4/2020 8

What are Summative Assessments? State Level Michigan Educational Assessment Program Michigan Merit Examination MI-Access English Language Proficiency Assessment Secondary Credit Assessments? District Level 11/4/2020 9

Uses of Summative Assessments District Prioritize schools for improvement assistance efforts Provide data for the local school board to understand student and school achievement Report academic progress of the district to parents and other taxpayers, including key community decisionmakers Increase community support for schools and for school improvement efforts Key Question: Did students learn what they should have? 11/4/2020 10

Uses of Summative Assessment State Determine priority schools for the interventions required by Federal (and state) law Guide state-level assistance to “high priority” schools - those continuing not to make adequate yearly progress Fund schools - could be general state aid and/or special programs to impact struggling schools Allocate human resources - such as providing staff or consultants to provide supportive management advice and strategies to schools 11/4/2020 11

What are Interim Assessments? Periodic assessments given school- or districtwide at fixed times during the school year Measure the school’s or district’s curricula in important content areas Provide evidence that all students have been taught key skills Serve to reinforce a common curriculum and sets of learning experiences 11/4/2020 12

Interim Assessments District-constructed assessments, or Commercially-available assessments Replicas of the MEAP assessments Pacing assessments that follow the curriculum Assessments that do not follow MEAP nor the district curriculum Key Question: Are students in each school on track for proficiency? 11/4/2020 13

Interim Assessments Examples Quarterly tests based on the instructional sequence(s) used to measure the pacing guides used instructionally Advantages May determine if students are learning the enabling skills Might catch students who are experiencing difficulties in learning before they fall far behind Challenges Instructional sequences are not always universal - are multiple forms of the assessments needed? Are the assessments of high quality and/or match instruction? 11/4/2020 14

Uses of Interim Assessment Increase instructional alignment - horizontal and vertical Inform parents of all students about the level of student achievement and improvement efforts under way Assure consistency of instruction across schools and the district Program evaluation - evaluate the effectiveness of new initiatives 11/4/2020 15

What is Formative Assessment? Classroom-based assessments used on an on-going basis in every classroom Carefully-thought-out strategies to engage students in learning in and outside of the classroom May encourage different ways of learning - moving from passive to active student learning May encourage student self-assessment/selfmonitoring 11/4/2020 16

Formative Assessment Teachers are engaged actively in the assessment process by: Sharing instructional targets with students Questioning students Observing students Examining and evaluating student work Providing feedback to students Determining instructional implications of the assessments Conferencing with students, other teachers and parents 11/4/2020 17

Formative Assessments Students are actively engaged in the assessment process by: Learning what is expected of them Taking responsibility for their own learning Actively participating in the learning process Participating in how students document what they have learned and how they learned it Demonstrating their learning to peers, educators and their parents 11/4/2020 18

Uses of Formative Assessment Guide student learning on a daily basis by providing information about what critical skills were and were not learned Provide extra learning opportunities to students who are struggling academically Provide additional learning opportunities for students who are doing well academically Report student progress to students, parents, and other educators Key Question: Has each student learned? 11/4/2020 19

Characteristics of Both Large. Scale and Classroom Assessments Shared model of student learning Should include learning progressions over time Shared concept of disciplinary knowledge and competence Focus assessment on the most valued knowledge and skills Focus on understanding and reasoning, not recall Assess enabling skills and procedural knowledge Signal to teachers and students what is most important Base assessment on clearly written standards 11/4/2020 20

Characteristics of Both Large. Scale and Classroom Assessments Measure a manageable body of knowledge and a limited number of the most important skills Target general forms of cognition problem solving and inductive reasoning skills that are more domain-specific Move from single-answer assessments towards greater use of tasks content based measure rich and well-structured knowledge are open to multiple approaches 11/4/2020 21

Characteristics of Both Large. Scale and Classroom Assessments Designed to be valid and useful to support decisions (large-scale and classroom) Are technically sound and timely Are designed in accordance with the purpose for which results will be used 11/4/2020 22

Characteristics of Both Large. Scale and Classroom Assessments Measure the knowledge and skills they purport to measure Are designed to be reliable, valid, and fair for the inferences that will be drawn from results Report results in enough detail to reveal needed instructional changes and highlight deficiencies in the system or in instruction Focus on student learning in school, rather than what students have learned outside of school 11/4/2020 23

Characteristics of Both Large. Scale and Classroom Assessments Provide opportunities for students of different backgrounds to connect their knowledge to relevant school expectations. A range of measurement approaches are used to provide a variety and range of evidence of student achievement Provide multiple opportunities for students to demonstrate competence Look at differences in how students perform in different assessments as rich data about students, not “error” 11/4/2020 24

Unique Characteristics of Large. Scale Assessment Provide comparative data (normative or standardsbased) judge the adequacy of performance determine specific curricular/instructional areas for improvement Provide quality feedback to educators about patterns of errors that could be target for instruction in the future 11/4/2020 25

Unique Characteristics of Large. Scale Assessment Must be cost-effective and feasible instructional uses must be worth the time taken from instruction Results must be reported to stakeholders 11/4/2020 26

Unique Characteristics of Classroom Assessments Must be on-going and integrated seamlessly into instruction so teachers and students receive frequent feedback on their progress Assess some desired proficiencies that cannot be effectively assessed on large-scale assessments Provide quality ongoing feedback to teachers about patterns of errors that could indicate the need for modification of future instructional strategies 11/4/2020 27

Unique Characteristics of Classroom Assessments Help teachers to identify and reconstruct students’ misconceptions Provide quality feedback to students about their performance and specific guidance about how to improve Help students learn how to assess their current levels of understanding in relation to learning goals; learn to recognize and produce quality work 11/4/2020 28

What are ways we can assess students? Selected-response tests Forced-choice - multiple-choice, true-false, matching Written response - short-answer, extended response Observation Structured Unstructured Samples of Student Work 11/4/2020 29

What are ways we can assess students? Performance Events Individual interviews Hands-on performance assessments Performance Tasks Individual assessments Group performance assessments Projects Datafolios Portfolios 11/4/2020 30

Forced-Choice Items Examples multiple-choice, true-false, matching Advantages Can cover much content in little time Inexpensive Easy to score - “Objective” Challenges Difficult to write high-quality items May stress memorization over understanding and application Could encourage instruction that emphasizes content coverage over deep understanding of concepts 11/4/2020 31

Written Response Items Examples essays, short-answer, solve math problems, fill in the blank Advantages Tap student understanding Can measure application of knowledge Can be administered in group settings Challenges More time-consuming, expensive to score Scoring may not be objective Students may not respond 11/4/2020 32

Structured Observation Examples A topic for group discussion question by small groups of students Teamwork exercise with observation protocol Science laboratory exercise Advantages Assess application of skills Can observe students using skills in real contexts Challenges Difficult to observe multiple groups (or individuals) at the same time Time consuming; labor intensive 11/4/2020 33

Unstructured Observation Examples Student classroom participation Quality of student responses to teacher’s questions Advantages Naturalistic observation of students Collect data (systematically) not usually collected or used Challenges Lack of opportunity for all students to be observed Anecdotal in nature - how generalizable? 11/4/2020 34

Samples of Student Work Examples Daily classroom work Term papers Advantages Captures work already being done in the classroom Could reinforce the on-going creation of products by students Challenges The information from each student may be different May be difficult to “standardize” across multiple students 11/4/2020 35

Performance Events Examples - Musical performance sing, play a musical instrument Science experiment Solve mathematical problem Advantages Often, these assess the most important skills Challenges Time consuming to assess all student Expensive to develop, use, and score 11/4/2020 36

Performance Tasks Examples Paper on an English topic Compose a song Plan for a school or community improvement project Advantages Sustain and extend classroom learning Encourage students to work deeply in the content area Challenges Time consuming - for students and teachers Students need support and assistance 11/4/2020 37

Projects Examples Senior project on a topic of the student’s interest Implementation of a plan to improve the school or community Advantages Students can engage actual work that interests them Challenges Very time consuming for both students and teachers, to carry out, to present, and to score 11/4/2020 38

Datafolios Examples Collection of data from one or more sources, such as observation, ratings of student work, test scores in one folio Advantages Captures and organizes information about a student in one place Challenges Time consuming for students and/or teachers to select and organize the information Data itself may not tell much of a story; narrative commentary may also be needed 11/4/2020 39

Portfolios Examples Showcase of best arts products Collection of essays written in English class Advantages Show student progress Capture best work of the student Encourage students to produce work Challenges Very time consuming to develop, collect, organize, and score May be idiosyncratic to the individual students 11/4/2020 40

Questions to Consider First Do we know what students need to learn? Do we understand what students need to learn? How do students best learn? What instructional strategies are most effective? How do we know that students have learned? Can state, district, and classroom assessments work together to promote quality student learning? 11/4/2020 41

Questions to Consider First What work on assessment is needed to improve how assessment contributes to student learning? What professional learning needs to occur for this work to happen? How can the state encourage work on all three parts of the balanced assessment system? 11/4/2020 42

Choosing Assessment Methods Each educator and school should use a variety of assessment methods (multiple methods) The methods selected locally (district, school, and classroom) should complement and build on the assessments used at the state and national level Coordinated assessments are most useful, since they can present a more complete picture of student achievement and other accomplishments 11/4/2020 43

Coordinated Assessment Systems Use both large-scale and classroom-based assessments Measure the same skills or related skills using different measures of related skills One possible model: State or district: end-skills Classroom: learning progressions leading to the accomplishment of end skills 11/4/2020 44

Coordinated Assessment Systems Developed to provide “comprehensive” information on learning and achievement Different assessment methods may or may not yield consistent information If the results are consistent, still use multiple methods If not consistent, look for reasons - may yield insights into student behavior and/or the quality of the measures used. Avoid using multiple assessments as a multiple-choice item - selecting one source and ignoring the others 11/4/2020 45

Developing the Needed System Examine the academic content standards valued at the state level Format - end-skills or learning progressions? Breadth and depth - how thorough? Completeness - few or many in number? Determine how the state will assess these What types of measures will be used? What types of measures are missing? What other types of data do classroom teachers need? 11/4/2020 46

Developing the Needed System If the skills being measured are end-skills, then create the desired learning progressions in gradelevel expectations Work should involve educators from all levels to assure horizontal and vertical alignment Determine the assessments methods to be used to assess the learning progressions Determine assessment methods needed to gauge student learning Develop the needed assessments 11/4/2020 47

Developing the Needed System Create and implement instructional plans to provide the instruction on learning progressions Make sure all teachers are “on the same page” Horizontal alignment Vertical alignment Use assessments to determine student learning Embed assessments in on-going instruction Review and use assessment results to improve instruction Examine assessment results to determine individual student and group instructional needs 11/4/2020 48

Michigan’s Assessment System In Michigan, the assessment system needs to be more fully balanced Summative assessments play a major role Interim assessments are needed Formative assessments are also needed Teachers need to be given time to actively discuss instructional strategies Assessments need to be embedded in good instruction Teachers need to be formally prepared to work on the assessments to be used 11/4/2020 49

Secondary Credit Assessment Program State and districts need to determine whether students have learned enough to receive credit in a variety of high school credit areas This could be done in conventional end-of-course exams or through a different manner We propose to do both - to develop a balanced assessment system in Michigan We invite interested districts, schools, and educators to become partners in this effort 11/4/2020 50

An Opportunity to be Different The Secondary Credit Assessments provide an opportunity to develop something very different Re-balance the assessment system Develop assessments that encourage high levels of student learning, rather than simply determining pass-fail Build a system of interim and formative assessments to go with the summative assessments 11/4/2020 51

A Modest Proposal We propose to develop a series of formative (classroom), interim, and summative assessments for each of the key high school credit areas Develop these assessments using interested local educators across the state The “system” will present good local educator work made available to others in the state 11/4/2020 52

Areas to be Developed English Language Arts English 9, 10, 11, and 12 Mathematics Algebra I Geometry Algebra II Pre-Calculus Data and Statistics 11/4/2020 53

Areas to be Assessed Science Biology Chemistry Physics Earth Science Social Studies American History and Geography World History and Geography Economics Civics 11/4/2020 54

How Will Work be Carried Out? Develop ISD/RESA teams to sponsor work groups of local educators Leadership team from one or more ISDs/RESAs Working groups of teachers and curriculum specialists in the four content areas Professional development assistance provided by MDE and others Use other existing resources with an interest in improved instruction and assessment 11/4/2020 55

How Will Work be Carried Out? There are existing “learning communities, ” formal and informal, across the state already engaged in thinking about How students learn best How teachers can draw out the best from students How we know whether students have learned How educators can learn more about learning, teaching and assessment More such learning communities are needed in the state 11/4/2020 56

What Work Will be Done? Develop model instructional lessons that address parts of the high school content expectations Build a variety of assessments within these instructional ideas Try out these lessons and assessments with students MDE: Collect, select and share the ideas 11/4/2020 57

Target Provide instructional and assessment ideas across the state no later than Fall 2008 Build an assessment system for use locally Deliver the assessments on an on-going basis, so that students can be assessed before they take a class, as they are taking it, or at the conclusion of it 11/4/2020 58

Summary Our goal is to seek to improve how students are taught, as well as how they are assessed Emphasize improved student learning, not just higher test scores. Continue to seek to improve educator skills in instruction, as well as assessment Build a balanced assessment system with coordination among the parts 11/4/2020 59

Summary The requirement to assess the High School Content Expectations will be used as the basis for modeling a balanced assessment system that includes formative and interim assessments Engage educators across the state in developing these formative and interim assessments Begin this activity this year and support it as it grows and spreads Recruit local and ISD educators who want to be part of building this new system 11/4/2020 60

For Questions and Comments Dr. Edward Roeber Michigan Department of Education Office of Educational Assessment & Accountability P. O. Box 30008 Lansing, MI 48909 (517) 373 -0739 voice Roeber. E@michigan. gov 11/4/2020 61