Professional Development to Practice Common Formative Assessment Sound

Professional Development to Practice Common Formative Assessment Sound Assessment Design The contents of this presentation were developed under a grant from the US Department of Education to the Missouri Department of Elementary and Secondary Education (#H 323 A 120018). However, these contents do not necessarily represent the policy of the US Department of Education, and you

Professional Development to Practice Welcome and Introductions Please take a moment to introduce (or reintroduce) yourself to the group, by telling your name, district, and position. Our trainers for the day are….

Professional Development to Practice

Professional Development to Practice Common Formative Assessment (CFA) Overview and Purpose of CFA Quality Assessment Design Developing Meaningful Learning Targets Quality Assessment Design

Professional Development to Practice Learner Objectives of the Formative Assessment Series q Understand the clear purposes of assessment by clarifying q. Why they are assessing q. Who will use the results of assessment data q. What they will do with the assessment data q Develop clear and meaningful learning targets to guide instruction and student learning. q Construct quality assessment instruments which are of sound design and measure pre-determined learning targets.

Professional Development to Practice Outcomes for the Day As a result of todays training you will… q develop a better understanding of the components and characteristics of a quality formative assessment. q develop a better understanding of various types of assessment items and the pros and cons of each type. q continue using a backwards design approach and a template form to write a formative assessment. q evaluate your formative assessment for quality.

Professional Development to Practice Module 3: Quality Assessment Design Essential Questions: 1. What decisions drive the type of assessment items to use in common formative assessments? 2. What are the essential components needed to create a quality formative assessment? 3. What are the characteristics of quality selected-response, constructed-response and performance tasks?

Professional Development to Practice Session at a Glance q Introductions/Objectives/Outcomes/ Norms q Brief Review of Assessment Principles and Reflection q Selected Response Items q Constructed Response Items q Performance Tasks q Continue with Assessment Development Process q Evaluate Your Test q Using Data to Inform Test Writing Skills q Implementation Steps, Roadblocks, and Supports q Additional Learning q Closure

Professional Development to Practice Norms q. Begin and end on time q. Be an engaged participant q. Be an active listener – open to new ideas q. Use notes for side bar conversations q. Use electronics respectfully

Professional Development to Practice Accurate Assessment ASSESS WHAT? What are the learning targets? Are they clear? Are they good? WHY ASSESS? What’s the purpose? Who will use the results? ASSESS HOW? What method? Written well? Sampled how? Avoid bias how? Be sure students understand targets too! Students are users too! Students track progress and communication, ! COMMUNICATE HOW? How is information managed? reported? Students can participate in the process too! Effectively Used Source: Adapted from Classroom Assessment for Student Learning: Doing it Right-Using it Well. , by R. J. Stiggins, J. Arter, J. Chappuis, & S. Chappuis, 2004, Portland, OR.

Professional Development to Practice Accurate Assessment ASSESS HOW? What method? Written well? Sampled how? Avoid bias how? Source: Adapted from Classroom Assessment for Student Learning: Doing it Right-Using it Well. , by R. J. Stiggins, J. Arter, J. Chappuis, & S. Chappuis, 2004, Portland, OR.

Professional Development to Practice Read and Reflect with a shoulder partner… Every educator must understand the principles of sound assessment and must be able to apply those principles as a matter of routine in doing their work. Accurate assessment is not possible unless and until educators are given the opportunity to become assessment literate. (They) must understand student achievement expectations and how to transform those expectations into accurate assessment exercises and scoring procedures. (NEA, 2003) Common Formative Assessments, Larry Ainsworth & Donald Viegut, 2006, Corwin Press, pg 53

Professional Development to Practice What is Assessment Literacy? “The ability to understand the different purposes and types of assessment in order to select the most appropriate type of assessment to meet a specific purpose. ” (Larry Ainsworth, 2006)

Professional Development to Practice What evidence do we need that students have met our stated purpose(s)? “Fruitful assessment often poses the question ‘what is an array of ways I can offer students to demonstrate their understanding and skills? ’ In this way, assessment becomes a part of teaching for success and a way to extend rather than merely measure learning. ” Quote by Carol Ann Tomlinson, 1995, taken from Common Formative Assessment by Ainsworth and Viegut

Professional Development to Practice Video Clip http: //www. youtube. com/watch? v=_Cqgn. Zhb --Q

Professional Development to Practice Assessment of Missouri Learning Standards q The knowledge, skills and processes specified in Missouri’s Learning Standards (MLS) for Mathematics and English and Language Arts will be measured by Smarter Balanced Assessment Consortium (SBAC) using a variety of test item types…. . selected response, constructed response and performance tasks. q Sample SBAC items may be viewed on the website: http: //www. smarterbalanced. org/sampleitems-and-performance-tasks

Professional Development to Practice Let’s define… q. Selected-response assessments q. Constructed-response assessments q. Performance Assessments Pull out this template and pair up with someone. Using your current level of understanding, create a definition, identify the benefits and drawbacks of each type of assessment.

Professional Development to Practice Selected-Response Assessments… q Require students to select one response from a provided list q Types include multiple-choice; true-false; matching; short answer/fill-in-the-blank q Also include short answer/fill-in-the-blank (with a listing of answer choices provided) q Assess the student’s knowledge of factual information, main concepts, and basic skills

Professional Development to Practice Benefits of Selected-Response Items q. Can be scored quickly q. Can be scored objectively as correct or incorrect q. Covers a wide range of content

Professional Development to Practice Drawbacks to Selected-Response Items q. Tends to promote memorization of factual information rather than higher-level understanding q. Inappropriate for some purposes (performance, writing, and creative thinking) q. Lack of student writing in most cases, unless part of assessment design (Haladyna, 1997, pp. 65 -66)

Professional Development to Practice Five Roadblocks to Effective Item Writing q. Unclear directions q. Ambiguous statements q. Unintentional clues q. Complex phrasing q. Difficult vocabulary Popham, 2003 b, p. 64

Professional Development to Practice Key Points for Writing SR Items q. Choose a selected-response format(s) that aligns to the standard being measured q. Make sure the items will produce the needed evidence to determine mastery of the standard q. Include the vocabulary of the standard selected for assessment (as appropriate) q. Make sure the test question(s) require the same level of rigor as that of the standard q. Write each stem first, then write distractors

Professional Development to Practice Example of a Selected Response Item Many experts will tell you that television is bad for you. Yet this is an exaggeration. Many television programs today are specifically geared towards improving physical fitness, making people smarter, or teaching them important things about the world. The days of limited programming with little interaction are gone. Public television and other stations have shows about science, history, and technical topics. Which sentence should be added to the paragraph to state the author’s main claim? A. Watching television makes a person healthy. B. Watching television can be a sign of intelligence. C. Television can be a positive influence on people. D. Television has more varied programs than ever before.

Professional Development to Practice Example of another Selected Response Item Smarter. Balanced Assessment Consortium (SBAC) has multiple-choice items that cue students to select more than one answer.

Professional Development to Practice A third example of a Selected Response Item Use the illustration and your knowledge of social studies to answer the following question. 5. What colonial claim about the Boston Massacre is supported by this illustration? a. b. c. d. Most American colonists in Boston were killed. British soldiers fired on unarmed colonists. There were more soldiers than civilians at the Boston Massacre. Colonists were better equipped for war than British soldiers were.

Professional Development to Practice Constructed-Response Items… q Require students to organize and use knowledge and skills to answer a question or complete a task q Types include short-answer; open response; extended response; essays q More likely to reveal whether or not students understand can apply what they are learning. q May utilize performance criteria (rubrics) to evaluate degree of student proficiency.

Professional Development to Practice Benefits of Constructed-Response Items q Responses will contribute to valid inferences about student understanding better than those derived from selected-response items. q Measure higher-levels of cognitive processes. q Allow for diversity in student responses or solution processes. q Provide a better picture of students’ reasoning processes. q Promote the use of evidence to support claims and ideas.

Professional Development to Practice Drawbacks of Constructed. Response q. Take longer to score q. Can have errors in design q. Dependent on student writing proficiency q. A challenge to score consistently and objectively q. Must have clear rubrics for scoring criteria so scoring is not subjective

Professional Development to Practice Video Clip q http: //www. youtube. com/watch? v=gp 7 W 6 w. V-obs

Professional Development to Practice Key Points to Writing Constructed Response Items q Items should be open-ended and require students to create a response q Students must demonstrate an integrated understanding of the “unwrapped” concepts and skills q Items must match the level of rigor of the “unwrapped” standards q A proficient answer reflects the understanding of higher-order instructional objectives q Constructed-response items MUST be accompanied by scoring guides.

Professional Development to Practice Example of Constructed Response Item The table shows the price of different quantities of mediumsized apples at Tom’s Corner Grocery Store. What is the least amount of money needed to buy exactly 20 mediumsized apples if the bags must be sold intact and there is no tax charged? Be sure to show all of your work. Number of Apples Total Price Bag of 1 Bag of 6 Bag of 12 $. 30 $1. 20 $2. 10

Professional Development to Practice Another example of Constructed Response Item Scenario: Your friend is using the computer to type his one-page report for history. It is just two lines over one page and he doesn’t know how to make it fit on one page. Question: Using the proper computer terminology, describe to your friend two valid solutions for making his report fit on one page without deleting any of the content.

Professional Development to Practice A third example of Constructed Response Item Stimulus: Information of Sally’s Experimental Design Evaluate Sally’s experimental design. Identify two things Sally could have done differently to make her results more valid. Give reasoning for each one of your suggestions.

Professional Development to Practice Performance Tasks… q Require students to construct a response, create a product, or perform a demonstration. q Are open-ended and usually allow for diversity in responses or solution processes q Are evaluated using scoring criteria given to students in advance of performance q Highly engaging for students q Promotes critical thinking and/or problem solving q Promotes peer and self assessment

Professional Development to Practice Benefits of Performance Tasks q. Have the ability to assess multiple learning targets and application of knowledge q. Highly engaging for students q. Promotes critical thinking and/or problem solving q. Promotes peer and self-assessment q. Offers multiple opportunities for students to revise work using scoring guide feedback.

Professional Development to Practice Drawbacks of Performance Tasks q. Rubrics are more involved and take longer to develop q. Performances take longer to score q. Can have error in evaluative design q. Success is often dependent on factors other than those targeted for assessment (i. e. writing ability, verbal skills, physical abilities, etc. ) q. A challenge to score objectively and consistently

Professional Development to Practice Key Points to Writing a Performance Task q Student performance should be recorded on a checklist or q q q scoring rubric. Contains a written prompt that cues the student to perform some type of task that requires a demonstration, presentation or product creation. Shows connections by measuring learning targets across strands or content areas. Model what application of learning looks like in life beyond the classroom. Should measure mastery of multiple learning targets and higher-level cognitive processes. May be completed on one or more sittings or over time.

Professional Development to Practice Example of Performance Task Part I: During the U. S. Civil War, quilts became a popular item for women to make. You will write an informative essay summarizing the history and purposes of civil war quilts. To gain the information needed to write your essay, you will watch a video and read two articles about quilts that were made during the Civil War. Take notes because you may want to refer back to your notes while writing your essay. Part II: Your class is planning a field trip to a history museum. To help prepare for what you will see, write an informative essay about Civil War quilts. In your essay, discuss the history of the quilts, including the reasons people made these quilts during the Civil War, and explain how the quilts were made. Include evidence from the sources in Part I to help support the information you include in your essay. The rubric is provided showing how your essay will be scored.

Professional Development to Practice Another example of Performance Task

Professional Development to Practice A third example of Performance Task

Professional Development to Practice Reflection Time Think about the content, skills and processes you teach in your classroom and answer the three questions below. What content information and simple skills do you teach in your classroom that would lend itself well to being assessed by using SR items? What concepts, principles and processes do you teach in your classroom that would lend itself well to being assessed by the use of CR items. When might you ask students to do a PT to show their application of multiple skills and processes?

Professional Development to Practice Module #3 is a continuation of Module #2! Module 2 Module 3

Professional Development to Practice Now let’s practice! Using either the sample CFA Development Template, or personal work created in steps 1 through 5 previously, complete Steps 6 through 9 by selecting the appropriate types of assessments, matching up the test items with the learning target.

Professional Development to Practice Step 10: Define Achievement Levels When items are written, complete step 10 by describing how information from the scoring guides can be used collectively to determine achievement levels for students. These levels will be used later in the Data Team Process.

Professional Development to Practice Step 11: Review and Revise Review test items collaboratively to affirm the quality and appropriateness of the test items.

Professional Development to Practice Evaluate Your Test q Collectively, do the assessment items produce the necessary evidence to determine whether or not the student has mastered the standard(s) targeted for assessment? q Are the assessment items the most appropriate type to use to measure the targeted standard? q Do the assessment items require students to demonstrate the same level of rigor as specified by the targeted standard? q Are the items worded clearly and concisely? q Are the directions clear so students clearly understand what they are to do? q Are the items free of any type of bias?

Professional Development to Practice Using Data to Inform Test Writing Skills q. Use of data from a careful item analysis can help a teacher improve his/her test writing skills. q. Additionally, looking at student results can give the teacher ideas as to improvements that need to be made in instruction and/or curriculum. q. See next slide. Written by Jana Scott, MAP Instructional Facilitator, University of MO-Columbia, 20

Professional Development to Practice Possible Causes for Faulty Items/Low Scores on Items 1. Basic content or skills have not been addressed or taught. 2. Students are unfamiliar with the process needed to solve the problem/answer the question (i. e. problem solving, deductive/inductive thinking, making an inference, etc. ) 3. Students are unfamiliar with the format needed to answer the question. (i. e. political cartoon, letter, graph, etc. ) 4. Students are unfamiliar with the meaning of the language used in the test item. (i. e. compare and contrast, paraphrase, illustrate, evaluate, etc. ) 5. Lack of reading ability. Vocabulary used in item stem or stimulus is too difficult. 6. Wording of the item is unclear or confusing. 7. The rubric does not align with the test item. The rubric holds students accountable for something that was not cued in the item stem. 8. The rubric holds students to a standard that is not grade-level appropriate. 9. The item is asking the impossible or improbable (i. e. Asking for two similarities and two differences when there are not that many. Asking for three details when there are not that many. ) 10. The stimulus material used as a basis for item development is at fault. Written by Jana Scott, MAP Instructional Facilitator, University of MO-Columbia, 2007.

Professional Development to Practice Reflection Based on what you have learned today, q. What steps might you take in order to become a “top notch” writer of formative assessments and of the various types of test items? q What potential challenges do you foresee? How might these be overcome? q. What tools and/or resources might you use to ensure the assessments and assessment items you write are top quality?

Professional Development to Practice Profile

Professional Development to Practice Implementation Fidelity

Professional Development to Practice For Additional Learning q In-depth additional training from RPDC staff members on how to write quality selected response items, constructed response items and performance events. q Books contained in Bibliography on the next slide. q Websites and Videos about Formative Assessment q http: //www. youtube. com/watch? v=2 K 8 qb. I_Fz. GE q http: //www. amle. org/Publications/Web. Exclusive/Assessment/tabid/ 1120/Default. aspx q http: //www. ncpublicschools. org/docs/accountability/educators/fastr esearchresources. pdf q http: //www. ncpublicschools. org/accountability/educators/vision/form ative

Professional Development to Practice Bibliography q Common Formative Assessments: How to Connect Standards-Based Instruction and Assessment; Larry Ainsworth, Donald Viegut; Corwin Press; 2006. q Common Formative Assessment Training Manual; Second Edition; The Leadership and Learning Center; Houghton Mifflin Harcourt; 2011.