Writing Multiple Choice Questions and Assessing Short Answer

Writing Multiple Choice Questions and Assessing Short Answer Responses Alice Aspinall Carlinda D’Alimonte

Writing Multiple. Choice Questions to Measure Higher-Level Thinking Skills This portion of the slide show is adapted by C. D’Alimonte from a Presentation by: Jana L. Scott Missouri Assessment Program, University of MO- Columbia 2

Rationale for Training • • EQAO Test Questions Poor performance of students Incongruence We know that students are capable of passing the test, but need help and practice. • Working on a student’s ability to answer multiple-choice questions and short answer responses will help to make sure he or she passes. 3

What is the OSSLT • A test of students’ writing and reading abilities they must pass in order to graduate. • Students read several types of text (narrative, informational, and graphical) and answer multiple-choice and short answer questions on them. • Students write several pieces for different purposes (news report, supported opinion, summary paragraph)

Our Focus • MCQ’s • Open Response: Short Written Response to Reading Passages

MCQ’s: Three Things to Consider 1. Student’s Process 2. Structure of the Question 3. Cognitive Process Involved in Response 6

Process

Three-Step Process Students Should Follow Step One • Before they attempt any of these, read the passage for basic understanding.

Step Two Read the first question. Do they know the answer? If yes, read through all the choices and make sure they select the most correct answer. Do not choose the first correct answer they see.

Step Three If they do not know the answer right away: - read the four choices - cross out obviously wrong answers - highlight key words in the question and scan the text for those words to try to find the right answer Be aware that some answers may seem correct, but there is only one best answer.

Next: • Go back to any questions that they circled and left unanswered. • If time permits, repeat the steps to answering multiple choice questions and review the passage. • Make a selection - always. Never leave an answer blank.

Repeat this three-step process with each question in the passage.

Review – Answering MC Questions

Structure

Structure: How the Question is Constructed - Use questions instead of incomplete statements - The student should be able to answer the question before reading the possible answers or have a clear, specific idea of what is being asked - Use plausible distracters - Keep options at a similar length - Be grammatically correct - Avoid clues to the answer - Use only “most correct” answer - Do not use “all of the above, ” “none of the above, ” or negative questions or answers - Give clear instructions

Cognitive Process

Multiple Choice Questions There are different types. Sometimes you 1. find the answer right in the text. 2. need to infer – that is, read between the lines or extend beyond what is written in the passage.

18

Making an Inference Means…. . • • • Drawing conclusions Making deductions Making conjectures Speculating Making presumptions 19

Cognitive Process Emphasize Higher-Level Thinking – Think of Bloom’s Taxonomy

MCQ’s Can Require or Ask For ØGeneralizations ØSummarizations ØConclusions ØJustify Motives or Reasons ØJudgments or Predictions ØShowing Relevance or Significance ØJustification/Evaluation of Methods, Procedures, Solutions, etc. ØSupport ØOpinions Related to Authorial Choices/Text Features ØApplication ØComparison ØInterpretation/Explaining Cause-Effect Relationships 21

Generalizations: A General Thought, Idea or Premise Arrived at as a Result of or Based on Specific Information ______________ 1. 2. Which word best describes the main character? the central conflict? the initial problem? What is most likely theme of the story? a. b. c. d. 3. 4. 5. Nice guys often finish last Kindness is not always its own reward Kindness is its own reward Time really does heal How can you tell that more discoveries are likely to be made in _____? Based on the data/chart, what might one generalize? Which equation best reflects the input/output information contained in the chart?

Summarizations: A Statement Which Specifies the Main Idea or Main Points ______________ 1. Which is the best summary of the article/paragraph/excerpt? a. b. c. d. 2. 3. There are many types of jobs and many people looking for work. It is important to find the best person suited to the job. Unemployment is on the rise. There are many unemployed people who are applying for different types of jobs. Job openings are on the rise. It is important that the unemployed work hard to try to find a job. Unemployment is on the rise. Job openings are on the decline due to many economic factors Which statement best summarizes the results of Experiment 2? Which statement best summarizes the impact of the Truman Doctrine/_____Treaty/etc. upon world affairs?

Conclusions: A Judgement Formed After Critical Thought ______________ 1. 2. What might the reader conclude based on John’s actions? John put some water in a graduated cylinder and wrote down the height of the water. Then, he placed a rock into the cylinder and wrote down the height of the water again. John subtracted the number of milliliters of the water without the rock from the number of milliliters of water with the rock. What did John do? a. b. c. d. 3. 4. Found the volume of the rock Found the mass of the rock Found the amount of pressure needed to displace water Found the size of rock needed to displace water Based on the data in the chart/graph/data set, what might one conclude? Based on the results of the problem, what might one conclude?

Justifying Motives or Reasons: Consider Actions, Ideas, Principles, etc. ______________ 1. What is the most likely reason Tom walked out of the meeting? a. He was afraid of being confronted about his previous actions b. He did not feel prepared and needed time to consider how he would handle his presentation c. He was angry about what had happened in his boss’s office that morning d. He remembered that he needed to make an urgent phone call 2. 3. 4. Which of the following best justifies Fred Scott’s decision to sue his employer? A student recorded the dimensions and location of a shadow cast by a tree at sunrise. Every hour throughout the day, until sundown, the student repeated this same process. At the end of the day, the student compared the dimensions and the locations. What might the student be trying to find out? Which statement best describes the role a circle plays in being able to find the area of a minor segment?

Making Judgements/Predictions: Determinations Related to What Might Happen or Occur Next ______________ 1. 2. Based on the information given, which statement best describes what might happen to John’s annual expense account if he takes in two more roommates for six months (December to May)? If the US had decided to remain uninvolved in WWII, what would most likely have happened? a. There would have been little change in the outcome of the war b. The war would have lasted much longer c. Japan would have invaded Germany d. Germany would have invaded Japan 3. 4. Based on the diagram showing how John wired the electrical circuit, which statement best predicts what will most likely happen when John pushes the ‘ON’ switch? Based on the data showing the amount of sunspot activity over the course of the past ten years, which statement best describes how future communications (radio, television, telephone, and satellite signals) might be affected?

Showing Relevance or Significance: Relate to the Matter Under Consideration or Importance of Something ______________ 1. What does the robe symbolize in the passage? a. Love b. Greed c. Charity d. Hope 2. 3. 4. Given a data set, what role would the data play in the creation of a testable question? What role did the “control group” play in the scientific investigation/experimentation/inquiry? Why is rewriting the problem using the reciprocal necessary when dividing fractions?

Requiring Justification/Evaluation: Examination of Methods, Procedures, Solutions ______________ 1. 2. 3. Which of the following is the best procedure to use to determine the amount of heat released when a substance is burned in oxygen? A student was asked the following question: ‘Briefly list and explain the various steps in creating and implementing a research plan. ’ The student’s answer was this: ‘A research plan should contain ____. ’ How would you judge this student’s answer? Which of the following best describes the effectiveness of FDR’s New Deal Plan as a basis to improve the depressed US economy? a. b. c. d. Highly effective Somewhat ineffective Highly ineffective

Providing Support: Reasoning or Evidence Behind Ideas ______________ 1. 2. 3. Which statement best supports the idea that…? Which of the character’s actions best supports the idea of John’s impulsiveness? Which of the following hypotheses about the relationship between temperature and the density of a solid is best supported by the data in Figure 4? As the temperature of a solid increases, what does the density of the solid do? a. Increases only b. Decreases only c. Increases, then decreases d. Decreases, then increases 4. What evidence best supports the idea that Truman’s decision to drop the atomic bomb led to an early end to World War II?

Asking for Opinions on Authorial Choices/Text Features: Ideas About the Writer’s Purpose, Techniques or Use of Format ______________ 1. 2. 3. What is the most likely reason the author uses slang/ a diary format/subtitles/italics/graphs/etc? What purpose is best served by the semicolon/brackets/bold/dash/quotation marks/etc. in paragraph 5? What is the most likely reason that the author included paragraph 4 in this passage? a. To provide a counter argument b. To provide more background information c. To show that she is being fair d. To create doubt in the reader’s mind

Requiring Application: The Use of Previously Learned Information ______________ 1. 2. 3. Which set of questions would be best to use to guide research about effective agricultural techniques used in the 19 th century? Which of the following BEST describes the change that occurs when a batter hits a baseball? The table shows the price of different quantities of medium-sized apples at Tom’s Corner Grocery Store. What is the least amount of money needed to buy exactly 20 medium-sized apples if the bags must be sold intact and there is no tax charged? # of Apples Bag of 1 Bag of 6 Bag of 12 Price $. 30 $1. 20 $2. 10 a. $3. 60 b. $3. 90 c. $4. 20 d. $5. 50

Making Comparisons: ______________ 1. To what popular children’s story reviewed in class might theme of this story/poem best be compared? a. Little Red Riding Hood b. Cinderella c. The Lion King d. Chicken Little 2. To what might _____ best be compared?

Requiring Interpreting/Explaining Cause and Effect Relationships: Describing Outcomes or Impact of Events or Situations ______________ 1. Why does investing money in the stock market pose a risk for an investor? a. b. c. d. 2. 3. 4. 5. There is a high level of corruption among investors The stock market is volatile History has shown that stock prices do not rise over long periods of time History has shown that stock prices decrease over long periods of time Which of the following best describes the effect of _____on _____? If the nominal gross national product (GNP) increases at a rate of 10% per year and the GNP deflator increases at 8% per year, what happens to the GNP? Use the line graph to answer the question. If the vertical scale started at 0 and each line was labeled by whole numbers up to 10, what effect would it have on the data displayed in the graph? Which statement best describes the impact the opening of the Panama Canal had upon the US economy?

Your Task • Work in a small group to write two higher-level multiple-choice questions that will measure a part of any two objectives. • Use the Planning Chart to write your questions and determine which higher order thinking task you are tapping. • Refer to the sample questions and read any clarifications. • Write structurally solid questions that force the student to use higher order thinking.

Worksheet 35

Example Which of the following best describes the reason it is important to follow the directions listed in the passage? a. To be able to accurately complete income tax forms b. To be able to learn to file taxes before deadline c. To ensure the government will not do an audit d. To ensure the correct information is on all forms X X 36

Share Your Favorite Question Tell …. • What higher order thinking skills you are tapping, • your question and answer choices. 37

Assessing Short-Answer (Open-Response) Responses

Types of Short-Answer Questions 1. Answer the question using evidence from the text. 2. Answer the question using evidence from the text and your own ideas.

Steps to Answering Short-Answer (Open Response) Questions After Reading the Text Step One: Categorize the Question (evidence from text only or both evidence from text and own ideas) Step Two: Plan the Answer Step Three: Write the Answer Step Four: Check Conventions (grammar, punctuation, spelling, capitalization)

Step One: Categorizing The Question

Short Answer Question One Explain why the final sentence is an effective conclusion to the selection. Use relevant and specific information from the selection to support your answer.

Short Answer Question Two Use information in this selection and your own ideas to explain whether or not Ben is a good waiter.

Process What was the same/different about your approach to the two questions?

Step Two: Planning The Answer

Step Two: Planning The Answer • Highlight the parts of the question that tell what’s expected. • Underline the words in the question that you will use to form your answer. • Rough out the answer. Use point form or highlight the examples from the text you want to include. Use the rough notes section for this. • Use point form to jot down connections to your own ideas (if required).

Question Type One Explain why the final sentence is an effective conclusion to the selection. Use relevant and specific information from the selection to support your answer.

Question Type Two Use information in the selection and your own ideas to explain whether or not Ben is a good waiter.

Highlight Details

Making Connections When Required • Jot down answers to these questions as you try to make connections to your own ideas: – Text to self connections (Have I been in a situation like this before? What was important? What did I learn? What was the same/different? ); – Text to media (Have I read, seen or heard about this before? What did I learn? Was important, etc); – Text to world (Do I know someone who has been in a situation like this? What was important what did the person learn. ).

Step Three: Writing the answer

Question Type One Explain why the final sentence is an effective conclusion to the selection. Use relevant and specific information from the selection to support your answer.

Open Response Type One • Answer • Evidence

Answer the Question • Question stem – Use key words from the question. • Main idea. For example: The final sentence is an effective conclusion to the selection because it provides us with the information we need to understand the behaviour of all the characters.

Evidence from the Text • Find the examples in the text to support the main idea. For example: It explains why Ali and Dana insisted on having a table in Ben’s section and later responded with rude comments. It also explains why Ben asks, “That’s it? You waited twenty minutes in the cold to order one plate of fries? ”

Answer and Evidence The final sentence is an effective conclusion to the selection because it provides us with the information we need to understand the behaviour of all the characters. It explains why Ali and Dana insisted on having a table in Ben’s section and later responded with rude comments. It also explains why Ben asks, “That’s it? You waited twenty minutes in the cold to order one plate of fries? ”

Question Type Two Use information in the selection and your own ideas to explain whether or not Ben is a good waiter.

Open Response Type Two • Answer • Evidence • Connections

Answer the Question • Question stem – Use key words from the question. • Main idea. For example: Ben is a good waiter because he is polite and treats his siblings as regular customers even though they are teasing him.

Evidence from the Text • Find the examples in the text to support the main idea. For example: Ben used a polite manner when he spoke to them. He also apologized that they had to wait twenty minutes for a table.

Connections • Connecting the text to media, to the world, to self • What does the reader already know that has a connection to the text? For example: Good waiters don’t argue with customers and are always polite. They believe the customer is always right.

Answer, Evidence and Connections Ben is a good waiter because he is polite and treats his siblings as regular customers even though they are teasing him. Ben used a polite manner when he spoke to them. He also apologized that they had to wait twenty minutes for a table. Good waiters don’t argue with customers and are always polite. They believe the customer is always right.

Keys to Answering OR Questions Students need to • know and use strategies • make connections when asked