Blooming Biology g n Aligning A i g

Blooming Biology g n Aligning A i g in Teaching and Assessment: l Blooming Questions Most of these PPT slides are from 2 presentations in 2014 by Mary Pat Wenderoth, Dept. of Biology, University of Washington, used with permission from mpw.

How People Learn National Research Council 1999 Three major findings: 1. Address student’s preconceptions. (prior knowledge & misconceptions) 2. Build BOTH a deep foundation of factual knowledge & strong conceptual framework. 3. Enhance student’s ability to monitor their learning. (metacognition and self-assessment)

Aligning Teaching and Assessment Understanding by Design Wiggins and Mc. Tighe

Understanding by Design Wiggins and Mc. Tighe

Backward Design What should your students be able to DO at end of class? Wiggins and Mc. Tighe 1998 LEARNING OUTCOME What evidence do you collect to show that they can DO it? ASSESSMENT What practice do you design to help them gain skill? CLASS ACTIVITIES

Ways of characterizing the levels. Bloom’s of goals and assessments Taxonomy Bloom’s Taxonomy

Bloom’s Taxonomy of Cognitive Domains Evaluation- critique Synthesis - create Analysis- compare & contrast Application-- solve Comprehension-- define Knowledge-- facts Bloom, B. S. , Krathwohl, D. R. , and Masia, B. B. (1956)

Ways of characterizing the levels. Bloom’s of goals and assessments Taxonomy Bloom’s Taxonomy Anderson, L. W. and David R. Krathwohl, D. R. , et al (Eds. . ) (2001)

Lack of Alignment Learning Outcomes Class Activities Assessment Student frustration

Misalignment Learning Outcomes Class Activities Students Assessment

Are you familiar with IF-AT Forms? I have never seen IF-AT Form I am familiar with IF-AT Forms I have used IF-AT Forms for assessment a few times I have used IF-AT Forms several times I have no idea, but I would like to wave

Individual Quiz (~5 minutes) Instructions: 1. Write your name at the top of the quiz quarter sheet. 2. Enter the letter of the correct response to each multiple choice question on your quarter sheet. 3. Do not refer to books and notes. 4. Do not talk to classmates.

Answer quarter sheet

Group Quiz (~5 minutes) 1. Each group should have : a penny, a scratch off answer card, a copy of the quiz 2. Discuss each question and reach group consensus about the correct answer. 3. One person will then scratch off the group’s choice. 4. The correct answer has a “star”. 5. Partial credit: 1 point: “star” revealed with one scratch off, ½ point: for correct answer on 2 nd try, 0 points: for 3 -5 answers revealed

Types of questions? What types of questions were on this quiz? ◦ ◦ Bloom’s levels? Difficulty level? What types of learning objectives (from the instructor perspective) can be addressed with this assessment?

Immediate Feedback (IF-AT) forms What are the potential learning opportunities using this IF-AT formative assessment?

Immediate Feedback (IF-AT) forms Learning through formative assessment Allows for group assessments and facilitate peer instruction increases student engagement & facilitates peer instruction Provides immediate affirmation and/or corrective feedback necessary for student learning & Permits instructors to give partial credit for “proximate” knowledge - opportunity to give partial credit for multiple attempts - ensures that the last response is the correct one - provides students the opportunity to re-evaluate, re-read - allows for self-correction of mistakes - can reveal misconceptions as incorrect and provide validation of correct responses

Bloom’s Taxonomy of Cognitive Domains Higher Order Analysis Synthesis Evaluation Application Comprehension Knowledge Lower Order Figure 2. 1, page 17 – Assessment book (Dirks et al. 2014) Bloom, B. S. , Krathwohl, D. R. , and Masia, B. B. (1956)

Bloom’s Taxonomy of Learning & Blooming Biology Tool can 1 – Take the mystery out of learning for students 2 – Enhance student metacognition 3 – Help YOU align your teaching and testing

Research Question: Does MCAT only require regurgitation of FACTS? Research: “Bloom” 700 questions from MCAT GRE First year Medical School Undergraduate Biology High school AP Biology courses Application of Bloom’s Taxonomy Debunks the MCAT Myth Zheng et al. (2008). Science 319

How we “Bloom” a question? Find HIGHEST Bloom level Compared to a normal resting male of the same height and weight, Lance Armstrong’s stroke volume is greatly increased. Provide a physiological explanation for a large stroke volume. ✔ ✔ Evaluation Synthesis Analysis Application Comprehension Knowledge

Developed a rubric based on Bloom’s

Results from MCAT study • MCAT • Intro Biology courses • AP Biology courses • ‘GRE Biology • 1 st yr Medical School Zheng et al. (2008). Science 319 MCAT 2. 57 K C Ap An S E Undergrad 2. 43 AP Bio 2. 35 GRE 2. 32 Med 1. 93

70% 9713 questions 44 institutions 77 courses 60% 50% 40% 30% 20% 10% 0% K C Ap An Syn Eval CBE-Life Science Education 2010 9: 453 -440

Syllabus goals vs. Assessments K C Ap AN Syn Eval CBE-Life Science Education 2010 9: 453 -440

Average Bloom level by class size Average Bloom level across Institution type 2. 5 Bloom level r =0. 097 * 2 1. 5 1 0. 5 ta l To to ra l oc D te r as M al au r cc Ba As s oc ia t e at e 0 Momsen et. al CBE-Life Science Education 2010 9: 453 -440

Blooming Biology Tool Crowe, Dirks & Wenderoth 2008. CBE- Life Science Education 7: 368. Appendix C, page 36 – Assessment book (Dirks et al. 2014)

Science-specific skills • Calculations • Concept maps • Diagnoses • Graphing • Specific types of analysis • Experimental techniques • GIS skills

Science-specific skills Appendix D, pages 37 -39 – Assessment book (Dirks et al. 2014)

Write 6 questions for the same concept, on the same topic. Take one concept for A&P and Create questions at each level of Bloom’s See three examples in the Supplemental Materials in Crowe et al. 2008

Physiology: cardiac output Knowledge Which two variables determine cardiac output for an animal? Comprehension Define cardiac output and why it is significant. Application Lance Armstrong has a normal resting cardiac output 6 L/min yet his resting heart rate is only 40 beats/min. What is his stroke volume? Analysis Compared to a normal resting male of the same height and weight, Lance Armstrong’s stroke volume is greatly increased. Provide a physiological explanation for a large stroke volume. Box 2. 2, pages 20 -21– Assessment book (Dirks et al. 2014)

How we “Bloom” a question. Find HIGHEST Bloom level Compared to a normal resting male of the same height and weight, Lance Armstrong’s stroke volume is greatly increased. Provide a physiological explanation for a large stroke volume. ✔ ✔ Evaluation Synthesis Analysis Application Comprehension Knowledge See Example of Geoscience Question in Appendix A, pages 26 -27– Assessment book (Dirks et al. 2014)

“Bloom” Geoscience Questions on worksheets Use the Biology Blooming Tool & Science Specific Skills to determine the Bloom’s Level of each question. Find the HIGHEST Bloom level for each question. If a particular question has been used as an example in class, then its Bloom’s level is Knowledge (recall) not application. An instructor can not accurately, objectively ‘Bloom’ their own exam. If you are “Blooming” an exam, make a histogram with % of the points on the exam vs. the Bloom’s level.

How can Table 3 from Crowe, et al. : Bloom’sbased Learning Activities for Students “BLASt” be used by students and faculty?

BLASt

How can Table 3 from Crowe, et al. : Bloom’sbased Learning Activities for Students “BLASt” be used by students and faculty? • Exam review, self-assessment of performance regarding HOCs and LOCs • Office hours, ask students to highlight what they do when they study. • Help guide and diversify student study time. • Help guide active learning in the classroom.

Post Exam PRE- Bloom’s: “But I studied so hard!” POST- Bloom’s: “I need help with analysis level questions” PRE- Bloom’s: “Question 3 was tricky!” POST- Bloom’s: “#3 was an application level question ”

Student’s comment “I remember initially thinking, “Why are we wasting valuable class time on Bloom's taxonomy? ” I felt that Bloom's taxonomy was a burden, but I now use Bloom's taxonomy unconsciously to attack many problems. It is a method used to help organize my thoughts before I act. ”

Designing Learning Activities How do you help students practice (formative assessment) and study for conceptual understanding and ability, i. e. higher level Bloom’s? Think – Pair – Share

Designing Learning Activities Formative Assessment & Learning Activities are interdependent. • • Assessments should be aligned with learning outcomes Assessment, outcomes & activities (practice) should be aligned by Bloom’s levels. How can small-scale formative assessments help you create opportunities for active learning? ◦ See Table 4. 1 (page 66) for Small Scale Formative Assessments. ◦ Which of these can be used ‘in-seat’ and online? Think – Pair – Share

Large-Scale Learning Activities How can large-scale formative assessments often involve transformation of courses or sections of your course, but can be used both in-seat and online. ◦ See Table 4. 2 (page 70) for Large Scale Formative Assessments.

Dirks, Wenderoth & Withers Handelsman, Miller & Pfund WH Freeman Publisher

How to do an Assessment Workshop See Chapter 8 of the Assessment book (Dirks et al. 2014): How would you like to disseminate what you have learned as a “change agent”?

How to do an Assessment Workshop See Chapter 8 of the Assessment book (Dirks et al. 2014): How would you like to disseminate what you have learned as a “change agent”? This chapter has a good overview. Workshop activities are clear and easy to follow. This work is important and in service of student learning and success – don’t hesitate to do a workshop for your peers!

Thanks to Mary Pat Wenderoth & the UW Biology Education Research Group – BERG including, Scott Freeman Alison Crowe and others https: //sites. google. com/site/uwbioedresgroup/

Thanks to my collaborators in Physiology Educuation Research: Mary Pat Wenderoth (UW Seattle) Joel Michael (Rush Medical School) Harold Modell (Bastyr University) Bill Cliff (Niagara University) the late Ann Wright (Canisus College) see physiologyconcepts. org & PULSE fellows: see http: //pulsecommunity. org/

Thank you! Thank you for your time and participation.

Reflection Reflect on the assessments your students did last year. What did you do that best allowed your students to demonstrate what they knew and how they could apply what they learned in your course? What left you frustrated? What would you like to improve? What did you discuss or hear or think about this morning that resonated with you? 11: 30 -11: 45 am

Action Plan: What one or two changes to formative or summative assessment will you attempt in your course(s) next year? What assessments will your stop doing, modify, or try for the first time will you do in your next class? What resources would you want to use? Resources could include people, books, papers, SAGE 2 YC community resources, etc. 11: 45 -12: 00 pm

What is an IF-AT form? • IF-AT = Immediate Feedback Assessment Technique • A multiple-choice assessment system – pre-made forms – scratching off the correct answer reveals a “star” – 4 choices (A-D) or 5 choices (A-E) – 10, 25 or 50 question forms • IF-AT studies have been done in several disciplines.

What is an IF-AT form? Simonson 2014

Cost? Minimum Order is 500 forms (4 different keys, 125 each) 10 questions, 5 answers – 500 forms for $90 25 questions, 5 answers – 500 forms for $115 50 questions, 5 answers – 500 forms for $115 Most cost effective to use longer forms, e. g. 50 questions Collect after each 10 point quiz Time consuming to get correct forms to each group for each 10 -point quiz http: //www. epsteineducation. com/home/order/default. aspx

Does Group Testing Harm Individuals? Individual and group performance on the two-stage review test questions. Mc. Donnell (2014)

Does Group Testing Harm Individuals? Maxwell et al. (2015)

Do 2 nd chances = guessing? Merrel et al (2015)

Why use IF-AT forms? “Assessment as learning” or “learning through assessment” Increase student learning Allows for group assessments and facilitate peer instruction increases student engagement facilitates peer instruction Provides immediate affirmation and/or corrective feedback necessary for student learning & Permits instructors to give partial credit for “proximate” knowledge - opportunity to give partial credit for 2 nd and 3 rd attempts - ensures that the last response is the correct one - provides students the opportunity to re-evaluate, re-read - allows for self-correction of mistakes - can reveal misconceptions as incorrect and provide validation of correct responses

IF-AT Review & Learning Gains Mohrweis & Shinham, Business, Northern Arizona University

Students think IF-AT helps Patrick Murphy, Economics, Saint Leo University

Students think IF-AT helps Maxwell (2015)

Instructor Feedback for Review Individual and group performance on the two-stage review test questions. Individual versus group scores identified the review material that students still struggled with, after the collaborative group test. Focus in-class review on starred topics for Mitosis / Meiosis. Lisa Mc. Donnell, Biology, UBC

Using IF-AT forms How could you imagine Using IF-AT forms in class? Any Reservations? What pedagogical or ‘scientific teaching’/cognitive science-based reservations do you have about this type of individual and/or group assessment? What Structural / Institutional Challenges and Barriers might exist?

Using IF-AT forms How might we use these? Low stakes vs. significant factor in course grade. Partial credit for more than one scratch off. Types of Groups: Fixed groups for entire quarter; self selected groups; assigned groups, random groups. Group #? 2 -5 Reservations (given Limitations and Feasibility): Do you have any reservations about this type of group assessment? Challenges and Barriers: Time consuming distribute, collect and grade in large classes, >100 Writing assessments that align with keys takes more time

References Maxwell, EJ, Mc. Donnell, L and Wieman CE. (2015) An Improved Design for In. Class Review. Journal of College Science Teaching 44(5): 49 -52. Mc. Donnell, L. (Dec 2014) Save class time with a more efficient review strategy: two-stage review activity. http: //ls-cwsei. biology. ubc. ca/2014/12/18/saveclass-time-with-a-more-efficient-review-strategy-two-stage-review-activity/ Merrell, JD, Cirillo, PF, Schwartz, PM and Webb, JA. (2015) Multiple-Choice Testing Using Immediate Feedback—Assessment Technique (IF AT®) Forms: Second-Chance Guessing vs. Second-Chance Learning? Higher Education Studies 5(5): 50 -55. Mohrweis, LC and Shinham, KM. (2015) Enhancing Students’ Learning: Instant Feedback Cards. American Journal of Business Education 8(1): 63 -69. Murphy, Patrick Ryan. Using IF-AT Cards in Principles Classes for Group Quizzes. Simonson, SR. 2014. Making students do the thinking: team-based learning in a laboratory course. Advances in Physiology Education. 38(1): 49 -55. A good introduction to IF-AT http: //faculty. business. utsa. edu/manderso/presentations/scratchoffquizzes. pdf