An introduction to physics education research assessments and

An introduction to physics education research, assessments, and modeling in laboratory courses Benjamin Zwickl University of Colorado Boulder & Rochester Institute of Technology (Fall 2013) Lunch Seminar in Fudan Physics Department, May 7, 2013

From cavity optomechanics to PER Heather Lewandowski Junior Faculty AMO Physics/JILA Jack Harris HS BS Ph. D, Yale Instructor Postdoc

PART 1: AN INTRODUCTION TO PHYSICS EDUCATION RESEARCH

Greetings from the University of Colorado Physics Education Research Group! Right next to the Rocky Mountains

A view of campus

Physics Education Research @ Colorado “Why not try a scientific approach to science education? ” 2001 Nobel Prize in Physics with Eric Cornell (CU) Wolfgang Ketterle (MIT) Carl Wieman Research into how students learn The Science Education Initiative at the University of Colorado create and support improvements in science education

A Growing PER Group at University of Colorado Faculty Graduate students Melissa Dancy Mike Dubson Noah Finkelstein Heather Lewandowski Valerie Otero Robert Parson Kathy Perkins Steven Pollock Carl Wieman (on leave) Ian Her Many Horses Takako Hirokawa George Ortiz Mike Ross Ben Spike Enrique Suarez Ben Van Dusen Bethany Wilcox Rosemary Wulf Post-docs Charles Baily Danny Caballero Stephanie Chasteen Julie Chamberlain Karina Hensberry Katie Hinko Emily Moore Ariel Paul Noah Podolefsky Benjamin Zwickl Staff Shelly Belleau John Blanco Kathy Dessau Jackie Elser Molly Giuliano Kate Kidder Trish Loeblein Chris Malley Susan Nicholson-Dykstra Oliver Nix Jon Olson Emily Quinty Sam Reid

A scientific approach to teaching “we [need to] approach the teaching of science like a science. ” Carl Wieman A scientific approach includes: 1. Doing research on how students learn 2. Applying those lessons in the classroom. 3. Improving teaching using evidence from your classroom. 4. Using technology effectively. Jo Handelsman, et al. Wieman, C. Why Not Try a Scientific Approach to Science Education? Change: The Magazine of Higher Learning, (September/October), 9– 15. (2007)

What is Physics Education Research (PER)? PER involves: • Developing models of learning Theory • Creating tools for measurement Experiment • Gathering evidence of impact • Designing new curricula and approaches to teaching Application Docktor, J. L. , & Mestre, J. P. (2010). A Synthesis of Discipline-Based Education Research in Physics, A National Research Council Study; Mc. Dermott, L. C. (1999). Resource Letter: PER-1: Physics Education Research. American Journal of Physics, 67(9), 755.

Highlights of Physics Education Research Effective use of technology: Ph. ET simulations > 100 Research-based simulations Free activity guides >100, 000 downloads Kathy Perkins Ph. ET Director phet. colorado. edu

Highlights of Physics Education Research Assessment: The Force Concept Inventory (29 multiple-choice questions) Example: Assessing Students’ understanding of Newton’s First Law For this question: Before Instruction: About 45% Correct After Instruction: About 80% Correct Hestenes, D. , Wells, M. , & Swackhamer, G. (1992). Force Concept Inventory. The Physics Teacher, 30(3), 141– 157.

Highlights of Physics Education Research Force Concept Inventory Results Red = Traditional lecture & recitation Blue = Research-based curricula & Active learning. Colorado: Traditional recitations Fraction of courses Less Learning Colorado: Tutorials & Learning Assistants More Learning R. Hake, ”…A six-thousand-student survey…” AJP 66, 64 -74 (‘ 98).

Highlights of Physics Education Research-based strategies to improve physics learning: 1. 2. 3. 4. 5. 6. 7. Elicit and address students’ difficulties. Engage in a variety of problem solving activities. Work in groups. Explicitly explain reasoning. Use conceptual and quantitative reasoning together. Organize ideas into bigger structures. And more! Meltzer, D. E. , & Thornton, R. K. (2012). Resource Letter ALIP– 1: Active-Learning Instruction in Physics. American Journal of Physics, 80(6), 478

“Active Learning” in advanced classes CU courses transformed using: • • • Explicit Learning Goals Interactive Lectures Transformed Homework problems Common Student Difficulties In-Class Group Activities/Tutorials Concept Tests ("Clicker" questions) Students at University of Colorado Active learning is more effective

“Active Learning” in advanced classes All resources available at: http: //www. colorado. edu/sei/departments/physics. htm

PART 2: ASSESSMENTS

A variety of assessments 1. Introductory conceptual surveys • • • FCI (Force Concept Inventory) FMCE (Force and Motion Conceptual Evaluation) BEMA (Brief Electricity and Magnetism Assessment) 2. Advanced conceptual surveys • • CUE (Colorado Upper-division Electrostatics Assessment) CCMI (Colorado Classical Mechanics Instrument) 3. Attitudes and beliefs surveys • • CLASS (Colorado Learning Attitudes about Science Survey) E-CLASS (Experimental Physics CLASS)

Attitudes and beliefs: CLASS What are attitudes and beliefs surveys? They ask questions about students views on • Understanding physics. • Learning physics. • Relevance of physics to everyday life. Why do such surveys matter? • • Students often respond less like experts. Less expert-like students have perform lower. What is not on these surveys? • • No questions about specific physics concepts. No solving problems.

Attitudes and beliefs: CLASS Examples of questions from the Colorado Learning Attitudes about Science Survey (CLASS) 1. A significant problem in learning physics is being able to memorize all the information I need to know. Strongly Disagree 1 2 3 4 5 Strongly Agree 2. When I am solving a physics problem, I try to decide what would be a reasonable value for the answer. Strongly Disagree 1 2 3 4 5 Strongly Agree 3. I think about the physics I experience in everyday life. Strongly Disagree 1 2 3 4 5 Strongly Agree 41 questions all together, grouped in categories. www. colorado. edu/sei/class

Example CLASS Results in Chinese High School 8 CLASS Categories Personal Interest Real world Problem solving general Problem solving confidence Problem solving sophistication Sense-making/effort Conceptual understanding Applied conceptual understanding More expert-like responses Other comparisons include: • CLASS scores on traditional and transformed curricula. • CLASS scores compared to students’ grades • CLASS scores for different majors (physics, engineering, non-science, etc. ) Zhang, P. , & Ding, L. Large-scale survey of Chinese precollege students’ epistemological beliefs about physics: A progression or a regression? Physical Review Special Topics - Physics Education Research, 010110

CLASS for Experimental Physics 1. A new survey focused on experimental physics 2. Validated for all levels of university students 3. A common evaluation tool than can be applied to a variety of lab experiences across the world.

Use learning goals to develop questions Modeling the physical system Designing apparatus and experiments Modeling the measurement system Statistical analysis for comparison MODELING LEARNING GOALS COMMUNICATION Argumentation Authentic forms in physics Trouble-shooting DESIGN Computer-aided data analysis TECHNICAL LAB SKILLS Computer-aided measurement Test and measurement equipment + enjoyment, teamwork, confidence

E-CLASS Dimensions Affect Scientific argumentation Confidence Experimental design Math-Physics-Data connection Modeling the measurement system Physics community Purpose of labs Statistical uncertainty Systematic error Troubleshooting

Validation • 42 interviews with all levels of college students. • Students took the survey and then explain how they answered it in an interview format. Students: “What do I think vs. what should I think? ” Add: “What would a physicist say? ” Students: about lab class or their research lab? Modify: “What would physicists say about their research? ” Students: what about theorists? Final: “What would experimental physicists say about their research? ” (final)

Scientists vs. Physicist vs. Experimental Physicist What do you think of when you hear the word scientist? Students: “vague, encompasses geology, biology, chemistry, physics”, “Bill Nye”, “generic person in a lab in a white coat” Can you name any physicists? “Newton, Einstein”, sometimes “Hawking”, “Michio Kaku”, “Archimedes”, “my professor”, a few mentions of “that guy who won the Nobel Prize”/“Wineland” What do you think the difference is between a physicist and an experimental physicist? “A physicist draws pictures on paper”, “A physicist is a paper and pencil physicist”, “I don’t know”, “An experimental physicist is a physicist that does experiments? ” Were the questions asking “What would an experimental physicist say about their research? ” awkward to you? “No, I think they’re still pretty applicable to what I do. ”

E-CLASS Design Paired Questions Pre and Post only 1. Students’ personal attitudes and beliefs 2. Students’ view of experts Core Statement: ( e. g. , Whenever I use a new measurement tool, I try to understand its performance limitations. ) 3. Does this practice help to earn a good grade? Actionable Evidence for Instructor

CLASS for Experimental Physics (E-CLASS) Examples of questions from the E-CLASS 1. When doing an experiment, I try to understand how the experimental setup works. Strongly Disagree 1 2 3 4 5 Strongly Agree What do YOU think? What would experimental physicists say about their research? 2. If I wanted to, I think I could be good at doing research. Strongly Disagree 1 2 3 4 5 Strongly Agree What do YOU think? What would experimental physicists say about their research? Pre: 30 paired questions. Post: Additional 23 tinyurl. com/E-CLASS-Sp 13 -Post

Example E-CLASS Results Fall 2012 results from an introductory lab course for scientists Circle = Pre, Arrow = Pre/Post Shift Results for all students in similar classes More expert-like

Example E-CLASS Results Fall 2012 results from an introductory lab course for scientists More expert-like

Example E-CLASS Results Fall 2012 results from an introductory lab course for scientists Number identifies the question Positive shifts Negative shifts This class only had a small correlation between importance for earning a good grade and the shift in attitudes.

E-CLASS All 6 faculty chose “agree” or “strongly agree. ” Benjamin M. Zwickl, Noah Finkelstein, and H. J. Lewandowski, PERC Proceedings 2012

MODELING IN THE PHYSICS LABORATORY

Modeling emerged as a key lab learning goal 22 faculty Modeling Design Literature Many Inputs LEARNING GOALS Community Personal experience Communication Technical lab skills B. Zwickl, N. Finkelstein, and H. J. Lewandowski, Am. J. Phys. 81, 1, 63 -70 (2013)

Defining expertise through learning goals Modeling the physical system Designing apparatus and experiments Modeling the measurement system Statistical analysis for comparison MODELING DESIGN LEARNING GOALS COMMUNICATION Argumentation Authentic forms in physics Troubleshooting Computer-aided data analysis TECHNICAL LAB SKILLS Computer-aided measurement Test and measurement equipment More info at: http: //tinyurl. com/Advanced-Lab-LGs B. Zwickl, N. Finkelstein, and H. J. Lewandowski, Am. J. Phys. 81, 1, 63 -70 (2013)

A model of a pendulum Model: An abstract version of a real system that is (1) Simplified (2) Predictive (3) has specified limits to its validity Real Abstract 1) Simplified 2) Predictive 3) Limited validity Modeling: Developing, testing and refining models

Modeling is both a process of experimental physics and a way to “understand” a complex experiment. 1. Combine quantitative predictive models with quantitative measurement. Ø The models are as important as the data. 2. Never forget the basic physics ideas and principles that govern the system. Ø It is never as simple as matching results to an equation. 3. Recognize the idealizations and simplifications that limit the model’s accuracy. Ø Is it valid to apply the model to the experimental apparatus?

Modeling is both a process of experimental physics and a way to “understand” a complex experiment. 4. Measurement devices are not “black boxes. " Ø Understand how the measurement tools work. 5. All sources of error and uncertainty are considered. Ø Random uncertainty AND systematic error. 6. Experiments are typically iterative and systematic. Ø Models and apparatus are refined in many stages.

A modeling framework for labs 1. All parts of apparatus included Measurement probes Real-world physical system Data abstraction Principles Approximations? Measurement model Specific situation Idealizations? Unknown parameters? 2. Data and theory compared Results with uncertainties Physical system model predictions Comparison. Is the agreement good enough? No Specific situation Idealizations? Unknown parameters? Yes How can I get better agreement? Improve the measurement model Approximations? Stop Improve the physical model 3. Iterative cycle

Modeling the simple pendulum predictions ninamccurdy. com Are the idealizations valid? What predictions are most useful?

Modeling the photogate Simple model for timing gate: • Light from IR LED shoots across to photodiode. • Gate starts/stops when light blocked • Point emitter & point detector

Comparison between data and predictions Data from photogate Predictions from model of pendulum Period (s) Comparison. Is the agreement good enough?

Model refinement as inquiry Systematic Error is just an RESOLVE THE ERROR: incomplete model. Identify systematic error sources in the: 1. pendulum model & 2. photogate Predict the effect of the systematic error source Experimentally test a systematic error source Refine pendulum model & calibrate photogate Redesign apparatus

Schematics from Teach. Spin PS 1 -A Lab Manual EXAMPLES OF USING MODELING TO UNDERSTAND COMPLEX EXPERIMENTS

The challenge: Doing without understanding A pulsed NMR experiment is used to measure T 2 (spin-spin relaxation time) ü ü ü Atomic scale dynamics. Invisible fluctuating B-fields Faster than a blink of an eye Sophisticated physics ideas. Sophisticated apparatus. (Graph and text from student lab report. ) Sophisticated Understanding? Good data and a fundamental misunderstanding in the same lab report.

Modeling Nuclear Magnetic Resonance How do proton spins change in time in response to the magnetic fields? Real-world physical system • • • Initial state Principles Hydrogen nuclei in sample vial Transmitter coils and pulsed RF drive Constant magnetic field abstraction Physical system model Specific situation Final state Schematics from Teach. Spin Student’s computer simulation PS 1 -A Lab Manual Mathematica code to numerically solve equations

Previous polarization of light lab • Qualitative prompts: Examine… Observe… What happens? • No predictive model • No interpretation of data

NEW: Transformed polarization of light lab Introduce Jones formalism for polarization Elliptically polarized light: Quarter-wave plate Compare quantitative data with quantitative models. Refining idealized models.

Future directions for modeling 1. Investigate how professional scientists use models • To communicate ideas in published journal articles. • In the laboratory to understand, design, and troubleshoot experiments. 2. Develop assessments of students’ modeling proficiency 3. Evaluate different styles of laboratory instruction to see how to best develop modeling abilities. The end. Thank you!