The Replication Crisis Problems and Solutions Almost Dr
The Replication Crisis: Problems and Solutions (Almost Dr) Priya Silverstein @priyasilverst Slides here: osf. io/ha 4 m 6
What will we be doing today? • What is the replication crisis • My story • Different types of replication • Wh questions • How to run a good replication • Registered Reports • Is replicability enough? Link to your presentation: https: //bit. ly/2 pi. V 3 cr
What is the replication crisis?
What is the replication crisis?
Activity: p-hacking exercise
Activity: p-hacking exercise https: //projects. fivethirtyeight. com/p-hacking/ “You’re a social scientist with a hunch: The U. S. economy is affected by whether Republicans or Democrats are in office. Try to show that a connection exists, using real data going back to 1948. For your results to be publishable in an academic journal, you’ll need to prove that they are “statistically significant” by achieving a low enough p-value. ”
My story
My story replication 1: replication + new DV replication 2: replication + new stimuli Silverstein et al. , 2019 replication 3: replication + new condition replication 4: collaborative replication 5: self-replication, new age group replication 3 replication 2 conducted; replication 1 conducted replication 2 published! conducted replication 4 & 5 conducted; replication 3 submitted, revise &
Discussion: replications
Discussion: replications Have you ever run a replication? Note: it might not “feel” like a replication, you might not have intended to do one, but you may have done a “replicate and extend” study for example.
Different types of replication
Different types of replication
Activity: find a replication study
Activity: find a replication study Find a replication study in your field. Get to know it. Think about where it lies on the spectrum.
Wh questions
Wh questions Why do a replication? Who can do a replication? When to do a replication? How to do a replication?
Comfort break
How to run a good replication
How to run a good replication • • • Pick an important study to replicate Decide upfront where on the spectrum Make sure you’re well-powered Talk to the original authors If you can, submit it as a Registered Report • If you can’t submit it as a RR, pre-register • Use Bayes Factors or equivalence testing
Discussion: good replications?
Discussion: good replications? Are the replications you found “good replications”? What did they do well? In what ways could they be improved?
Registered Reports
Registered Reports
Activity: RR checklist
Activity: RR checklist Go here: https: //bit. ly/2 Xd. JKP 9 Pick a study in your field that you think would be ripe for replication Go through the RR checklist, as if you were planning a RR for a replication of that study
Is replicability enough?
Is replicability enough? Go here: https: //bit. ly/2 CDd. P 13
Discussion: who are we?
Discussion: who are we? Tess: theory tester Mave: the maverick Bo: the boundary tester Rey: the replicator
Presentation organisation
- Slides: 30