The Replication Crisis Problems and Solutions Almost Dr

The Replication Crisis: Problems and Solutions (Almost Dr) Priya Silverstein @priyasilverst Slides here: osf. io/ha 4 m 6

What will we be doing today? • What is the replication crisis • My story • Different types of replication • Wh questions • How to run a good replication • Registered Reports • Is replicability enough? Link to your presentation: https: //bit. ly/2 pi. V 3 cr

What is the replication crisis?

Activity: p-hacking exercise

Activity: p-hacking exercise https: //projects. fivethirtyeight. com/p-hacking/ “You’re a social scientist with a hunch: The U. S. economy is affected by whether Republicans or Democrats are in office. Try to show that a connection exists, using real data going back to 1948. For your results to be publishable in an academic journal, you’ll need to prove that they are “statistically significant” by achieving a low enough p-value. ”

My story

My story replication 1: replication + new DV replication 2: replication + new stimuli Silverstein et al. , 2019 replication 3: replication + new condition replication 4: collaborative replication 5: self-replication, new age group replication 3 replication 2 conducted; replication 1 conducted replication 2 published! conducted replication 4 & 5 conducted; replication 3 submitted, revise &

Discussion: replications

Discussion: replications Have you ever run a replication? Note: it might not “feel” like a replication, you might not have intended to do one, but you may have done a “replicate and extend” study for example.

Different types of replication

Activity: find a replication study

Activity: find a replication study Find a replication study in your field. Get to know it. Think about where it lies on the spectrum.

Wh questions

Wh questions Why do a replication? Who can do a replication? When to do a replication? How to do a replication?

Comfort break

How to run a good replication

How to run a good replication • • • Pick an important study to replicate Decide upfront where on the spectrum Make sure you’re well-powered Talk to the original authors If you can, submit it as a Registered Report • If you can’t submit it as a RR, pre-register • Use Bayes Factors or equivalence testing

Discussion: good replications?

Discussion: good replications? Are the replications you found “good replications”? What did they do well? In what ways could they be improved?

Registered Reports

Activity: RR checklist

Activity: RR checklist Go here: https: //bit. ly/2 Xd. JKP 9 Pick a study in your field that you think would be ripe for replication Go through the RR checklist, as if you were planning a RR for a replication of that study

Is replicability enough?

Is replicability enough? Go here: https: //bit. ly/2 CDd. P 13

Discussion: who are we?

Discussion: who are we? Tess: theory tester Mave: the maverick Bo: the boundary tester Rey: the replicator

Presentation organisation