Open Science in Psychology Workshop hosted by Sabanci
Open Science in Psychology Workshop hosted by Sabanci University, Istanbul, Turkey Slides available here: https: //osf. io/kqdc 4/ Lorne Campbell University of Western Ontario
Open Science Resources • A lot has been developed in a short period of time Improving Science in Individual Labs
Why Should Science Be Open? • Evaluation requires understanding the methods used, and in what context • Reproducibility • Replicability
Canadian Government Commitment to Open Science • Commitment 14: Increase Openness of Federal Science Activities (Open Science) – The Government of Canada will take appropriate steps to make the science performed in support of Government of Canada programs and decision-making open and transparent to Canadians. • …the Government of Canada wants to build on past work by taking bold steps to make government-funded* science open and transparent to Canadians – * such as by the tri-councils, or the agencies that fund most research in Canada
The Transparency and Openness Promotion Guidelines (TOP) • Eight modular standards, each with three levels of increasing stringency for openness • Signatories – 757 journals – 63 organizations • Many psychology journals are signatories – E. g. , JESP, JSPR, Perspectives, Social Psychology, PLo. S One, Frontiers, Collabra
The Research Process Ask a Question Review Literature Formulate Hypothesis Design/Run Study Make Inferences Report Results
The Research Process: Status Quo Public Ask a Question Review Literature Formulate Hypothesis Private Design/Run Study Make Inferences Report (positive) Results
The Research Process: Open Science Public Ask a Question Review Literature Formulate Hypothesis Design/Run Study Documenting the Research Workflow Make Inferences Report (all) Results
Doing Open Science • We receive a lot of training on research methods and statistical procedures (but likely not enough —another talk!) • But, not much (if any) on how to do open science (Feynman, 1974, and Lykken, 1991 made similar arguments) • Technology today allows for open science practices
What to do?
Challenges, not Roadblocks • “Well I tell them there’s no problem, only solutions”
The Open Research Process Ask a Question Review Literature Open Notebook Formulate Hypothesis Design/Run Study Make Inferences Report Results
The Open Research Process Pre-registration • • Ask a Question Review Literature Open Notebook OSF Aspredicted. org Formulate Hypothesis Design/Run Study Make Inferences Report Results
The Open Research Process Pre-registration • • Ask a Question Review Literature Open Notebook OSF Aspredicted. org Registered Reports Formulate Hypothesis Design/Run Study Make Inferences Report Results
The Open Research Process Pre-registration • • Ask a Question Review Literature Open Notebook OSF Aspredicted. org Registered Reports Formulate Hypothesis Design/Run Study Register study design, procedure, materials, data analytic plan Make Inferences Report Results
The Open Research Process Pre-registration • • Ask a Question Review Literature Open Notebook OSF Aspredicted. org Registered Reports Formulate Hypothesis Design/Run Study Register study design, procedure, materials, data analytic plan Data Management Plan Make Inferences Report Results
The Open Research Process Confirmatory and Exploratory analyses Pre-registration • • Ask a Question Review Literature Open Notebook OSF Aspredicted. org Registered Reports Formulate Hypothesis Design/Run Study Register study design, procedure, materials, data analytic plan Data Management Plan Make Inferences Report Results
The Open Research Process Confirmatory and Exploratory analyses Pre-registration • • Ask a Question Review Literature Open Notebook OSF Aspredicted. org Registered Reports Formulate Hypothesis Design/Run Study Register study design, procedure, materials, data analytic plan Data Management Plan Make Inferences Report Results Preprint Server • • Psyarxiv Socarxiv
The Open Research Process Confirmatory and Exploratory analyses Pre-registration • • Ask a Question Review Literature Open Notebook OSF Aspredicted. org Registered Reports Formulate Hypothesis Design/Run Study Register study design, procedure, materials, data analytic plan Data Management Plan Make Inferences Report Results Preprint Server • • Psyarxiv Socarxiv Journal • • Traditional Open Access
The Open Research Process Confirmatory and Exploratory analyses Pre-registration • • Ask a Question Review Literature Open Notebook OSF Aspredicted. org Registered Reports Formulate Hypothesis Design/Run Study Register study design, procedure, materials, data analytic plan Data Management Plan Make Inferences Share data, Code, and Meta-data Report Results Preprint Server • • Psyarxiv Socarxiv Journal • • Traditional Open Access
Intermission
Some Open Science Key Terms • Registration • Pre-registration
Key Terms for Using the Open Science Framework (OSF) Types of Pages • Project – dynamic • Registration – fixed Can be: • Private or Public And can contain: • Components – Nested within higher order components and projects
Key Terms for Using the Open Science Framework (OSF) • Version Control • Wiki, commenting • Licensing and DOIs • Compatibility with other websites and/or apps – E. g. , Dropbox, Google Drive, Github, Figshare, Dataverse, Box
Example: How My Lab Uses the OSF to Organize our Open Science
Disclosure Statements • • • Study Rationale & Hypotheses Methods, Procedures and Study Scales Data Analytic Plan Participant Recruitment Plan (if applicable) Post-Analytic Discussion
Sharing Analytic Code
Sharing Analytic Code
How to use the OSF • 1) create an account: https: //osf. io
How to use the OSF • 2) Create a new project
Let’s not Fool Ourselves
New Project: Are we Scared More in the Dark?
Example: Using the Wiki
Example: Using the “Storage”
Example of Version Control
Different Types of “Contributors” Add colleagues for different projects Use “view only” links, including sharing anonymous links for reviewers
- Slides: 42