Open Science principles OPEN SCIENCE WORKING GROUP SCHOOL

Open Science principles OPEN SCIENCE WORKING GROUP SCHOOL OF PSYCHOLOGY UNIVERSITY OF ABERDEEN

General principles of open science Transparency! Lockdown methods and analysis strategy before looking at the data ◦ Note: consider the distinction between exploratory and confirmatory research Avoid biases in the analysis and interpretation of data Make data widely available to scientific community Make publications available to the world ◦ See: Munafo, M. R. et al. (2017) A manifesto for reproducible science. Nature Human Behaviour.

Transparency and Openness Promotion Eight principles for journals from the Centre for Open Science ◦ ◦ ◦ ◦ High citation standards Data transparency: data in trusted repository Analytic methods transparency: code/analysis in repository Research materials fully described and available Design of study clear: adhering to reporting standards Study preregistration to lockdown methods Analysis plan preregistration Replication encouraged https: //cos. io/top/

Perspectives from philosophy of science & statistics False paradigms are more likely to persist in low powered disciplines –contrary to Kuhn's ideas of scientific advancement through paradigm change. ◦ Akerlof, G. A. & Michaillat, P. (2018). Persistence of false paradigms in low-power sciences, PNAS. According to contemporary philosophy of statistics, good theory needs to repeatedly pass challenging tests – pre-registration is useful in ensuring that the tests are clear. ◦ Mayo, D. (2010). “Can Scientific Theories be Warranted with Severity? Exchanges with Alan Chalmers” in Error and Inference: Recent Exchanges on Experimental Reasoning, Reliability and the Objectivity and Rationality of Science (D. Mayo and A. Spanos eds. ), Cambridge: Cambridge University Press: 73 -87.

Transparent methods

2018 American Psychological Association guidelines for reporting psychology methods For quantitative research: Appelbaum, M. , Cooper, H. , Kline, R. B. , Mayo-Wilson, E. , Nezu, A. M. , & Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73, 3 -25. For qualitative and mixed methods research: Levitt, H. M. , Bamberg, M. , Creswell, J. W. , Frost, D. M. , Josselson, R. , & Suárez-Orozco, C. (2018). Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73, 26 -46. http: //www. apa. org/news/press/releases/2018/01/journal-article-reporting. aspx

Pre-registration of methods and analysis plans Open Science Framework (OSF) https: //osf. io/ Good for practice and employability. Prefab templates or register methods sections. Cannot completely delete projects once uploaded. ◦ Consider carefully all aspects before uploading the information Ideally upload pre-registration before data collection ◦ But can be done during pilot work and data collection ◦ As long as the data is not analysed before the pre-registration is uploaded. Here is an example of a preregistered project: https: //osf. io/grbd 9/

https: //aspredicted. org/ Another alternative site for pre-registration Series of questions to complete Can delete projects, unlike OSF

Registered reports with specific journals List of journals which publish registered reports: https: //cos. io/rr/ Look under the participating journals tab For most: peer reviewed introduction, hypothesis, methods, analysis. May also be discussion If registered report of methods accepted then the journal has to publish the results, once achieved.

Registered reports Consider carefully when to pre-register and when not. ◦ ◦ Suitable for larger scale or multi-site studies Geared particularly to replication studies Format does not work so well for multi-experiment studies. For most exploratory/pilot research this is not the best route. Here is an example of the final publication from a pre-registered report: ◦ https: //www. sciencedirect. com/science/article/pii/S 0010945216300582

Transparent data

Open data Publishing the dataset along with the paper. ◦ This is now becoming more and more the norm. ◦ Many journals now require authors to send the dataset along with the article submission. ◦ Some journals and funders expect data to be made available. They may have their own specific guidelines and preferences. UKRI common principles on data policy: https: //www. ukri. org/funding/information-for-award-holders/data-policy/common-principles-on-data -policy/ Some general guidelines on preparing data for open access, particularly in OSF: ◦ http: //journals. sagepub. com/doi/full/10. 1177/2515245918757689

Key principles of repositories for data sharing 1. Findable: globally unique and persistent identifiers 2. Accessible: publicly searchable 3. Interoperable: able to exchange across systems without restrictions 4. Reusable Remember to consider GDPR issues, confidentiality and accessibility of the data. University Research Governance advice: ◦ https: //www. abdn. ac. uk/staffnet/research-governance-304. php

Sites for open data provision A selection from many: ◦ ◦ ◦ Open Science Framework: https: //osf. io/ UK Data Service: https: //www. ukdataservice. ac. uk/ Digital curation centre: http: //www. dcc. ac. uk/ UK data archive: http: //www. data-archive. ac. uk/ Zenodo: https: //zenodo. org/ Figshare: https: //figshare. com/ Data can also be made available via the University PURE system.

Transparent materials

Open materials: making experiment programs and stimuli available Open Science Framework can host materials: ◦ Individual files up to 5 GB on OSF with no project limit Copyright/licensing/ownership issues need to be explored carefully, e. g. adaptations of existing materials, co-authorship. Consider the consent of people in social stimuli, and any issues of confidentiality of materials.

Open access publication

Preprint servers to make articles available pre-publication Stage of submission to these? ◦ To get peer feedback before publication ◦ To make publicly available if not published ◦ To make publicly available once published Need to be careful with publisher copyright Can lead to problems with version control Should science be peer-reviewed before being made public? E. g. https: //psyarxiv. com/

Open access publications Making accepted journal articles publicly available. ◦ Green: author can make available content (but not format) of accepted journal article – this is usually free. ◦ Gold: author can make available formatted journal article – generally at a cost (on average > £ 2 K per article). Essential for REF Searchable list of journals and their open access status: http: //www. sherpa. ac. uk/romeo/index. php Link to University info on open access: https: //www. abdn. ac. uk/library/support/open-access 212. php

Creative commons

Adding a creative commons licence https: //creativecommons. org/. This adds copyright to your data or science so that people must tell you that they are using it: Advice available from the University library open access team. The key info for most research publications: https: //creativecommons. org/licenses/by/4. 0/

Links

Useful links 2018 OSF workshop on skills for executing a transparent research project: https: //osf. io/ghz 48/ 2019 meeting of Society for the Improvement of Psychological Science (SIPS): https: //osf. io/ndzpt/ University of Aberdeen open access advice: https: //orwg-aberdeen. github. io/ https: //www. abdn. ac. uk/library/support/open-access-212. php University of Aberdeen contact points: ◦ research. IT@abdn. ac. uk for data storage, formatting etc. ◦ openaccess@abdn. ac. uk for Open Access journal queries ◦ paperaccepted@abdn. ac. uk for final uploads of papers in Pure repository