SISOB CONCEPTUAL MODEL Richard Walker May 30 2011

SISOB CONCEPTUAL MODEL Richard Walker May 30, 2011

Overview • • • Goals General approach Entities Operationalizing model D 2. 2 Table of Contents

Goals • Common vocabulary and approach • Homogeneous approach to case studies

General approach • Inspired by computer science • Abstract classes for networks, actors and outcomes • Each class has attributes • Real actors in case studies are instances of abstract classes • Measurements give value to attributes

Entities • Production – Actors – Networks – Context • Distribution – Actors – Networks – Context • Consumption – Actors – Networks – Context • Outcomes – Scientific – Economic – Social

Operationalizing model • What are the questions I want to ask? • What are the operational entities relevant to my model? – Example • Who are the production actors? • What are the production networks? • What are my data sources? – Do I already have access to the data I need? – Do I need crawling / data from other partners • How do I characterize my entities using my data sources? – Example • What measurements do I use to characterize networks

D 2. 2 Table of Contents • • • • • • • Objectives and structure of this document 2 The Impact of science on society (Frontiers with contributions from all partners ) 3 The SISOB conceptual mode 3. 1 Goals of the model (Frontiers with input from all partner) 3. 2 Overview 3. 3 Entities in the model (Frontiers UDE and ELTE for measurements and UM for tools, all partners) 4 Operationalizing the model – Researcher Mobility (Unito) 4. 1 Background 4. 2 Goals and hypotheses of the case study 4. 3 The model – an overview 4. 4 Model entities 5 Operationalizing the model – Knowledge Sharing (UDE) 5. 1 Background 5. 2 Goals and hypotheses of the case study 5. 3 The model – an overview 5. 4 Model entities 6 Operationalizing the model – Literature review (Frontiers) 6. 1 Background 6. 2 Goals and hypotheses of the case study 6. 3 The model – an overview 6. 4 Model entities 6. 4. 1 Appendix 1: Requirements on SISOB tools Summary of required measurement tools REFERENCES Appendix A: Common Network Indicators 1

PEER REVIEW – DATA REQUIREMENTS Richard Walker

Overview • • Scientific questions Operationalization of conceptual model Sample hypotheses Data sources required

Scientific questions • How does peer review affect impact of science • “Traditional” issues – Cognitive biases – Traditional cronyism – “Cognitive cronyism” • “Social” issues – How do relationships among reviewers affect review process? – How do relationships among reviewers and authors affect the review process? • “New” issue – How do new models of review affect review process?

Operationalizing the conceptual model Conceptual model Operational model Production actors Authors Production network Coauthoring network Shared institution network Distribution actors Reviewers Distribution network Co-reviewer network Reviewer-editor networks Author-reviewer-editor networks Artifacts Publications Outcomes Acceptance of publication Post-publication review by readers Academic citations Non-academic citations Productivity (individual, community)

Sample hypotheses • Traditional – Papers with a woman as a first author are more likely to be accepted review committee includes a woman • Social – Papers are more likely to be accepted if authors are “close” to reviewers in author-reviewer network (cognitive cronyism) • Different techniques of reviewing – Open reviewing (Frontiers) is less affected by bias x than traditional reviewing

Data sets required/1 • Frontiers papers – Attributes of papers, authors, reviewers • Source: Frontiers • Use: reconstruct author and reviewer networks – All papers by Frontiers authors and reviewers (last 10 years) • Source: crawling • Use: enhance author and reviewer networks – All citations of papers in Frontiers data set • Source: crawling • Use: outcome measurement – Productivity of authors and reviewers • Measirement: number, citations, outside references • Source: crawling • Use: outcome measurement – Non-academic citations of papers in Frontiers data set • Source: crawling • Use: outcome measurement

Data sets required • Conference papers (UMA) – Attributes of papers, authors, reviewers • Source: ? ? • Use: reconstruct author and reviewer networks – All papers by authors and reviewers (last 10 years) • Source: crawling • Use: enhance author and reviewer networks – All citations of papers in data set • Source: crawling • Use: outcome measurement – Productivity of authors and reviewers • Measirement: number, citations, outside references • Source: crawling • Use: outcome measurement – Non-academic citations of papers in data set • Source: crawling • Use: outcome measurement

Open issues • Availability of data sources • Network indicators • BIG ISSUE 1 – how do we make this useful for policy makers? • BIG ISSUE 2 – how do look this in a community perspective