Open Access Bibliometrics a way of demonstrating the

Open Access Bibliometrics: a way of demonstrating the importance of institutional research Niamh Brennan Trinity College Library Dublin JISC RSP Webinar July 3 rd, 2012

Outline • Context: Institutional, National and International • Key Resources 1: high level tools • Key Resources 2: components: task, data, display, story - types of task (what you need to report) - where to get the data [list] - how to display your results [list] - what’s the story? • Institutional Example • Practical Exercise 1: Your institutional data • Practical Exercise 2: Your institutional collaborations • Practical Exercise 3: Your repository and its impact • Resources & Inspiration • Questions

Context: National, Institutional, International

“Higher education operates in the global arena and this is the context in which academic outputs, standards and quality are benchmarked and referenced”. – HEA Landscape document 2012

The Rankings (depending on the particular ranking [QS/THE/ARWU] 20%-60% of the scores are based on publications/citations)

‘SFI’s support for world-class scientific research is contributing to the success in attracting foreign direct investments to Ireland. ‘For example, half of all new project investments won by IDA Ireland last year had a research and development component worth about € 500 million to the economy, ’ said Minister Lenihan. He said the quality and volume of our scientific publications has dramatically improved. ‘In a little over two decades, Ireland has gone from a level below Bangladesh to breaking into the top 20 in the scientific global rankings according to Thomson Reuters Essential Scientific Indicators. ‘And largely on the back of SFI investments in biotechnology over the past decade, Ireland now ranks third in the world in immunology, ’ said Minister Lenihan. http: //www. sfi. ie/news-events/press-releases/sfi-links-349 -firms-with-collegeresearchers-to-create-high-quality-jobs-minister-lenihan-sfi-annual-reportsays-links-between-academia-and-industry-grew-by-25 pc-last-year-ministerlenihan/

Caveats regarding Bibliometrics. . . • • • Limited scope Only part of the picture Never use as the only metric Never compare across fields Practically useless for AHSS Biases in the data. . .

Key Resources 1: High level tools

‘The Value of Libraries for Research & Researchers’ – RLUK/RIN 2011 http: //www. rin. ac. uk/system/files/atta chments/value_of_libraries_for_scre en_1. pdf

RAE: ‘Research Assessment Exercise [UK]; REF: Research Excellence Framework - i. e. ‘Research Information Management Systems’ / CRIS

Benefits to the University ‘The Value of Libraries for Research & Researchers’ – RLUK/RIN 2011 http: //www. rin. ac. uk/system/files/atta chments/value_of_libraries_for_scre en_1. pdf

Research Assessment Data for Higher Education Management ‘Assessing Europe’s University-Based Research’ – European Commission Expert Group on Assessment of University-based Research 2010 http: //arrow. dit. ie/cserrep/17/

‘Assessing Europe’s University-Based Research’ – European Commission Expert Group on Assessment of University-based Research 2010 http: //arrow. dit. ie/cserrep/17/

• Key Resources 2: components: tasks, data, display, story - types of task (what you need to report) - where to get the data [list] - how to display your results [list] - what’s the story?

• Key Resources 2: components: tasks, data, display, story - types of task (what you need to report)

Fundamental Tasks • • Show the research outputs of your university: over time, by publication type, by academic unit, per staff FTE, by research priority area, benchmarked [etc. ] Show the scholarly impact of the institution’s research: over time, by academic unit, by number of citations, by impact (citations per paper), per staff FTE, benchmarked [etc. ] Show the national picture in terms of research impact – and your institution’s contribution to that impact Show your institution’s research collaborations in terms of co-authorship (broken down by all of the above)

Types of reports • • • Annual report (Dean of Research) Automated ‘On-demand’ reports – per School Director of Research /Head of School reports Rankings data (supply, review, report) Bespoke reports: § School research audits § Institutional reviews (external, disciplinebased) § Reports to Government agencies § Input to quality reviews § Individual researcher bibliometric ‘health check’ (on demand, time permitting) § Reports on collaborations – Vice Provost for Global Relations

Outline • Key Resources 2: components: tasks, data, display, story - where to get the data [list]

Components – Standard • • • Number of research outputs (peer reviewed journal articles, peer reviewed conference papers, Books, book chapters) Research productivity report: % of staff who are ‘research productive’ (local definition) Bibliometric report: citations & impact over time (10 years / 5 years) – data for TCD Collaborations: by country, by institution Significant research (papers in top percentiles globally etc. / people with significant research – ‘hot’ papers etc. Related to: Key Performance Indicators / local ‘Research Quality Metrics’

Components - Occasional • • Bibliometric report: citations & impact over time (10 years / 5 years) – data for TCD benchmarked against other comparative institutions Journal analysis – with impact factor Funder analysis H-index – per researcher / academic unit Citations analysis – by institution, country, journal Capacity report – in depth analytics, impact analysis & benchmarking Open Access report – to university, funders (supporting mandates)

Data • • CRIS system + Open Access Repository (ideally integrated with one another) Range of commercial & non-commercial tools for bibliometric analysis § Thomson Reuters Resources (Web of Science, Incites, Essential Science Indicators) § Elsevier Resources: Sci. Val & Sci. Verse (Scopus, Strata etc. ) § Google Scholar Citations, Publish or Perish, Altmetrics (etc. ) § Google Analytics! • People!

Excellent guide to Altmetrics & Co. http: //www. surffoundation. nl/nl/publicaties/Docume nts/Users%20 narcissism%20 and%20 control. pdf

CURRENT RESEARCH INFORMATION SYSTEM - CV-driven - Fully integrated with complementary systems - initial population from Human Resources records - updated & enhanced ‘live’ by researcher, - mediated input on-demand (Library-based service)

Bibliographic records: - Started with our institutional Citation Report from ISI - Now: APIs available from Scopus, Web of Science, Pub. Med, Others - Specifically requested by researchers

- Fully integrated with Research Support System - Added value to records in both systems • • Research publications Etheses Epublishing/Grey Literature Images

Repository with Added Value Link to Researcher’s Profile Citation: Published Version of the Paper Link to Published Version of the Paper

• Key Resources 2: components: tasks, data, display, story - how to display your results [list]

Tools I use a lot (See Resources at end for more) • • • Google Charts Excel Charts Google Fusion Tables Many Eyes Picture Number Open Heat Map Occasionally: • Tableau Public • Xlnode • VUE

Understanding our Research Outputs: Reports from the CRIS

Open Access TARA deposits from the Research Support System

TARA Deposits from the RSS: Publication types TARA Deposits from the RSS: Peer Reviewed vs Non Peer Reviewed 6% 94%

Compliance with Open Access Policies & Mandates 1600 1400 TCD OA Policy October 2010 1200 1000 800 TCD journal articles in TARA TCD Wo. S journal articles 600 400 200 0 450 400 Science Foundation Ireland OA Mandate 350 300 250 SFI-funded Articles on Open Access via TARA 200 SFI-funded Artices in Web of Science 150 100 50 0

TCD has had multiple collaborations with Brazilian researchers in at least 67 Brazilian research institutions Map shows locations of Brazilian research Institutions who have had at least one research collaboration with TCD. An interactive online version (with institutional names) is available at this link: https: //www. google. com/fusio ntables/Data. Source? snapid= S 551851 icn m

TCD-Brazilian Collaborations: Overview Number of Publications in Web of Science: Number of Citations: Impact (citations per paper): H-index: 132 3, 051 23. 11 24

Outline • Key Resources 2: components: task, data, display, story - what’s the story?

Data should tell a story. . . Stories need change or revelation Trends Change over time Before and after Us and them Something new Impact: more, greater, better, deeper, Results: objectives & outcomes

• Institutional Example

RISS CASE STUDY: New Research Institute Team effort: The Library Contribution 2005 -6 External review of the Discipline in TCD CRIS & TARA (Repository) development: reports on Publications, Esteem, Bibliometrics; Repository used for reviewer access to key papers 2009 -10 Funding Proposal input Systems support for submission of grant proposal: CRIS autocapture of people & publications data, reporting workflow; bibliometric & impact reports 2011 Additional Fundraising Support Supply of data for information/marketing purposes: on people, publications, research strengths, impact 2012 Reports & Evaluat -ions Ongoing supply of publications and bibliometric/impact data for quarterly funder reports, TARA support for funder OA mandate

RISS Case Study Trinity College Dublin Publications Citations Relative Impact Collaborations Citations Analysis World Class Researcher Profiles Rankings Research Fronts Competencies / Emerging Competencies Innovation & Knowledge Transfer Industry collaboration Top % in World Rankings

RISS Case Study: Trinity College Dublin Data Sources & Resources Used ² TCD Integrated Research Systems [CRIS & Repository] ² Web of Knowledge [Wo. S, ESI] ² In. Cites [Global Comparisons] ² Patents analysis, patent citation databases ² Technology transfer & industry liaison information (internal) ² Institutional reports ² Government reports ² International reports (OECD, World Bank, EU etc. ) ² Media reports including scientific journal news ² Times Higher ‘Top Nations’ ² Science. Watch [Thomson Reuters – free] ² Google Analytics & Dspace statistics

• Practical Exercise 1: Your institutional data Possible approach: Get number of publications data for past 5 years from CRIS, Web of Science, Scopus Use charts from resource (CRIS, Wo. S, Scopus), or Output to. csv Visualise using Excel Charts, Google Charts etc. Get number of citations data for past 5 (or 10) years from Web of Science, Scopus, etc. Use charts from resource (eg Wo. S, Scopus), or Output to. csv Visualise using Excel Charts, Google Charts etc.

• Practical Exercise 2: Your institutional collaborations Possible approach: Get collaborations data for past 5 years from Web of Science or Scopus a)By country Output to. csv Visualise using Excel Charts, Google Charts etc. Map: upload collaborations by country to Google Fusion Tables and/or Open Heat Map Take screenshot, share and/or embed the map/s Added story value: repeat for previous 5 years; present comparison a)By institution Output to. csv Visualise using Excel Charts, Google Charts etc. , or: Create table of ‘Top 20 Institutional Collaborators’ Added story value: repeat for previous 5 years; present comparison

• Practical Exercise 3: Your repository and its impact Possible approach: % of institutional peer reviewed papers in repository: Get breakdown of content data from your repository: Number of peer reviewed journal articles deposits per year of publication Compare with Web of Science/Scopus/CRIS publications data Use Excel & Excel Charts (bar charts) or equivalent to display Repository content by School/Field: Get numbers of publications per School/Unit from Repository Display on pie-chart (Excel Charts/Google charts etc. ) Google Analytics: Get data to show hits over time (output to csv & display) Get demographic data: geographic location (output to csv & map) Get domain data; output to csv; analyse by: academic institution, commercial entity, . gov organisations, cultural/media institutions. Display Warning: manual labour required! Not clean or comprehensive – but shows clear impact and usage

Notes on Esteem & REF “‘Esteem’ is no longer included as a distinct element in the assessment” 67. All information provided by HEIs in submissions to the 2014 REF must be capable of verification [note: can CERIF help provide the reference or source of data? ] Under ‘Definition of Impact for the REF’: “a. Impacts on research or the advancement of academic knowledge within the higher education sector (whether in the UK or internationally) are excluded. (The submitted unit’s contribution to academic research and knowledge is assessed within the ‘outputs’ and ‘environment’ elements of REF. )” Part 3 Section 5: Environment template (REF 5) 183. Information is required about the research environment for each submitting unit relating to the period 1 January 2008 to 31 July 2013. Each submission must include a single completed REF 5 form, consisting of the following sections: • Overview. • Research strategy. • People, including: – Staffing strategy and staff development. – Research students. • Income, infrastructure and facilities. • Collaboration and contribution to the discipline. REF 5 ‘Environment’: • Maximum score reduced from 30% (RAE 2008) to 15% (REF 2014) From: Assessment framework and guidance on submissions, HEFCE, July 2011 • Indicators of Esteem = http: //www. hefce. ac. uk/research/ref/pubs/2011/02_11. pdf. Collaboration & Contribution to the Discipline

Your mission. . . Can bibliometrics (in the broadest sense) help provide at least some of the data required on your institutional REF submission (section REF 5)? i. e. • Your institutional collaborations • Your researchers ‘contribution to discipline’

Outline • Resources & Inspiration

Guardian Data Blog

Guardian Data Blog: Top Tools http: //www. guardian. co. uk/news/datablog/ 2012/mar/28/data-visualisation-tools-free

http: //dailytekk. com/2012/02/27/over-100 -incredible-infographic-toolsand-resources/

Inspiration • • • Information Aesthetics [blog] Information is Beautiful [book] Gavin Sheridan: The. Story. ie (data journalism) Presentation at Media Lab Prado [Google Fusion, Google Refine etc. ] http: //medialab-prado. es/mmedia/6350/view/? lang=en

Open Access Questions? Niamh Brennan Trinity College Library Phone: Email: +353 1 896 1646 niamh. brennan@tcd. ie