Measuring and promoting research impacts using alternative metrics

Measuring and promoting research impacts using alternative metrics as a means towards SMART research in African Universities By Malekani, A. W (Ph. D) Sokoine University of Agriculture (SUA, Sokoine National Agricultural Library (SNAL), P. O Box 3022, Morogoro, Tanzania E-mail: malekani@sua. ac. tz Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities

Outline: § § Introduction Methodology of the study Global presence of African research output Measuring scholarly output – traditional methods – alternative methods § Altmetrics – What is Altmetrics; what are the advantages of altmetrics? What is measured? Who is doing it? • Plum. Analytics, Inside Story, PLOS, KUDOS, Altmetric. com Carnegie CPD programme – Controversy / limitations/conclusions Enhancing librarians' ICT skills for research enablement in African universities 2

Introduction LIS workplace and profession has changed • Then… • Now… Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 3

…and so has the sources…. Then… Now… Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 4

And so has the library user…. Then… Now… Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 5

Introduction • Quantifying scholarly output via traditional citation metrics is the time-honoured method to gauge academic success. • These methods are used by Universities for promotion purposes, etc. • Assessing the impact of a scholar’s work can be measured by evaluating several factors including the number of peer-reviewed publications, citations to these publications, and the influence of the publications. • However, these metrics take a relatively long time to accumulate. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 6

Introduction cont… § This paper outlines alternative metrics for measuring research output as opposed to traditional citation metrics like impact factors (IF) and Hindex which are what a lot of researchers are privy to. § It presents the “alternative metrics” (altmetrics) that can be promoted by LIS profession that measure the impact of a publication by looking at several factors like the number of times the paper has been downloaded, viewed, shared, liked and mentioned on social media. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 7

Methodology of the study § The paper uses mainly literature review as a method for the study. § The paper highlights first global presence of African research output and the traditional metrics and their limitations and then explores the alternative metrics that researchers should opt to publish their research articles for Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 8

Global presence research output of African § In 2006, there was low global presence of African research output. § According to Dick (2013) and Schmaus (2008), Sub. Saharan Africa contributed 0. 7% to world science output. § There was not just a challenge of raising the quality and quantity of scientific output in Africa, but also of raising the visibility of research in Africa. § At the same time, 20% of African researchers carried out most of their research outside the walls of academia and traditional research bodies, usually for international organisations, foreign intergovernmental bodies and NGOs. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 9

§ From 1996 to 2012, number of research papers in scientific journals with at least one African author quadrupled (12, 500 to 52, 000). § For the same period, the share of world’s articles with African authors doubled (1. 2% to 2. 3%). § The continent’s share of global research has increased from 2. 31% in 2010 to 3. 24% in 2016 (Green, 2018). § The reasons for emerging on world stage are attributed Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 10

African research output and global share of articles, December 2013 (Source: Scopus) Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 11

§ And there is still room for improvement!!! HOW? § According to Schemm (2013), while African authors have nearly doubled their article share over the past decade, the returns could be many times greater over the next decade if awareness, usage and research capacity are tackled in a collaborative and integrated manner by African institutions, access programmes and publishers. § Above all, with the use of altmetrics, the quantity, quality + impact may increase tremendously beyond the present level. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 12

Traditional methods § There a number of traditional methods that the scientific community uses to measure academic success. § These assessments are tied to the scholarly publishing process and the peer review procedure. § Some of the traditional methods to gauge academic success are tied to the journal title itself. § The first, and most well-known of these metrics, is the Impact Factor. The Impact Factor “is a measure of the frequency with which the "average article" in a journal has been cited in a particular year or period” (Thomson Reuters 2013) Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 13

§ To calculate the Impact Factor (IF), Thomson Reuters averages the number of times articles from the journal in the past two years have been cited in the past year. § This information is available through their Journal Citation Reports (Thomson Reuters 2012). § Because it is a standard metric, it is frequently discussed as the only metric to evaluate journal quality. § It is important to recognize that the only titles that are included in the Journal Citation Reports and given an IF are titles indexed by the Web of Knowledge database. § Not only does this exclude some journal titles, it also excludes book chapters and conference proceedings, which are important scholarly output in some disciplines (Cameron 2005). Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 14

§ Additional methods of gauging academic success focus on individual articles rather than the journal in which an article is published § Simply counting the number of later articles that cite the original article is one of those metrics. § While the metric is not complicated, gathering the data to count the citations is. § Both Scopus and Web of Knowledge databases include citation linking, and with that linked data one can create reports of the number of times an article, or all articles by an author, have been cited. § As discussed earlier, neither set of data is complete. Google Scholar also includes a “cited by” count in its display of an individual article. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 15

§ That information can be collected for a single author by using Google Scholar Citations, which allows an author to create a profile displaying various citation indices. § Criticisms of Google Scholar’s citation counting. – The numbers may be inflated if citations to articles are counted twice; for example, if the citing paper is published in a journal and also posted to a researcher’s website. – Google Scholar does cover many more conference papers and book chapters than either Scopus or Web of Knowledge (Jacso 2006). Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 16

§ The h-index is another method for evaluating the scholarly output of a researcher. § This metric does not just evaluate one journal or one article but tries to give a broader view of the impact of a researcher’s work. § The h-index is the number of a scholar’s papers, h, that have been cited at least h times by other publications (Hirsch 2005). § The h-index is frequently represented on a graph with the number of papers along the x axis and the number of citations along the y axis. H is found where the papers equal the number of citations (Figure 2). Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 17

Figure 2: the h-index Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 18

§ While the metrics described above capture a work’s influence in a particular discipline, this narrow approach to gauging impact discounts the many new ways in which users engage with scholarly information. § As scholars communicate more frequently and more meaningfully with social media tools, the importance of quantifying these interactions is evident. § For as long as scholarly output has been measured, scholars have thought about ways to increase their impact. § One way to do so is by maintaining an online presence and by active involvement in social media. § These forms of engagement offer scholars a method to disseminate their research, increase their professional profile, and communicate their findings with the public. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 19

From Bibliometrics to Altmetrics Whole Internet, cyberspace Informetrics Bibliographies – largely references Bibliometrics Cybermetrics Web presence, visibility and impact – links, pages, documents Science of Science Scientometrics Altmetrics Webometrics Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities Alternative metrics – views, downloads, web citations, etc

Altmetrics: § Altmetrics are new metrics proposed as an alternative to the widely used journal impact factor and personal citation indices like the h-index. § The term altmetrics was proposed in 2010, as a generalization of article level metrics. (not alternative metrics – although many authors now refer to it as such…) § Although altmetrics are often thought of as metrics about articles, they can be applied to people, journals, books, data sets, presentations, videos, source code repositories, web pages, etc. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 21

Altmetrics: § Altmetrics cover not just citation counts, but also other aspects of the impact of a work, such as how many data and knowledge bases refer to it, article views, downloads, or mentions in social media and news media § Various websites and projects are calculating altmetrics, including Impact. Story, Altmetric. com, Plum Analytics, and Cited. In. § Several publishers have started providing such information to readers / authors, including Bio. Med Central, Public Library of Science, Frontiers, Nature Publishing Group, and Elsevier. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 22

Advantages of altmetrics § There are four potential advantages according to Piwowar (2013): -a more in depth understanding of impact, showing us which scholarly products are read, discussed, saved and recommended as well as cited; -often more timely data, showing evidence of impact in days instead of years; -a window on the impact of web-native scholarly products like datasets, software, blog posts, videos and more; -indications of impacts on diverse audiences including scholars but also practitioners, clinicians, educators and the general public. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 23

What is measured? How many times an output – article, website, blog, dataset, grey literature, software, etc has been: – Viewed (Publisher websites, Dryad) – Downloaded (Slideshare, publisher websites, Dryad) – Cited (Pub. Med, Cross. Ref, Scopus, Wikipedia, DOI, Web of Science) – Reused/Adapted (Github) – Shared (Facebook, Twitter) – Bookmarked (Mendeley, Cite. ULike, Delicious) – Commented upon (Twitter, Mendeley, blogs, publisher websites, Wikipedia, Faculty of 1000) Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 24

Alt. Metrics - Who Does It Plum Analytics - Service provider with API. Profile page and analytics. Say they cover more Metric sources than anyone else Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities

Alt. Metrics - Who Does It Impact Story - Displays data in a CV like format. Could become the Linked-In for Scientists. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities

Impactstory. org Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 27

Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 28

Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 29

Alt. Metrics - Who Does It PLOS - One of the first publishers to use article level metrics. One of the current leaders. Metrics i. iii. iv. Views Cited Saved Discussed Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities

Alt. Metrics - Who Does It Altmetric. com - Service provider for publishers, bibliographic websites and more. also offer bookmarklet. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities

Alt. Metrics - Who Does It Institutional repositories – An online archive for collecting, preserving, and disseminating digital copies of the intellectual output of an institution, particularly a research institution Metrics - Total views - Country views - Cities views - Trend of views Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities

Controversy / limitations: § The usefulness of metrics for estimating impact is controversial, but the community shows a clear need: – funders demand measurables on the impact of their spending. § Like other metrics, altmetrics are prone to self-citation, gaming, and other mechanisms to boost one's apparent impact. § However, it should be kept in mind that the metrics are only one of the outcomes of tracking how research is used. § Even more informative than knowing how often a paper is cited, is which papers are citing it. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 33

Controversy / limitations: § Altmetrics can be gamed: for example, likes and mentions can be bought. § Altmetrics can be more difficult to standardize than citations. One example is the number of tweets linking to a paper where the number can vary widely depending on how the tweets are collected. § Another source of objections against altmetrics, or any metrics, is how universities are using metrics to rank their employees. § The score on each field does not directly tell you anything about the quality or impact of the paper. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 34

Controversy / limitations: § Still more valuable sources to cover - classes, seminars, lectures, posters, journal clubs and more. § Aggregation is difficult - data in many formats, stored in different ways, in many places, some not accessible (NISO standard being developed). § Author names are not standard and multiple copies of media under different names - ORCID, Data One, Bio. Share Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 35

Conclusions Alternative metrics or Altmetrics (besides usage statistics and citation counts) can be used as a value -added service to showcase content impact and give some hints on how this tool can be used to supplement the traditional research performance assessment exercise in an institution. Altmetrics can provide a measure of impact for all non-journal scholarly works available in Open Access (OA), like usage statistics, but it can go further by contextualizing the readership of an author’s research output. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities