Multiple Bibliometric Indicators Approach to Croatian Open Access
Multiple Bibliometric Indicators Approach to Croatian Open Access (OA) Journals Franjo Pehar, Jadranka Stojanovski Department of Information Sciences, University of Zadar Libraries in the Digital Age 2014
“Better to understand a little than to misunderstand a lot. ” Anonymous writer
Rise of metrics ● importance and usage of citation metrics has increased substantially in recent years ● the most popular metric indicator is still JIF ●. . . initially intended as a way to help libraries to select the journals to be acquired… ●. . . and to help academics to ‘filter content’ ● used to measure academic achievement o at individual, project (grant) and institutional level
. . . ● ● ● academic career research project assessment institution ranking funding decisions … ● new situation has drawn significant criticism o practices that encourage conformity in research o “salami-slicing” or incremental publishing o unequal treatment of researchers from low-cited disciplines
“JIF is not created for all the purposes that some people want to use them for. A lot of people who are using JIF are doing it for advertising purposes. ” “The real problem is when you start using JIF as the substitute for citation analysis of individuals. I have always said that you should not use JIF to evaluate a person or department. If I manage to get my paper published in Nature, does that guarantee that it is a great paper? Even if the paper is never cited? ” “It is crucial to know how citations are connected, how they are related. ” http: //www. garfield. library. upenn. edu/ papers/medicalwritingv 8(1)1999. html Eugene Garfield at LIDA 2004
http: //researchimpact. files. wordpress. com/2011/10/research-impact-cartoon. jpg
What is a citation? ● simple line in the reference list (not) pointing to the paper ● paper which act as a foundation for our own work ● paper which help us to broaden our knowledge ● tool to confirm that we prepared well prior to our own research ● tool to flatter someone ● tool to “self-cite” at different levels. . .
Citation as a metric tool ● many factors can influence the rate of citation o age of article, type of research conducted, the subject area o objectivity of citations? real benefit? (erroneous references with hundreds, thousand citations) ● tools for evaluation and assessment are evolving constantly, giving rise to new metrics o different aspects of citation and publication behaviour
Journals ● ● fee and free journals „OA journal” as a synonym for APC model journals? - or more portals very prominent journals = high JIF journals = very for-profit journals ● „international journals” and „local journals” ● „regional journals”
Journals ● journal as an old concept from the printed world ● journal as (still) main channel of scholarly communication ● journals slowing down a possible development of science (? ), not implementing ICT advantages and possibilities… ●. . . or implementing it at very slow pace
What’s about journals from the scientific semi-periphery?
Narrower context ● Croatia o 4. 5 M population, 11 k scholars, 7 universities, 25 research institutes, 25 k papers per year and 1300 islands ● Croatian journals o challenges for small language and semi-peripheral science context
Croatian journals in numbers
Academic career policies in Croatia “papers must be published in journals with JIF above average for the specific subject category” “at least two-thirds of the papers must be published in the journals listed in SCIE” “paper is rated regarding ratio of the JIF and median of the subject category, and the minimal value must be 0. 5 “the value of the qualitative criteria Q is calculated as a sum of the ratios of JIF and median JIF of the subject category, multiplied by the factor Fd of the candidate” “Example: Journal of Engineering Mechanics in 2011 has JIF 0. 99, subject category is Engineering – Mechanical with median JIF 0. 743 …” “in the A 1 category are the papers published in journals with JIF in Q 1, Q 2, i Q 3 of the subject category. . . in the A 2 category are papers from the journals in Q 4…” “in the A 1 category are the papers published in journals indexed by Web of Knowledge. . . if the JIF falls in the first 50% of the subject category. . . then the paper is counted as two papers”
Hmmmm…. are Croatian journals important? is it reasonable just to transfer the Croatian journal funds to APC or fee based prominent and popular “international” journals? what justification do we have to publish Croatian scholarly journals?
HRCAK - improved visibility, readability. . . popularity. . . citations
HRCAK by disciplines
OBJECTIVE OF THE STUDY ● Which aspects of journal usage metrics differ from citations metrics ● In particular, following issues were be addressed: Croatian OA journals usage Relationship of full-text paper downloads and citation counts o Comparison between full paper downloads and citations counts relative to different subject areas o o
MATERIALS AND METHODS ● HRCAK usage statistics o o Cover journal's web page visits Issue web page visits Article (bibliographic records) web page visits Full-text (PDF) downloads o Unique visits from the same IP address during the period of 30 minutes
. . . ● For the purpose of the study o Usage data for 115, 458 full-text papers (documents) acquired from 340 unique OA journal titles § March 2007 to the May 1 st 2014 § Article page visits § Full-text downloads
674 downloads 1293 visits
Research sample ● 47 OA journals (28, 121 papers) available in HRCAK repository and indexed by o Web of Science™ Core Collection (Thomson Reuters) § 28, 725 indexed papers o Scopus (Elsevier) § 28, 953 indexed papers
Preprocessing challenges ● Assembling a single data source based on three data sets o lack of (P)ID’s for matching documents from different data sets ● Titles were reduced to the same form ● Levenshtein distance algorithm to determine similarity between paper titles (2 iterations) 15, 023 (53%) papers in Web of Science and 16, 592 (57%) papers in Scopus matched with HRCAK titles o 13, 127 (ca. 46%) unique papers merged from all three data sets o ● 2 journals dropped-out
. . . ● Initial cleaning of usage data o validity, reliability, transparency. . . ● Missing data o HRCAK page visit & download dates/years, more reader demo-geographic information ● Matching/merging procedure o o two journal titles completely lost further improvements possible > expected matching rate ca. 85% (cat(ti. . . jn … is … vl … pg)
RESULTS ● download and citation frequencies (March 2007 - May 2014) Matched articles HRCAK visits HRCAK Scopus Downloads citations 13, 127 2, 084, 174, 888 38, 106 WOS citations 30, 324
Visits vs Downloads Article page visits: 1 -250=84%; Downloads: 1 -250=59%, 251 -500=22%
cor(log(1+HRVisits, HRDown), method=”spearman”)
Possible reasons …
Scopus vs Web of Science citations Scopus: 0=41%, 1 -5=44%; Web of Science: 0=47%, 1 -5=42%
cor(log(1+HRDown, Scopus. Cit, Wo. SCit), method=”spearman”)
Visits, downloads and citations relative to different subject fields Subject area No. journals % No. articles % Biomedicine and health 10 22. 2% 5856 44. 6% Biotechnical sciences 5 11. 1% 1420 10. 8% Humanities 7 15. 6% 527 4. 0% Sciences 8 17. 8% 2037 15. 5% Social sciences 6 13. 3% 1365 10. 4% Technical sciences 9 20. 0% 1922 14. 6% 45* 100. 0% 13127 100. 0% Total
Median number of visits and downloads & Mean number of citations Mean number of Scopus and Web of Science citations relative to six subject areas Median number of HRCAK visits and downloads relative to six subject areas
KEY RESULTS ● Different usage (reader) and impact (author) characteristics ● Very strong, strong and low correlations o Scopus and Wo. S citation counts o article page visits and downloads o downloads and citation counts ● But, with significant differences between and within six subject areas
CONCLUSION ● Important to have different (MULTIPLE) but complementary measures for better reflecting the multifaceted and multidimensional character of scientific work (output, impact, activities …)
ACKNOWLEDGMENT • HRCAK team (University Computing Centre( • Tomislav Jagušt (Faculty of Electrical Engineering and Computing , University of Zagreb(
Bieber’s response to yesterday's accusation
- Slides: 39