Research Metrics Challenges for Legal Scholars Bibliometrics Research
Research Metrics: Challenges for Legal Scholars
Bibliometrics & Research Metrics • Grounded in sciences • Shortcomings well canvassed
Evaluation for Social Sciences & Humanities Appropriate mix of: 1) (Traditional) quantitative methods Quantity / quality / structural indicators 2) Qualitative methods 3) Socio-economic impact Dutch Protocol UK REF “Impact Case Studies” / “Impact Reports”
Particular challenges for legal scholars? Resistance to metrics OR resistance to inapt measures? Sean Rehaag, “Are Publication and Citation Counts Reliable Indicators of Research Productivity or Impact? ” (2017) 43: 2 Canadian Law Library Review 26.
Challenges For Legal Scholars • Multiple primary audiences & fora • Nature of legal journals & their contributors • Monographs & other nonarticle works • Referencing conventions • Local & Domestic v. International • Multiple methodologies • Interdisciplinary - “Law & …” • “Aging rate” • Databases / indices issues
Database & indices issues: scope / coverage Web of Science (Thompson Reuters) Arts & Humanities Citation Index 1766 journals 5 law journals 0 Canadian law journals Social Science Citation Index 3224 journals 135 law journals 0 Canadian law journals Elsevier – Scopus “the largest database of peerreviewed literature” 37953 journals (including trade) 574 law journals 7 Canadian law journals
Database & indices issues Methodology & results • Not always transparent AU Council of Law Deans 2016 report: assessment of Journal Citation Reports JCR − Database “defies description” − “unreliable” − “mysterious” results
Career Implications • Co-author problems • Law journals rankings
- Slides: 9