Inputs Outputs and Emergent Properties The New Scientometrics

  • Slides: 22
Download presentation
Inputs, Outputs and Emergent Properties: The New Scientometrics Phill Jones, Ph. D Director of

Inputs, Outputs and Emergent Properties: The New Scientometrics Phill Jones, Ph. D Director of Publishing Innovation Digital Science @phillbjones

Who are Digital Science? • Founded in 2011 and wholly owned by Holtzbrink Publishing

Who are Digital Science? • Founded in 2011 and wholly owned by Holtzbrink Publishing Group • Investing in innovation by researchers, for researchers • Creating change from the ground up 2012 2013 2017 Make Better Decisions Faster

The Evaluation Gap *Big. Stockphoto. com

The Evaluation Gap *Big. Stockphoto. com

Prof. Paul Wouters Centre for Science and Technology Studies, Leiden University The … fundamental

Prof. Paul Wouters Centre for Science and Technology Studies, Leiden University The … fundamental gap between. . the dominant criteria in scientific quality control (in peer review as well as in metrics approaches), and on the other hand the new roles of research in society. https: //citationculture. wordpress. com/2014/ 08/28/a-key-challenge-the-evaluation-gap/

How do we close the evaluation gap? Introducing altmetrics SOCIETAL IMPACT ACADEMIC IMPACT Journal

How do we close the evaluation gap? Introducing altmetrics SOCIETAL IMPACT ACADEMIC IMPACT Journal Impact Factor Citation counts Traditional metrics + Policy documents Grey literature Mentions in blogs Peer review websites Mentions in news reports Mentions in social media Reference manager readers … etc. Alternative metrics “altmetrics” Output-centric instead of journal-centric Make Better Decisions Faster

Awarded grants data –A new source of intelligence GRANT AWARDED PROCEEDINGS IDEA CITATION PUBLICATION

Awarded grants data –A new source of intelligence GRANT AWARDED PROCEEDINGS IDEA CITATION PUBLICATION 1 -5 Years 2 -3 Years 3 -8 Years Make Better Decisions Faster

Patents are a measure of commercialization Make Better Decisions Faster

Patents are a measure of commercialization Make Better Decisions Faster

Research outputs Impact Funding /Grants One lab Collaborators at the Institution Publications Researchers at

Research outputs Impact Funding /Grants One lab Collaborators at the Institution Publications Researchers at other institutions The big-wide-world Make Better Decisions Faster

Research outputs Impact Funding /Grants Publications Make Better Decisions Faster

Research outputs Impact Funding /Grants Publications Make Better Decisions Faster

How to analyze research activity Make Better Decisions Faster

How to analyze research activity Make Better Decisions Faster

…before you analyze it… …you have to classify it Make Better Decisions Faster

…before you analyze it… …you have to classify it Make Better Decisions Faster

If it moves, it’s BIOLOGY If it smells, it’s CHEMISTRY If it doesn’t work,

If it moves, it’s BIOLOGY If it smells, it’s CHEMISTRY If it doesn’t work, it’s PHYSICS Make Better Decisions Faster

Make Better Decisions Faster

Make Better Decisions Faster

Keywords and semantic modeling Ways to create a taxonomy 1. Use an existing one

Keywords and semantic modeling Ways to create a taxonomy 1. Use an existing one if available (Me. SH) 2. Consult with subject matter experts (your editors) 3. Text mining abstracts or full text article (your content) Make Better Decisions Faster

Data needs to be cleaned and curated to be of use Harvard University of

Data needs to be cleaned and curated to be of use Harvard University of Harvard Medical School Harvard-Smithsonian Martinos Center for Biomedical Imaging Harvard Medical School Business School Center for Astrophysics Martinos Center for Biomed Imaging Make Better Decisions Faster

Interactive visualization https: //www. digital. Makescience. com/visualizations/ref-case Better Decisions Faster -study-similarity-network/

Interactive visualization https: //www. digital. Makescience. com/visualizations/ref-case Better Decisions Faster -study-similarity-network/

Let’s Talk Make Better Decisions Faster

Let’s Talk Make Better Decisions Faster

Predicting the future with grants data Roll-Over: Dimensions checks for data monthly, but funders

Predicting the future with grants data Roll-Over: Dimensions checks for data monthly, but funders don’t always report promptly Make Better Decisions Faster

Predicting the future with grants data We compare funding to a baseline – a

Predicting the future with grants data We compare funding to a baseline – a broad basket of research in the overarching field Make Better Decisions Faster

Finding real trends requires domain expertise Make Better Decisions Faster

Finding real trends requires domain expertise Make Better Decisions Faster