Usage metrics Usage concerns Rupert Gatti Usage metrics

Usage metrics & Usage concerns Rupert Gatti

Usage metrics: Uses At OBP we have used usage metrics to demonstrate: • The high levels of engagement with our content • The geographic reach of our published content • Differences in engagement/popularity by format • Engagement with publications over time • University usage for reports to libraries But we have reservations for using it to actually measure anything …

https: //reports. openbookpublishers. com/public/report/10. 11647/OBP. 0 125

Ethics for A-level: Lessons • • • Accessed over 23 k times – in just over a year More users from the US than the UK Online users seem to mildly prefer the html edition to the pdf reader. Book downloads are about 15% of online usage Usage highest during school/uni term, and higher this academic year than last academic year We have made some effort to market this to first year undergraduate courses in the US (as well as to A-level students in the UK) – so we find this type of data useful in evaluating the effectiveness of our efforts.

https: //reports. openbookpublish ers. com/public/report/10. 11647 /OBP. 0025

Oral Literature in Africa: Lessons • Over 170 k online users • Over 60% of online users come from Africa (Kenya, Nigeria, Tanzania) • Online readership has not tailed off in the six year since publication - but book downloads seem to have peaked in 2014 • On the OBP website – relative popularity of HTML and PDF readers has varied over time • Open Edition is becoming an increasingly important discovery/access platform for online users, while Google Books less important • Free downloads from retail distributors (amazon & google play) have been significant.

Library usage reports OBP has a library membership scheme, and as part of this provides annual usage reports for the university Which they compare with ebook usage reports for non-OA titles libraries want COUNTER compliant reports we need to aggregate usage figures across types OBP only has data for usage on the OBP site so ‘missing’ usage data from titles hosted on all other sites

DANGER – numbers give a false sense of accuracy Just defining a usage/download is download • Huge numbers of bots, which are screened out of the statistics • Defining a “use” • what is a “session”? • what if somebody clicks the download button twice? We have three different ways of measuring usage on our website – Google Analytics, Matomo and direct analysis of our weblogs (COUNTER) all three give us different numbers! Which should we use? (we report GA on website, COUNTER to libraries)

GA v COUNTER: 25% difference Oral Literature in Africa Jan Feb Mar Apr May Jun Jul 2018 HTML Reader GA COUNTER PDF Reader 30 38 34 43 66 83 40 50 51 64 Aug Sep Oct Nov Dec 28 35 54 68 33 41 23 29 33 41 74 93 49 61 GA 789 880 927 1188 1344 1021 894 1198 1041 993 1135 586 COUNTER 920 1307 1456 1539 1680 1276 1180 1473 1289 1125 2066 768

Downloads vs Online Readers • Can I just add these together? • Some sites allow you to do either (eg OBP) – so you can browse before downloading • Some sites only allow downloads (eg OAPEN) – so you need to download to see anything • How different are uses and users (do users come back often to read/download from the same platform)? • Downloaded titles can be shared … how do I assess those? • Will I ever know the number of uses of a downloaded title? • Actually I could monitor that – but should I?

Volume vs Chapter data • Many sites only allow chapter downloads (eg JSTOR), others only whole book downloads (eg OAPEN), and some allow both (eg OBP) • How do you combine these different types of data? • In some cases session data is available (OBP) – but often not (JSTOR) • OBP only reports session data (so presently not reporting JSTOR) • With chapter level data - readers wanting the whole book generate more downloads if the book has more chapters Ref: my blogpost “Handle with Care: pitfalls in analysing book usage data”

https: //www. springernature. com/gp/open-research/journalsbooks/the-oa-effect

All three effects are potentially bogus! But I’ll just look at the downloads figure now: • SN aggregates everything into chapter downloads – so if you download a single pdf of the entire book (with, say, 20 chapters) it is recorded as 20 ‘chapter’ downloads. • Thus – if users faced with buying content prefer to by a single chapter to the entire book, but prefer to download the entire book rather than a single chapter when both a free – then a transition for non-OA to OA should increase the number of recorded downloads by 20, even thought the number of users is unchanged. • (of course the user now has access to more content, but … is that we are counting? ) “Handle with Care: pitfalls in analysing book usage data” https: //rupertgatti. wordpress. com/

Where is the reference (Do. I) landing page? At OBP we use the ‘book page’ as the reference url for our titles. Readers then receive quite a lot of info about the book, and then can make a choice about if/how they would like to access the full content: download/online pdf reader/html/xml or purchased copy We think this is the most useful place for a reader to land. But we would have many more download/usage records if we chose one of the content pages as the landing page.

We are missing lots of data … • How much? – we don’t know • But we do know our books can be downloaded from many sites where we don’t get any usage reports • Author homepages, academia. edu, research gate, university repositories …. • Which is great for disseminating the book – but problematic if the usage stats ‘matter’ to either us or the author • Geographical data even worse … HIRMEOS is directly trying to reduce these problems

Gaming the stats …. • Of course – if anything important depends on the usage stats reported (for author or publisher), then there are incentives to change behaviour • • • Relink Do. I urls Redefine session lengths Discourage posting/sharing of content on sites with no usage data Be less rigorous in cleaning bots Create bots to inflate usage data …. .

Social media data is even worse! We observe that the majority of ‘tweets ’ about OBP books do not include a url to a bookpage.