Consuming JSONLD Experiments with Primos Latest Linked Data

Consuming JSON-LD: Experiments with Primo's Latest Linked Data Corey Harper 2015 -09 -05 IGe. LU 2015 – Developers Day

Curl | jq | less to Beta Search REST API curl "http: //bobcatdev. library. nyu. edu/ primo_library/libweb/webservices /rest/v 1/pnxs? q=any, contains, mo nkeys&inst=NYU" | jq -C '. docs' | less -r

Ruby RDF & JSON-LD require 'rdf‘ ‘json/ld’ 'rdf/turtle‘ 'openssl' resource = RDF: : Resource(RDF: : URI. new("http: //bob catdev. library. nyu. edu/primo_library/libwe b/webservices/rest/v 1/pnxs/L/nyu_aleph 0 01770007? inst=NYU")) graph = RDF: : Graph. new << JSON: : LD: : API. to. Rdf(resource) graph. dump(: ttl, prefixes: {list prefixes})

What just happened? • This doesn’t actually work. • Not _quite_ valid JSON-LD • It needs an actual context • Add the context yourself, and you get errors when validating / linting ▫ http: //json-ld. org/playground/ ▫ https: //github. com/Ex. Libris. Group/Primo. PNX -context/blob/master/PNX-context. json

Still works in JQ (but maybe not JSON Tools? )

Consuming JSON (-LD? ): Experiments in with (and Without) Primo's New RESTful Analytics Search API

Distribution of Titles

Why do this stuff? • Understanding your collections • Understanding queries and usage • Identifying Strengths • Topic Modeling ▫ Clustering ▫ Recommendation systems ▫ (Automatic Classification? )

On Github (Also, DPLA Examples) https: //github. com/chrpr/dplaanalytics/blob/master/primo/RESTapi. ipynb

Tree Map of Title Words

Data Science Venn Diagram http: //drewconway. com/zia/2013/3/26/the-data-science-venn-diagram 2015 -04 -18 Harper - Can Metadata be Quantified? - DPLAFest 2015 14

Analytics Writ Large • APIs have much potential • Limited in scope (for now) • NYU External Data Warehousing • Analyze Query Logs • Analyze Dedup Merger • Analyze URLs with (and outside) of Aleph

Toolkit • Tableau – Business Intelligence • R – Stats Analysis • Python • Sci. Py, Num. Py, Pandas, etc • NLTK • jq, awk, sed, grep, sort, uniq, tr, wc, etc. 2015 -04 -18 Harper - Can Metadata be Quantified? - DPLAFest 2015 16

Data Quality Control

Duplicate OCLC # Analysis

Collection Management Decisions Warehouse combines: • Primo dedupmrg & frbr matches • Ebook SUSHI/Counter Stats • Aleph Circ Stats Offsite & De-accessioning Decisions Regression analysis to demo correlations 2015 -04 -18 Harper - Can Metadata be Quantified? - DPLAFest 2015 20

CRISP-DM "CRISP-DM Process Diagram" by Kenneth Jensen Own work. Licensed under CC BY-SA 3. 0 via Wikimedia Commons http: //commons. wikimedia. org/wiki/File: CRISP-DM_Process_Diagram. png 21 Harper - Can Metadata be Quantified? - DPLAFest 2015 -04 -18

Possibilities & Next Steps • • Exploratory Data Analysis. More. Answering questions about data quality More Topic Maps Bi- and Tri-gram Tokenization, Hapaxes • Data Cleanup and QA • Processing income batch data • Integrate with other data streams: • Google Analytics, App. Dynamics, Kibana • Cross system logs, searches, etc. 2015 -04 -18 Harper - Can Metadata be Quantified? - DPLAFest 2015 22

Thanks! corey. harper@nyu. edu 212. 998. 2479 @chrpr 2015 -04 -18 Harper - Can Metadata be Quantified? - DPLAFest 2015 23