SYSTEMATIC IMPROVEMENT OF USER ENGAGEMENT WITH ACADEMIC TITLES

SYSTEMATIC IMPROVEMENT OF USER ENGAGEMENT WITH ACADEMIC TITLES USING COMPUTATIONAL LINGUISTICS Nim Dvir Ndvir@albany. edu www. nimdvir. com @nimdvir Ruti Gafni rutigafn@mta. ac. il

THE INFORMATION INTERACTION The use of ICT has increased radically 93% of Americans report of consuming information online, either via a mobile device or a computer (Stocking, 2017) 79% of U. S. adults reported ever making an online purchase, spending nearly $350 billion annually (Smith, 2017) 80% of Americans have used at least one online government service (Im, Porumbescu, & Lee, 2013) 71% of Americans report looking online for health information (Rock Health, 2015) 2

Information interactions are becoming highly asymmetrical Only a small portion of information receives significant attention, while the remaining majority is barely noticed (Szabo & Huberman, 2008) From the information that is noticed, less than a third is read by users (Nielsen, 2015) 3

CHALLENGES 1. Increased competition Substantial increase in online information, published by commercial businesses, governments, healthcare organizations, and private citizens, lead to billions of Web pages published (Lucian, 2014) 2. Information overload and limited attention Massive amounts of potentially relevant information limit cognitive abilities. As more web content becomes available user attention stretches even thinner across the online space, making it possible only for a small percentage to receive significant traffic (Arapakis, Cambazoglu, & Lalmas, 2017) 4

OBJECTIVE 1. Systematically and methodological evaluate and optimize the informing process 2. Specifically, explore how the expression of the information on the user interface (presentation and phrasing) impact user experience (behavior and perception) 5

DOMAINS • Information systems (IS) • Effective informing influences users to take desirable actions • Provision of information is a central construct in the IS discipline HCI IS BE • Behavioral economics (BE) • Digital nudging – Approach to how changes to the decision environment influence choices • Human-Computer Interaction (HCI) • User interface (UI) and UX design • User experience • User engagement 6

VARIABLES Independent Content effect Digital nudging Dependent Information Interaction / User experience (UX) 7

USER ENGAGEMENT (UE) THEORY Aims to provide a conceptualization that is applicable across a range of situations Integrates knowledge and methods from different disciplines (Boyle et al. , 2012; O’Brien & Cairns, 2016) (Brodie et al. , 2013) Includes adoption and usage of technology, the feeling when using the technology and the motivation to use it (O’Brien et al. , 2018; Perski et al. , 2017) 10

ENGAGEMENT 1. A state of emotional involvement or commitment 2. Process of drawing favorable attention and interest (Merriam-Webster dictionary) 11

THE USER ENGAGEMENT SCALE (UES) Attribute-based approach for evaluating engaging outcomes with a technology. Taps into four dimensions of experience: Novelty & Aesthetic appeal reflect users’ attraction to the media system or interface Focused attention & Felt involvement capture cognitive and emotional focus on media content Perceived usability Endurability & Reward represents evaluations of success and voluntary participation to recommend the website to others 13

CUMULATIVE PROSPECT THEORY (CPT) • A model for descriptive decisions under risk and ncertainty (Kahneman & Tversky, 1979; Tversky & ahneman, 1981, 1992) • Decision-making - the cognitive process resulting in the selection of a belief or a course of action among several alternative possibilities • The process produces a final choice, which may or may not prompt action 14

CPT - PRINCIPALS • Heuristics • Simple rules used to form judgments / “mental shortcuts” • ognitive biases • Unconscious, automatic influences on judgement • Framing • The presentation of the choices • Relative to a certain reference point 15

RESEARCH MODEL 16

PREDICTIVE R 1: CAN THE PHRASING OF DIGITAL INFORMATION – THE WORDS BEING SED – IMPACT UE? “Sticky words” - three extrinsicattributes of digital content of UE 17

PRESCRIPTIVE R 2. CAN ENGAGEMENT WITH INFORMATION BE SYSTEMATICALLY EVALUATED, REATED OR MANIPULATED? We develop a computational linguistic process to employ “Sticky words” 18

A pilot study (Qualtrics. com) METHOD One-way Two-group design - two variations of the same titles (one change per title) Online user studies – college student (n=216) Randomized A/B test (between-subjects) Behavior (selection) Measured: Affect (UES) 19

CREATING A DATASET Derive a list of potential Sticky Words from IMDB. com (the Internet Movie Database) Classify based on polarity using sentiment analysis analyze academic titles to find and replace synonyms 20

1 Identification of vacant homes using system dynamics Identification of zombie homes using system dynamics 2 End of the library: Organized information and digital ubiquity Death of library: Organized information and digital ubiquity 3 Should reproduction and adorationbe related in HIV research? Should sex and love be related in HIV research? 4 Grappling cancer treatment using new types of treatment Fighting cancer using new types of treatment 5 Be an exemplar: Effective civic engagement through open data and digital government Be an action hero: Effective civic engagement through open data and digital government

RESULTS - SELECTION Original Treatment 78% 51% 22

This article is interesting I believe that reading this article is worthwhile This article is easy to understand This article is appealing, I wish to read more 6. 00 RESULTS EVALUATION 4. 96 5. 00 4. 91 4. 67 4. 62 4. 70 4. 44 4. 39 4. 30 4. 00 3. 77 3. 65 3. 36 3. 27 3. 25 3. 17 4. 25 3. 76 3. 64 3. 49 4. 27 3. 00 2. 62 2. 41 2. 37 2. 11 1. 89 1. 76 2. 00 1. 97 1. 63 1. 33 1. 08 2. 28 1. 53 1. 30 1. 25 1. 74 1. 66 1. 43 1. 24 1. 36 1. 05 1. 00 0. 00 Be an exemplar: Be an action Grappling cancer Fighting cancer End of the Death of library: Should sex and Identifying Effective civic hero: Effective treatment using new types library: adaptable Reproduction love be related vacant homes zombie homes engagement civic new types of of treatment Adaptable organized and adoration be in HIV research? using system through open engagement treatment organized information and related in HIV dynamics data and digital through open information and digital ubiquity research? government data and digital ubiquity

RESULTS – EVALUATION (UES) Original Treatment 3. 86 3. 21 24

DISCUSSION This study suggests the possibilities to: ➊ Set criteria for effective (engaging) content strategy(message) which is relevant to various domains ➋ ➌ Identify determinants / characteristics of effective content that can be isolated and measured Develop a framework for systematic valuation, optimization, and reation f engaging content using text analysis and computational linguistics

FUTURE STEPS 1. FRAMEWORK AND MODELS DEVELOPMENT For systematic and strategic • valuation • Optimization • reation f engaging and interactive content 2. EMPIRICAL STUDIES Test the • Validity • Reliability • Generalizability of the framework and model in various ontexts

TODA! Nim Dvir ndvir@Albany. edu @nimdvir Ruti Gafni rutigafn@mta. ac. il 27