ADVANCING THEORY OF EFFECTIVE USE THROUGH OPERATIONALIZATION Glen

ADVANCING THEORY OF EFFECTIVE USE THROUGH OPERATIONALIZATION Glen Murphy Erwin Fielt Rebekah Eden QUT Business School Information Systems School Queensland University of Technology Queensland University fo Gd. murphy@qut. edu. au e. fielt@qut. edu. au rg. eden@qut. edu. au a university for the real S c h o o l ® world o f I n f o r m a t i o n School of Information Systems S y s t e m s 1

Introduction “Learning what effective use actually involves is a complex challenge. The route is long and difficult. ” (Burton-Jones, Bremhorst, Liu, Trieu, 2017) • Effective use: “using a system in a way that helps attain the goals of using the system” (Burton-Jones and Grange 2013, p. 633) Dimension Definition (Burton-Jones and Grange, 2013) Transparent Interaction “The extent to which a user is accessing the system’s representations unimpeded by its surface and physical structures. ” Representational Fidelity “The extent to which a user is obtaining representations from the system that faithfully reflect the domain being represented. ” Informed Action “The extent to which a user acts upon the faithful representations he or she obtains from the system to improve his or her state. ” • Guidance into how to operationalise and measure effective use is paramount • Model I: Multidimensional Formative • Model II: First Order Operationalization “How can effective use be operationalized and measured? ” School of Information Systems Service Science: Service Languages, Techniques and Methods 2 2

Operationalising Effective Use: Model I – Multidimensional Formative H 1: Effective use positively influences individual impact School of Information Systems Service Science: Service Languages, Techniques and Methods 3 3

Operationalising Effective Use: Model II – First. Order Operationalisation • H 2: Transparent interaction positively influences representational fidelity • H 3: Representational fidelity positively influences informed action. • H 4: Informed action positively influences individual impact School of Information Systems Service Science: Service Languages, Techniques and Methods 4 4

Method: Measurement and Operationalization • Survey method conducted with interviews to understand the context • A field survey was performed to understand the effective use of the financials module of an enterprise system at a large Australian University. • Measurement items were drawn from existing literature (Burton-Jones and Grange; Gable et al. 2008) School of Information Systems Service Science: Service Languages, Techniques and Methods 5 5

Results: Analysis of Model I - Multidimensional Formative School of Information Systems Service Science: Service Languages, Techniques and Methods 6 6

Results: Analysis of Model II – First Order Operationalization Indirect Effects on II Path Weight 95% Confidence Interval Lower CI (2. 5%) Upper CI (97. 5%) TI -> RF -> IA -> II β = 0. 037, p<0. 05 0. 008 0. 077 TI -> IA -> II β = 0. 035, p>0. 05 -0. 002 0. 082 TI -> RF ->II β = 0. 162, p<0. 01 0. 060 0. 273 Total Indirect Effect β = 0. 234, p<0. 05 0. 138 0. 342 *TI: Transparent Interaction; RF: Representational Fidelity; IA: Informed Action; II: Individual Impact; CI: Confidence Interval; VAF: Variance Accounted For; NS: Non significant VAF 5. 9% NS 25. 7% 31. 6% School of Information Systems Service Science: Service Languages, Techniques and Methods 7 7

Discussion & Conclusions Which operationalisation to use? • We propose: • When the objective is to understand the consequences of effective use, it is likely to be more appropriate to use Model I. • When the objective is to understand how to improve the effective use the first order relational approach is likely to be especially useful. Future Research Topics • Measurement: triangulating self-reported measures with independent assessments. • Contextualising dimensions: identifying context specific dimensions of effective use in a way that maintains content validity (see Burton-Jones and Volkoff, 2017) • Users: relevance of the dimensions for different user groups School of Information Systems Service Science: Service Languages, Techniques and Methods 8 8

For more information contact Rebekah Eden (rg. eden@qut. edu. au) School of Information Systems Service Science: Service Languages, Techniques and Methods 9