Information Systems and Organisations Lecture 5 Evaluating IS

Information Systems and Organisations Lecture 5: Evaluating IS V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 2 Scope and Coverage This topic will cover: • Tangible and Intangible benefits • Formal-rational evaluation • Wider evaluation approaches V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 3 Learning Outcomes By the end of this topic students will be able to: • Understand typical tangible and intangible benefits of IS • Describe a variety of approaches to IS evaluation • Critically analyse evaluation approaches V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 4 Contents • • Tangible and intangible benefits Relevance of cost benefit measures to evaluating IS Aspects of evaluation Interaction of evaluation and organisational contexts • Emphasis – Technology and Organisations V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 5 Costs of IS (Tangible) • Costs of the technology: - Capital cost of development or purchase - Cost of implementation (project costs) • Costs of the resources to run it: - V 1. 0 Hardware Network Software Service and support © NCC Education Limited

Evaluating IS Lecture 5 - 5. 6 Benefits of IS (Tangible) • Benefits (related to IS/IT) - Reduce fixed costs of IS/IT service - Reduce management overhead • Benefits to the organisation - Increased efficiency in processes (automation) - Increased quality with reduced costs - Increased sales (activity) per unit of resource • (essentially PRODUCTIVITY benefits) V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 7 Fixed and Variable Cost In general, managers may prefer VARIABLE costs in many types of operations because they provide a built in method of reducing costs if activity falls and coping with higher activity if it should rise. IS and IT can help reduce FIXED COSTS and in some cases, through appropriate process changes, replace them with VARIABLE COSTS. Meaning that the organisation is viable at lower levels of activity. . e. g. as in diagram. V 1. 0 Reducing fixed costs will lower the breakeven point © NCC Education Limited

Evaluating IS Lecture 5 - 5. 8 IS Driving Efficiency • Some years ago, the following estimates of banking transaction costs were made - Traditional branch based transaction $1. 07 Telephone transaction $0. 54 ATM transaction $0. 27 Internet (online)transaction $0. 13 • In fact online transactions have sometimes been estimated as low a $0. 01 – the exact figure will vary depending in complexity of the transaction V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 9 ERP Benefits Firm Infrastructure Human Resource Management 55% Technology Development Procurement Inbound Logistics Operations Outbound Logistics 13% Marketing and Sales M arg in Service 32% ERP Benefits % - Source AMR Research April 2004 V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 10 IS Effect is Very Variable • There is no simple relationship between IS/IT investment and productivity • In some cases (e. g. healthcare, social work) productivity is difficult to measure in any event • In some cases (e. g. electronic design, retailing) measuring productivity is easier but outcomes can be difficult to interpret • A late 90’s study of banks in the USA documented a large variability in return on IT investments V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 11 IT Spending and Profitability 24% Wells Fargo Return on Equity First Tennessee Nations Bank Services Northern Trust First Chicago 12% Bank of America Fleet Financial 6% $1000 $10, 000 $100, 000 1995 Information Technology Spending per employee (Banking Sector) Source: Strassman (1997) V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 12 Costs of IS (Intangible) • Opportunity costs? - Should we be doing something else? - Overhead of running IS/IT • Management - Management time and attention - Governance • Risk - Obsolescence not predictable - ‘Game changing’ innovations V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 13 Benefits of IS (Intangible) • Benefits to the organisation - Improved communications Staff morale Reputation (branding benefits) Increased customer satisfaction (branding. . ) Flexibility and agility Organisational learning. . • Intangibles – implicitly ‘un-measurable’ • A management judgement is needed to weigh intangible benefits V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 14 Intangible and Tangible Benefits • Should we ensure that there always significant tangible benefits when we make an investment decision? • Should we make investment decisions with no way of measuring potential benefits ? • Are intangible benefits simply benefits that we cannot yet measure ? V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 15 . . . Should We Even Worry ? • Historical project experience shows that significant numbers of organisations often never seek to seriously measure benefits following implementation of a new IS V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 16 Evaluation Perspectives • Formal and rationale all very well – but: • What can be evaluated ? • Who is evaluating ? • What is the purpose of the evaluation ? • Success results from a combination of aspects V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 17 Main Dimensions of IS Success Development and Process Costs Use Process User Information Satisfaction SUCCESS Quality of the IS Product Benefits Impact of IS on Organization V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 18 Who Evaluates? • • • V 1. 0 Business managers IS/IT users External consultants External authorities (regulatory bodies perhaps) Auditors © NCC Education Limited

Evaluating IS Lecture 5 - 5. 19 Purpose of Evaluation • Perspectives and Aims - To measure tangible costs and benefits To endorse success To investigate failure To identify good practice To improve implementation process • Overt and covert motivations may exist V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 20 IS Success Model - Research • De. Lone and Mclean - a large meta study (1992 and 2003) • What factors lead to IS success ? • What is the relationship between these ? V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 21 De. Lone Mc. Lean Model System Quality Intention to Use Information Quality Service Quality V 1. 0 Net Benefits User satisfaction © NCC Education Limited

Evaluating IS Lecture 5 - 5. 22 References • Boddy, D. , Boonstra , A. , Kennedy, G. (2008) Managing Information Systems : strategy and organisation 3 rd ed. FT Pearson. ISBN-13: 978 -0273 -71681 -5 XXX V 1. 0 © NCC Education Limited

Evaluating IS Lecture 5 - 5. 23 Lecture 5 – Evaluating IS Any Questions? V 1. 0 © NCC Education Limited