IT audit training for INTOSAI IT audit training

IT audit training for INTOSAI IT audit training Value For Money March 2007 VFM Audit : S 1/ 1

IT audit training VFM audit of IT for F Objective F March 2007 By the end of the module you will be able to: V build a business model; V identify significant IT systems V identify and evaluate evidence of failing IT systems V recognise explanations for IT system failure V assess the development environment to see that sound management practices minimise risks VFM Audit : S 1/ 2

IT audit training The course schedule for Scope of Value For Money audit F VFM audit method F VFM audit experience F March 2007 VFM Audit : S 1/ 3

IT audit training for VFM audit concepts Value For Money = Performance = 3 Es F Economy is concerned with spending less by minimising the cost of resources of a given quality. F Efficiency is concerned with spending well by minimising resources used for a given output. F Effectiveness is concerned with spending wisely by ensuring that the outputs achieved match the policy or operational objectives. March 2007 VFM Audit : S 1/ 4

IT audit training Economy for F Failing to take bulk discounts F Using top quality paper for temporary outputs F Using expensive workstations where a dumb terminal would do F Making routine international calls at peak rate F Buying in resources when there is spare in-house capacity F Building your own system when a suitable off-the-shelf system is already available and cheaper, or visa versa. F Delivering IT services in-house when a cheaper outsourcing option is available. March 2007 VFM Audit : S 1/ 5

IT audit training for F F March 2007 Efficiency Controls that serve no real purpose Making frequent calls when one would do Duplication of systems that perform the same function Entry of information already held by the system VFM Audit : S 1/ 6

IT audit training Effectiveness for F F F March 2007 Systems that serve no business function Unreliable systems Systems that produce reports that are never used VFM Audit : S 1/ 7

IT audit training VFM audit objectives for F F March 2007 Improving clients’ business management Validating central guidance Explain waste Recommendations for corrective action VFM Audit : S 1/ 8

IT audit training VFM audit method for F Survey V F Evidence of success or failure V F Why is it going wrong? Recommendation V March 2007 Is it going wrong? Explanation V F Is it important? What can be done now? VFM Audit : S 1/ 9

IT audit training Survey for F F F F March 2007 Objectives Organisation Resources Performance Prioritisation IT Strategy and Environment Systems VFM Audit : S 1/ 10

IT audit training for Objectives What the client aims to do F Owner F Hierarchical F Link to performance indicators F Link to activities March 2007 VFM Audit : S 1/ 11

IT audit training Weighting for F 1. V V March 2007 Mission=100 1. 1 40 » 1. 1. 1 » 1. 1. 2 60 » 1. 2. 1 » 1. 2. 2 30 10 15 45 VFM Audit : S 1/ 12

IT audit training for Organisation How the client aims to meet objectives F Policies and standards F Control environment F Culture F Activities March 2007 VFM Audit : S 1/ 13

IT audit training Activities for F F F March 2007 Produce outputs Business units Cost centres Link to objectives Link to performance indicators Supported by information systems VFM Audit : S 1/ 14

IT audit training Information systems for Support activities F Contribute to objectives F Monitor progress F Record resource usage F IT Strategy and Environment F V March 2007 IT Governance Maturity Model VFM Audit : S 1/ 15

IT audit training Resources for F F F March 2007 Inputs to activities Resource management Internal versus external VFM Audit : S 1/ 16

IT audit training Performance for F F F March 2007 Impact versus output Performance regime Performance indicators Benchmarking Stakeholders VFM Audit : S 1/ 17

IT audit training Performance regime for F F F March 2007 Key indicators Clear responsibilities Good resource management system Regular review Decisive action Targets VFM Audit : S 1/ 18

IT audit training Performance indicators for F F F March 2007 Link to objectives and activities Measurable Comprehensive Consistent Comparable Verifiable VFM Audit : S 1/ 19

IT audit training Benchmarking for F F March 2007 Activities Impacts Across time Across organisations VFM Audit : S 1/ 20

IT audit training Stakeholders for F F March 2007 Customers Politicians Journalists Academic and professional bodies VFM Audit : S 1/ 21

Framework for Audit and Control of Effectiveness IT audit training for T A Objectives S Review K O Impacts D E Inputs ACTIVITIES C O S T S R S H E AIM L TARGET Feedback Outputs Performance Indicators March 2007 VFM Audit : S 1/ 22

IT audit training Study selection for F F F March 2007 Poor performance High cost Strategic importance Management weakness Systematic failure Relevance to many clients VFM Audit : S 1/ 23

IT audit training for Materiality of IT Programme costs IT costs Administration Expenditure March 2007 VFM Audit : S 1/ 24

IT audit training for Importance of systems Programme delivery SYSTEMS IT Administration March 2007 VFM Audit : S 1/ 25

IT audit training Place of IT in studies for F March 2007 Part of explanation V Large projects V Comparative studies V Gross or systematic failures V Central guidance V Emerging technology VFM Audit : S 1/ 26

IT audit training for VFM audit method - Evidence F Survey V F Evidence of success or failure V F Why is it going wrong? Recommendation V March 2007 Is it going wrong? Explanation V F Is it important? What can be done now? VFM Audit : S 1/ 27

IT audit training Evidence for F Quality user dissatisfaction V unreliable V poor integration V costly to run or maintain V disputes with suppliers F V March 2007 F Time V abandoned or delayed systems Cost V expensive systems VFM Audit : S 1/ 28

User dissatisfaction sources of evidence IT audit training for F F F March 2007 Post implementation reviews System owner Survey Interviews Help desk System usage statistics VFM Audit : S 1/ 29

IT audit training for User dissatisfaction - explanation F F F March 2007 Strategy formulation and review Requirements capture Inadequate quality control Operational management Training and awareness VFM Audit : S 1/ 30

IT audit training for Unreliable systems - sources of evidence F F F March 2007 Operations manager System owner Support staff Maintenance records Error logs VFM Audit : S 1/ 31

IT audit training for Unreliable systems - explanation F F F March 2007 Design standards Development standards Maintenance Operational management Infrastructure VFM Audit : S 1/ 32

Poor integration sources of evidence IT audit training for F F F March 2007 Duplicate entry Complex data conversion Data administrator VFM Audit : S 1/ 33

IT audit training for Poor integration - explanation F F March 2007 IT strategy End user computing Procurement control Development control VFM Audit : S 1/ 34

Cost overruns sources of evidence IT audit training for F F F F March 2007 System owner Business case Project board minutes IT steering committee Project control documents Management accounts Post implementation reviews VFM Audit : S 1/ 35

IT audit training for Cost overruns - explanation F F F March 2007 Investment appraisal Project management Design standards Development standards Operations management VFM Audit : S 1/ 36

IT audit training for Delays - sources of evidence F F F March 2007 Strategic plans System owner Project board minutes IT implementation schedules Business case VFM Audit : S 1/ 37

IT audit training Delays - explanation for F F F March 2007 Unrealistic timetable Project management User opposition VFM Audit : S 1/ 38

Failed projects sources of evidence IT audit training for F F F March 2007 IT strategies IT steering committee Post implementation reviews Potential users Finance department VFM Audit : S 1/ 39

IT audit training for Failed projects - explanation F F F March 2007 Business case Requirements capture Project management VFM Audit : S 1/ 40

Costly maintenance sources of evidence IT audit training for F F March 2007 Comparison with other systems Resource management system Change management records System owner VFM Audit : S 1/ 41

IT audit training for Costly maintenance - explanation F F March 2007 Requirements capture Flexibility Development standards Skills shortage VFM Audit : S 1/ 42

Supplier disputes sources of evidence IT audit training for F F F March 2007 Correspondence with supplier Interview owner Records of meetings VFM Audit : S 1/ 43

IT audit training for Supplier disputes - explanation F F F March 2007 Inadequate specification Unrealistic bid Service level agreements VFM Audit : S 1/ 44

Lack of evidence is evidence of danger IT audit training for F F F F March 2007 No business plan No IT strategy No business case No owner / users No project plan or budget No resource monitoring No post implementation reviews VFM Audit : S 1/ 45

IT audit training for VFM audit method - Explanation F Survey V F Evidence of success or failure V F Is it going wrong? Explanation V March 2007 Is it important? Why is it going wrong? VFM Audit : S 1/ 46

IT audit training Explanation for F F F F March 2007 IT standards IT strategy User involvement Business case Procurement Project or operational management Business continuity Obsolescence VFM Audit : S 1/ 47

IT audit training for Explanation - IT standards F F F March 2007 Strategy Management Design and development Technical integration Change management Standards ignored? VFM Audit : S 1/ 48

IT audit training for Explanation - IT strategy F F March 2007 No link to business plan Ignored Unrealistic Uncoordinated VFM Audit : S 1/ 49

IT audit training Explanation - User involvement for F F March 2007 No system owner Poor requirements capture Inadequate training and awareness No formal user acceptance VFM Audit : S 1/ 50

IT audit training for Explanation - Business case F F F March 2007 Unrealistic time scale Benefits overestimated Costs underestimated Too complex Deliverables under-specified Inadequate quality plan VFM Audit : S 1/ 51

IT audit training Explanation - Procurement for F F F March 2007 Ambiguous terms of reference No service level agreements Inadequate monitoring No competition Lack of intelligent customer VFM Audit : S 1/ 52

IT audit training Explanation - Project management for F F F F March 2007 No method Lack of skills Deliverables vague or undefined Inadequate quality assurance Lack of internal audit Poor cost control Inattention to user training and awareness VFM Audit : S 1/ 53

Explanation Operational management IT audit training for F F F F March 2007 Inadequate service level agreements No capacity planning Poor configuration management Inadequate maintenance No performance monitoring / scheduling Unreliable infrastructure No help desk VFM Audit : S 1/ 54

IT audit training for Explanation - Business continuity F F March 2007 Failure to identify critical systems Inadequate backup Infrequent or unrealistic testing Lack of documentation VFM Audit : S 1/ 55

IT audit training Explanation - Obsolescence for F F March 2007 Users dissatisfied Frequent or expensive changes Frequent system failure Poor integration VFM Audit : S 1/ 56

IT audit training Recommendations for F Pointing out system failures may not be enough V F Recommendations for corrective action “mid-course” V F Improve effectiveness/ efficiency of IT system under review Depends on environment in which SAI functions, and its audit strategy V V March 2007 Lessons only for future IT systems, not current IT system being reviewed Post facto audit OR Concurrent audit VFM Audit : S 1/ 57