CISA REVIEW The material provided in this slide

CISA REVIEW The material provided in this slide show came directly from Certified Information Systems Auditor (CISA) Review Material 2010 by ISACA.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Learning Objectives: • Evaluate the business case for the proposed system development/acquisition to ensure that it meets the organization's business goals. • Evaluate the project management framework and project governance practices to ensure that business objectives are achieved in a costeffective manner while managing risks to the organization. • Evaluate proposed control mechanisms for systems and/or infrastructure during specification, development/acquisition and testing to ensure that they will provide safeguards and comply with the organization's policies and other requirements. • Evaluate the processes by which systems and/or infrastructure are developed/acquired and tested to ensure that the deliverables meet the organization's objectives. • Evaluate the readiness of the system and/or infrastructure for implementation and migration into production.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Learning Objectives, cont. • Perform post-implementation review of systems and/or infrastructure to ensure that they meet the organization's objectives and are subject to effective internal control. • Perform periodic reviews of systems and/or infrastructure to ensure that they continue to meet the organization's objectives and are subject to effective internal control. • Evaluate the process by which systems and/or infrastructure are maintained to ensure the continued support of the organization's objectives and are subject to effective internal control. • Evaluate the process by which systems and/or infrastructure are disposed to ensure that they comply with the organization's policies and procedures.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Organizations need to develop and change through a systematic process that maximizes benefits. With the growth of information technology, most business benefits are obtained through technology -enabled changes. Every proposed IT system for an organization should have identifiable benefits for both the organization and its customers. Identifying these benefits will require an understanding of the work processes of the organization and its customers.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Organizational benefits of new IT systems may include: • enabling some personnel to perform two different jobs with little or no extra training, • allowing organizational changes that reduce the number of managers, • permitting some jobs to be eliminated entirely. These benefits are usually measured in terms of productivity gains, staffing reductions and improved organizational effectiveness.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management IT governance – as part of enterprise governance – should be driven by business goals and objectives. The IS auditor should evaluate whethere is a business strategic planning process in place by considering questions such as: • Is there a clear definition of business vision and mission? • Is there a business strategic planning methodology used? • Is the level of the individuals involved in this process appropriate? • Is this planning periodically updated?

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management The benefits realization process involves: • Assigning a measure and target, • Establishing a tracking/measuring system, • Documenting any assumptions, • Establishing key roles and responsibilities, • Planning for the benefit to be realized, • Validating the benefits predicted in the business. This is a continuous process, and enterprise wide benefits realization studies should be collected and synthesized to finetune the benefit realization process.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Examples of benefits for IT systems include: Accuracy – Will the proposed system provide better accuracy by reducing the number of data entry errors? Availability – How long will it take to develop and implement the system? Will one alternative be available sooner than another? Compatibility – How compatible is the proposed alternative with existing facilities and procedures? Will one alternative require less training of personnel or less new equipment or software? Efficiency – Will one alternative provide faster or more accurate processing of inputs? Will one alternative require fewer resources for the processing? Maintainability – Will the maintenance costs for one alternative be lower than the costs for the others? Are the maintenance resources easier to acquire for one alternative? An example of this would be availability and cost of programmers to maintain the software.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Examples of benefits for IT systems include: Reliability – Does one alternative provide greater hardware or software reliability? Greater reliability translates to higher productivity in using and/or operating the system and less time for operations and user support. Security – Does one alternative provide better security to prevent fraud, waste or abuse? Are privacy, confidentiality and data integrity enhanced?

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management The business case needs to provide enough detail to justify the decision to develop and continue a project. The business case for a new system should, for example: • Determine the strategic benefits of implementing the system, either in productivity gains or in future cost avoidance, • Identify and quantify the cost savings of the new system, • Estimate a payback schedule for the cost incurred in implementing the system or show the projected return on investment (ROI), • Identify intangible benefits, such as improved morale, and quantify these wherever possible.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management The objective of the feasibility study is to give an assessment of the business requirement or opportunity and determine whethere are feasible solutions before the commitment of full life cycle resources. Several key questions need to be addressed in advance of conducting the study. • What is the specific requirement or opportunity and what group within the organization is responsible? • What are the new information needs that are associated with this requirement or opportunity? • What should be the scope of the solution?

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management An impact assessment looks at the potential effects of a proposed development project on current projects and resources. The report of the impact assessment should outline the benefits and drawbacks of pursuing a specific course of action. Value analysis uses a team approach to analyze the functions of facilities, processes, systems, equipment, services and supplies. The objective is to achieve the essential functions at the lowest life cycle cost (LCC) consistent with required performance, reliability, quality, safety and achievement of the organization's stated mission priorities.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Exercise: Complete each statement by choosing one of the four terms below. A business case A feasibility study An impact assessment A value analysis [Blank] looks at how to achieve the organization's essential functions at the lowest life cycle cost consistent with requirements. [Blank] includes decision criteria, comparisons of potential solutions and a proposed solution. [Blank] looks at the potential effects of a proposed development project on current projects and resources. [Blank] for a new system should identify both intangible benefits and return on investment.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: A value analysis looks at how to achieve the organization's essential functions at the lowest life cycle cost consistent with requirements. A feasibility study includes decision criteria, comparisons of potential solutions and a proposed solution. An impact assessment looks at the potential effects of a proposed development project on current projects and resources. A business case for a new system should identify both intangible benefits and return on investment.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Systems development life cycle (SDLC) The phases deployed in the development or acquisition of a software system. Typical phases of SDLC include: • feasibility study, • requirements definition, • design, • development, • testing, • installation and post-implementation review.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management The requirements definition identifies and specifies the business requirements of the system chosen for development during the feasibility study. Requirements include descriptions of: • What the system should do, • How users will interact with the system, • Conditions under which the system will operate, • Information criteria the system should meet.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management The requirements definition includes these tasks: Identify stakeholders – Consult with them to determine their expectations. Analyze requirements – Determine priorities. Look for conflicts and correct them. Identify system boundaries – Define what is part of the system and what the system will be interacting with. Convert user requirements into system requirements – For example, create a prototype user interface that demonstrates the screen look and feel. Record requirements – Requirements decisions need to be presented and retained in a structured format. Verify requirements – Ensure they are complete, consistent, unambiguous, verifiable, modifiable, testable and traceable. Resolve conflicts – Identify where the requirements do not match the available resources and where the requirements of various stakeholders differ and determine a course of action.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management There is a large payoff to an effective review of requirements because of the high cost of rectifying requirements problems in the downstream phases of development.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management To ensure control over the IT process of identifying automated solutions to satisfy the business requirements, the IS auditor needs to verify that technically feasible and cost-effective solutions are achieved. The IS auditor must ensure: • Business and technical requirements have been defined, • Feasibility studies have been completed, • Approval (or rejection) of the requirements and the feasibility study results is measured by: ØNumber of projects where stated beliefs were not achieved due to incorrect feasibility assumptions, ØPercent of feasibility studies signed off by the business process owner, ØPercent of users satisfied with functionality delivered. Alternate solutions satisfying the business requirements should be identified to help ensure the optimal solution is selected.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Key Personnel in the Systems Development Process The project manager is appointed by the IS Steering committee and is responsible for: • Providing leadership and project management, • Ensuring the project follows the overall direction outlined in its charter, • Involving the affected departments, • Complying with local standards, • Ensuring deliverables meet the quality expectations of key stakeholders, • Resolving interdepartmental conflicts, • Monitoring and controlling costs and schedules.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Key Personnel in the Systems Development Process Senior Management is responsible for: • Approving the resources to undertake and complete the project, • Ensuring, through its commitment, the involvement of others.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Key Personnel in the Systems Development Process User Management is responsible for: • Assuming ownership of the project and resulting system, • Providing qualified representatives to the team, • Actively participating in business process redesign, system requirements definition, test case development, acceptance testing and user training, • Reviewing and approving system deliverables as they are defined and implemented.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Key Personnel in the Systems Development Process Project Steering Committee is responsible for: • Providing overall project direction, • Ensuring appropriate representation of major stakeholders, • Retains ultimate responsibility for all deliverables, project costs and schedules.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Key Personnel in the Systems Development Process Project Sponsor is responsible for: • Providing funding for the project, • Working closely with the project manager to define critical success factors (CSFs) and metrics for the project.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Key Personnel in the Systems Development Process Quality Assurance is responsible for: • Reviewing results and deliverables within each phase and at the end of each phase, • Confirming compliance with requirements.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Exercise: Match each role to its corresponding responsibility. Key Roles Project sponsor Project steering committee Quality assurance Senior management User management Responsibilities Works with project manager to define CSFs, Retains ultimate responsibility for all deliverables, project costs and schedules, Confirms compliance with requirements, Approves the resources to undertake and complete the project, Assumes ownership of the project and resulting system.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: Project sponsor Works with project manager to define CSFs Project steering committee Retains ultimate responsibility for all deliverables, project costs and schedules Quality assurance Confirms compliance with requirements Senior management Approves the resources to undertake and complete the project User management Assumes ownership of the project and resulting system

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Exercise: Think About It At the minimum, what are the basic elements necessary for ensuring the success of a life cycle project?

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: Elements critical to ensuring the overall success of a project include but are not limited to: • Ensuring the project has strong executive sponsorship, • Developing project management skills and awareness at staff and executive levels, • Staying true to the strategy and vision, • Maintaining the enterprise perspective, • Delivering on promises, • Ensuring trading partner agreements are clear and concise, • Providing representation on the steering committee for all major stakeholders.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management System Design: The project requirements definition provides a general preliminary design and user requirements. Based on this, the detailed design is to be developed. Typically this is done by a team of programmers and business and system analysts who define the software architecture, depicting a general blueprint of the system and then detailing or decomposing the system into its constituent parts such as modules and components. This permits resources to be allocated to design and to defining how the system will address all its information requirements.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management The design phase of the project includes these activities: • Developing system flowcharts and entity relationship models to illustrate, how information will flow through the system, • Describing inputs and outputs, such as screen designs and reports, • Determining processing steps and computation rules when addressing functional requirement needs, • Determining data file or database system file design, • Preparing program specifications for the types of requirements or information criteria defined, • Developing test plans for various levels of testing, • Developing data conversion plans to convert data and manual procedures from the old system into the new system.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management The focus for the IS auditor in the design phase of a project is: • Determining if an adequate system of controls is incorporated into system specifications and test plans • Determining if continuous online auditing functions are built into the system (particularly for e-commerce applications and other types of paperless environments) • Evaluating the effectiveness of the design process including the use of: Structured design techniques Prototyping and test plans Software baselining The IS auditor should verify the implementation of a formal software change process that effectively freezes the inclusion of any changes to system requirements without a formal review and approval process.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Continuous online auditing allows IS auditors to monitor the operation of a system and gather selective audit evidence while normal processing takes place. The audit evidence is stored in a separate audit file for later verification by the IS auditor.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management System Development: The detailed design is used in the development phase to start coding. This phase is largely the responsibility of the programmers and systems analysts. The principal activities performed during the development and testing phase include: • Coding and developing the program and system-level documents, • Debugging and testing the programs developed, • Developing programs to convert data from the old system for use on the new system, • Creating procedures to enable users to transition to the new system, • Providing training on the new system for selected users, • Ensuring modifications are documented and applied accurately and completely to vendor-acquired software to ensure future updated versions of the vendor's code can be applied.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management In auditing systems development, acquisition and maintenance, IS auditors should meet with key members of the systems development and user project team to: • Determine the system's principal components, objectives and user requirements in order to identify the areas that require controls. • Determine major risks to and exposures of the system and rank these items. • Identify controls to mitigate the risks to and exposures of the system using references to authoritative sources and meetings with the project team. • Advise the project team regarding the design of the system and implementation of controls through evaluation of available controls and discussions with the team. • Monitor the systems development process to ensure controls are implemented, user and business requirements are met, and the systems development/acquisition methodology is being followed through review of the documentation and deliverables as well as in team meetings.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management IS auditors should also ensure documented controls are in place to address all security, edit and processing issues by reviewing and evaluating application system audit trails. Any program changes should be handled by a change management system that includes: • History of all work order activity (work order date, programmer assignment, changes made and date closed), • History of logons and logoffs by programmers, • History of program deletions, • Evaluation of system maintenance standards and procedures, • Testing of system maintenance procedure to ensure procedures are being applied as described in the standards, • Evaluation of the system maintenance process to determine whether control objectives were achieved by analyzing test results and other audit evidence • Identification and testing of existing controls to assess the adequacy of production library security in order to ensure the integrity of the production resources

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management The activities the IS auditor should perform in reviewing the detailed design and development of a system include: • Reviewing system flowcharts to see that they comply with the general design, • Verification that all changes were discussed and approved by appropriate user management, and that all approvals are recorded, • Reviewing the appropriateness of the input, processing and output controls that are designed into the system, • Interviewing system users to assess their understanding of the system and their degree of input into screen format and output report design, • Determining if the audit trails can adequately provide traceability and accountability of system transactions, • Verifying the integrity of key calculations and processes, • Verifying that the system can correctly identify and process erroneous data, • Verifying all corrections to programming errors were made and the audit trails or embedded audit modules were coded as recommended into the appropriate programs.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management In general, systems are easier to maintain and enhance to the degree that individual software items perform a single, dedicated function (cohesion) and retain independence from other comparable items (coupling), because this facilitates determining where and how to apply a change and reduces the chance of unintended consequences.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Exercise: Determine if each statement pertains to the Requirements, Design, or Development stage of the project. Statement Developing test plans Building programs to convert data to new system Identify Stakeholders Developing data conversion plans Debugging and testing programs developed Requirements Design Development

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: Statement Requirements Design X Developing test plans X Building programs to convert data to new system Identify Stakeholders Developing data conversion plans Debugging and testing programs developed Development X X X

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Following are examples of possible sources of risk for a system development project: • System requirements factors: complexity, difficulty, feasibility, novelty, verifiability and volatility • SDLC deliverables: correctness, integrity, maintainability, performance, reliability, security, testability and usability • Developmental model: manageability, measurability, quality and traceability of the processes used to satisfy customer requirements • Communication, cooperation, domain knowledge, experience, technical knowledge and training of the personnel associated with technical and support work on the project • Budget, external constraints, politics, resources and schedule of the external system environment • Methods, tools and supporting equipment for system development: capacity, documentation, familiarity, tool support, and usability

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Question: Think About It What factors and/or elements should you as an IS auditor consider when assessing systems development and related risks?

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: You should determine whether audit procedures for systems development and acquisition and related risk management adequately consider: • Level of support for systems development by senior management and the board of directors • Adequacy of the management structures to establish accountability and responsibility for IT systems and technology initiatives • Development of IT solutions that meet the needs of end users • Extent of end-user involvement in the systems development process • Adequacy of the institution's systems development methodology and programming standards • Quality of practices followed by developers, operators, executive management, vendors and end users • The independence of the QA function and the adequacy of controls over program changes • Quality and thoroughness of system documentation

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management System Maintenance Once a system has been developed and moved into production, it will be subject to changes in response to alterations in the business, the IT structure, adverse incidents such as intrusions and viruses, changes in the classification of an item, and audits. System maintenance practices must be used to manage changes to system processes and configurations and to ensure the continued integrity of both the production source and executable code.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management System Maintenance: A standard process for performing and recording changes is needed to control the ongoing maintenance of the system. This process should mirror the organization's SDLC process to make sure that all system changes are: • Appropriate to the needs of the organization, • Authorized, • Documented, • Tested thoroughly, • Approved by management. This process needs to be applied not only to software changes but to the overall IT and business structure.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Change requests may be initiated by end users, operational staff or system development/maintenance staff. The request should be submitted in a formal document format such as a change request form, a memo or e-mail. The request should include: • Requestor's name, • Date of the request, • Date change is needed, • Priority of the request, • Description of the change request, • Description of any anticipated effects on other systems or programs, • Reason for change, such as a cost-justification analysis, and expected benefits of the change, • Changes submitted by end users should also include evidence that the change request has been reviewed and approved by user management.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Testing Changes: To ensure the changes perform the functions intended, changed programs should be tested and certified with the same rigor as is applied to newly developed systems. Depending on the outcome of a risk analysis of the change, additional testing may be required to ensure: • Existing functionality is not damaged by the change. • System performance is not degraded because of the change. • No security exposures have been created because of the change.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Deploying Changes Back Into Production After a change has been approved by user management, the modified programs can be moved into the production environment. A group that is independent of computer programming – such as computer operations, QA or a designated change control group – should perform the migration of programs from test to production. Appropriate access restrictions must be in place to ensure only authorized individuals can migrate programs into production. This may be accomplished through operating system security or an external security package. Migrating changes into an application in production requires a detailed plan that includes checkpoints at which "go" or "no-go" decisions are to be made. This allows the change to be rolled back if problems are encountered.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Question: Think About It What is process change management and what is it designed to accomplish?

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: Think About It What is process change management and what is it designed to accomplish? Process change management involves defining process improvement goals and, with senior management sponsorship, proactively and systematically identifying, evaluating and implementing continuous improvements to the standard software process of the organization and defined software processes of the project. The purpose of process change management is to continually improve the software processes used in the organization. The objective is to improve software quality, increase productivity and decrease the cycle time for product development.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Question: To which of the following should a change management process or review be applied? 1. A new function to incorporate improved processing and information delivery 2. New LAN hardware 3. Changes to the existing software that the requester has classified as minor 4. A last-minute change to resolve a problem 5. A security patch provided by the software vendor

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: To which of the following should a change management process or review be applied? All these changes should be subject to a change management process or review. The last-minute emergency change should be subjected to a retroactive change management process. The vendor -provided patch should be reviewed to ensure it will not have a negative effect on the system, as the organization has deployed it.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management An IS auditor should ensure controls are in place to protect production application programs from unauthorized changes. The control objectives are: • Restricted access to program libraries, • Conduct supervisory reviews, • Approved and documented change requests, • Assessment of potential impact of changes.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management The change request, documented on a standard form, should include the following: • Specifications, along with a cost analysis and a target date • Signature of the user, to designate approval • Evidence of review and approval by programming management • Information on the assignment of the work to an analyst, programmer and programming group leader for supervision The audit should include a sample of program changes made during the audit period. The selected changes should be traced to the maintenance form to determine whether the changes were authorized, that the form includes appropriate approvals and that the date on the form can be matched to the date of production update.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management There is potential exposure for unauthorized change to an application system program if: • A change request form and procedures are not formally established, • The user responsible for the application and appropriate management do not sign a change request approving the start of the work, • Programmers have access to production libraries containing programs and data including object code, • The changed source code is not properly reviewed by the appropriate programming personnel, • Programmers have the opportunity to put in extra code for personal benefit, • The user and appropriate management do not sign the change form signifying acceptance before the change is updated, • Changes received from the acquired software vendor are not tested or the vendor is allowed to load the changes directly into production or to the site.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Question: Think About It What elements should the IS auditor look for in a well-developed change management plan, related to life cycle projects?

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: A well-developed change management plan should describe the project's process for managing requested project changes, including: • Initiating a change request • Logging and tracking change requests • Assigning change requests for analysis and recommendation • Implementing change request resolution actions (acceptance, delayed acceptance or rejection of change request) • How accepted changes will be integrated into the project control documents (e. g. , schedule and task plan, risk management plan, acquisition plan, etc. ) • Roles and responsibilities in the change management process • Examples of the change request form and the change request log used by the organization's project team

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Question: Think About It What is scope creep? What causes it? What problems does scope creep create, and what can be done about it?

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: Scope creep is the gradual growth of work done under a contract or development plan. It results from well-meaning people – deciding that certain work not originally part of the original plan needs to be done to ensure good quality and performance in a system. Among the problems this unplanned work creates are increased cost and delays. Often the changes occur without the knowledge of the senior-level project managers and do not become apparent until the work effort and cost have grown substantially. Typically, because the increase was not budgeted, it has to be paid for out of internal funds. Project managers have a responsibility to exercise thorough and rigorous discipline at all levels of the project, and use integrated cost and schedule performance analysis to control it.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Application Controls: Application controls help ensure data accuracy, completeness, validity, verifiability and consistency. In this way, data integrity and data reliability can be achieved. Implementing application controls helps ensure system integrity, that systems operate as intended, and that information in the systems is relevant, reliable, secure and available when it is needed.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Application controls are controls over input, processing and output functions. They include methods for ensuring: • Only complete, accurate and valid data are entered and updated in a computer system, • Processing accomplishes the correct tasks, • Processing results meet expectations, • Data are maintained. These controls include edit tests, totals, reconciliations, and identification and reporting of incorrect, missing or exception data. To ensure the proper investigation of exceptions, automated controls should be coupled with manual procedures.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management There are four basic internal control objectives: • Safeguarding of assets • Accuracy and reliability of financial reporting and supporting information • Operational efficiency and effectiveness • Adherence with applicable laws and regulations

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management The first two objectives relate to internal accounting controls in order to provide reasonable assurance that: • Transactions (such as cash receipts and cash disbursements) are executed in accordance with management's general or specific authorization, • Transactions are recorded in a manner that permits preparation of financial statements that conform to generally accepted accounting principles (or other similar criteria) and maintain accountability for assets (such as cash), • Access to assets is allowed only in accordance with management authorization, • Recorded accountability for assets is regularly compared with existing assets (such as bank reconciliations) and action is taken on any differences. The last two objectives – regarding operational efficiency and compliance with regulations – relate to administrative controls which will be covered later.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management The IS auditor's tasks in evaluating controls include: Identifying components and flow – Identifying significant components of the application and flow of transactions through the system, and gaining a detailed understanding of the application by reviewing available documentation and interviewing appropriate personnel Finding strengths and evaluating weaknesses – Identifying application control strengths, and evaluating the impact of the control weaknesses on the development of a testing strategy by analyzing accumulated information Testing controls – Testing controls to ensure functionality and effectiveness by applying appropriate audit procedures Evaluating control environment – Evaluating the control environment by analyzing test results and other audit evidence to determine control objectives were achieved Comparing to standards – Considering operational aspects of the application to ensure efficiency and effectiveness by comparing the system with efficient programming standards, analyzing procedures used and comparing them to management's objectives for the system

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Internal controls are usually a response to user requirements for accuracy, completeness and security within an information system. These controls can be segmented into three classes: • controls over input, • controls over processing, • controls over output. Input controls are the most important and the most numerous. Because most errors are generated during input processing, welldefined and well-established input controls must be used to help meet overall stated control objectives.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Some common input control techniques are: Check digit verification – Check digits are used to review the accuracy of specific fields. Consistency tests – Accuracy is verified by comparing the values of various fields to determine whether the combinations make sense. Validity tests – In some cases fields can take only a limited range of values or must have a predetermined format. Matching the actual value to the allowable values will detect errors. Batch numbering – Processor checks can be made to ensure all transactions are accounted for and processed in a logical order. Control totals – These totals help avoid errors during data entry. Various input fields (e. g. , check amount or quantity received) are added both manually and automatically for comparison. In some cases hash totals are developed for fields that would normally not be added. In either case, both the expected totals and the individual transactions are passed to the application system. The application system then recalculates the totals from the individual records received and compares them to the expected totals. If they do not match, an error has been detected. Transaction counts – Use this method to keep track of the number of transactions that should have been processed by the application system.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Well-designed source documents can increase the speed and accuracy of data recording, control work flow, and organize data in machine-readable form. Source documents can also aid in subsequent reference checking. Source documents should be standardized, preprinted forms that include appropriate headings and instructions. The forms should be easy to use and should: • Group similar fields together to facilitate input, • Include predetermined input codes to reduce errors, • Contain an identifier for research and tracking, • Use boxes to identify field size errors, • Provide a space to record authorization. Source documents should be controlled and procedures implemented to ensure all source documents have been entered and tracked.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Knowledge Review: Match each example to an input control: Input Controls Batch numbering Check digit Consistency test Control total Validity test Examples 1. "State or province" field only accepts letters, not numbers 2. If "country" field indicates "Canada", postal code field must accept six letters and numbers in a specific configuration 3. For each set of transactions being processed, a hash total of the account numbers is provided before input and checked by the system after input. 4. Each day's transactions are assigned a digit beginning with the date and the order in which they were received. 5. The system applies a formula to an account number. The calculation shows that the account number is invalid.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: Match each example to an input control: Batch numbering Each day's transactions are assigned a digit beginning with the date and the order in which they were received. Check digit The system applies a formula to an account number. The calculation shows that the account number is invalid. Consistency test If "country" field indicates "Canada", postal code field must accept six letters and numbers in a specific configuration Control total For each set of transactions being processed, a hash total of the account numbers is provided before input and checked by the system after input. Validity test "State or province" field only accepts letters, not numbers

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Input processing requires that controls be identified to verify that data are accepted into the system correctly and that input errors are recognized and corrected. Errors can occur due to duplication of transactions and inaccurate data entry. These errors can, in turn, greatly impact the completeness and accuracy of the data. Corrections to data should be processed through normal data conversion processes and should be verified, authorized and reentered into the system as a part of normal processing. Input error handling can be processed by: • Rejecting only transactions with errors and processing the rest, • Rejecting any batch containing errors for correction prior to processing, • Holding transactions in error in suspense pending correction.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management After errors are corrected, input control techniques verify that data are accepted into the system correctly. These controls include: Transaction log – Contains a detailed list of all updates. The log can be either manually maintained or provided through automatic computer logging. A transaction log can be reconciled to the number of source documents received to verify that all transactions have been input. Reconciliation of data – Controls whether all data received are properly recorded and processed Documentation – Written evidence of user, data entry and data control procedures Error correction procedures – These include: • Logging of errors, • Timely corrections, • Upstream resubmission, • Approval of corrections, • Suspense file, • Error file, • Validity of corrections. Anticipation – The user or control group anticipates the receipt of data Transmittal log – Documents transmission or receipt of data

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Input data should be validated and edited as close as possible to the time and point of origination. Data validation identifies data errors, incomplete or missing data and inconsistencies among related data items. Edit controls are preventive controls used before data are processed. Where transaction data are input online, controls should detect data errors and permit editing in real time. Error checks and edits should include, as appropriate, authorization, sequence, limit, range, time, validity, reasonableness, existence, completeness, duplicate and logical relationship checks, plus table lookups, key verification and check digits. If input procedures allow a supervisor to override data validation and editing, this should be automatically logged, and the overrides should be reviewed by a manager who did not initiate the override.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Sequence Check: The control number follows sequentially and any out-of-sequence or duplicated control numbers are rejected or noted on an exception report for follow-up purposes. For example, invoices are numbered sequentially. The day’s invoices begin with 12001 and end with 15045. If any invoice larger than 15045 is encountered during processing, that invoice would be rejected as an invalid invoice number.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Limit Check: Data should not exceed a predetermined amount. For example, payroll checks should not exceed US $4, 000. If a check exceeds US $4, 000, the data would be rejected for further verification/authorization.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Range Check: Data should be within a predetermined range of values. For example, product type codes range from 100 to 250. Any code outside this range should be rejected as an invalid product type.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Validity Check: Programmed checking of the data validity in accordance with predetermined criteria. For example, a payroll record contains a field for marital status and the acceptable status codes are M or S. If any other code is entered, the record should be rejected.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Reasonableness: Input data are matched to predetermined reasonable limits or occurrence rates. For example, a widget manufacturer usually receives orders for no more than 20 widgets. If an order for more than 20 widgets is received, the computer program should be designed to print the record with a warning indicating that the order appears unreasonable.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Table Look-Ups: Input data comply with predetermined criteria maintained in a computerized table of possible values. For example, the input clerk enters a city code of 1 to 10. This number corresponds with a computerized table that matches the code to a city name.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Existence Check: Data are entered correctly and agree with valid predetermined criteria. For example, a valid transaction code must be entered in the transaction code field.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Key Verification: The keying-in process is repeated by a separate individual using a machine that compares the original keystrokes to the repeated keyed input. For example, the worker number is keyed twice and compared to verify the keying process.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Check Digit: A numeric value that has been calculated mathematically is added to data to ensure the original data have not been altered or are incorrect, but valid, value substituted. This control is effective in detecting transposition and transcription errors. For example, a check digit is added to an account number so it can be checked for accuracy when it is used.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Completeness: A field should always contain data rather than zeros or blanks. A check of each byte of that field should check be performed to determine that some form of data, not blanks or zeros, is present. For example, a worker number on a new employee record is left blank. This is identified as a key field and the record would be rejected, with a request that the field be completed before the record is accepted for processing.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Duplicate Check: New transactions are matched to those previously entered to ensure they have not already been entered. For example, a vendor invoice number agrees with previously recorded invoices to ensure the current order is not a duplicate and, therefore, the vendor will not be paid twice.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Logical: If a particular condition is true, then one or more additional conditions or data input relationships may be required to be true and consider the input valid. For example, the hire date of an employee check may be required to be more than 16 years past his/her date of birth.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Processing controls are designed to ensure data in a file or database are maintained intact until they are changed by authorized modification or processing. The following control techniques can help ensure the data remain complete and accurate: • Manual recalculations, • Editing, • Run-to-run totals, • Programmed controls, • Reasonableness verification of calculated amounts, • Limit checks on calculated amounts, • Reconciliation of file totals, • Exception reports.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Manual Recalculations: Sample of transactions may be recalculated manually to ensure processing is accomplishing the anticipated task. An edit check is a program instruction or subroutine that tests the accuracy, completeness and validity of data. It may be used to control input or later processing of data. Run-to-run totals provide the ability to verify data values through the stages of application processing. Run-to-run total verification ensures data read into the computer were accepted and then applied to the updating process. Programmed Controls: Software can be used to detect and initiate corrective action for errors in data and processing. For example, if the incorrect file or file version is provided for processing, the application could display messages instructing that the proper file and version be used.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Application programs can verify the reasonableness of calculated amounts. The reasonableness can be tested to ensure appropriateness to predetermined criteria. Any transaction that is determined to be unreasonable may be rejected pending further review. Limit Checks on Calculated Amounts: An edit check can provide assurance, through the use of predetermined limits, that calculated amounts have been keyed correctly. Any transaction exceeding the limit may be rejected for further investigation. Reconciliation of file totals should be performed on a routine basis. Reconciliations may be performed through the use of a manually maintained account, a file control record or an independent control file.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management An exception report is generated by a program that identifies transactions or data that appear to be incorrect. These items may be outside a predetermined range or may not conform to specified criteria.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Exercise Instructions: Match each item to its corresponding description. Control Item Check digit Edit check Exception report Programmed controls Transaction log Description • Program instruction that tests the accuracy, completeness and validity of data • Software that detects and initiates corrective action for errors in data and processing • Generated by a program that identifies transactions that appear to be incorrect • Numeric value added to data to ensure original data have not been altered • Record of all updates to verify input against source documents

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: Check digit Numeric value added to data to ensure original data have not been altered Edit check Program instruction that tests the accuracy, completeness and validity of data Exception report Generated by a program that identifies transactions that appear to be incorrect Programmed controls Software that detects and initiates corrective action for errors in data and processing Transaction log Record of all updates to verify input against source documents

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Data File Processing Controls: There are four basic categories of data files or database tables: System control parameters – Entries in these files can change system operations and may affect system controls such as the tolerance allowed before an exceptional transaction is reported or blocked. Changes to these files should be controlled like changes to the program. Standing data – These are data that do not change often (such as customer names and addresses) and are referred to during processing. Any entry or maintenance on these data should be authorized and a report of changed data generated. Master data/balance data – Any adjustment to running balances and totals that are updated by transactions must be subject to strict approval and review controls. Because there may be financial reporting implications for any change, an audit trail is necessary. Transaction files – These should be controlled with validation checks, control totals, exception reports, etc.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Output controls are intended to make sure the data delivered to users are presented, formatted and delivered securely and consistently. Output controls are needed for: • Storing negotiable, sensitive and critical forms • Generating negotiable instruments, forms and signatures • Managing report distribution and retention • Detecting and managing errors • Retaining records Secure storage must be provided for any negotiable, sensitive or critical forms. These should be logged and routinely reconciled to inventory, and any discrepancies promptly researched.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management There should be specific parameters regarding output reports that include: • Verifying reports are complete and delivered according to schedule, • Logging all reports before distribution, • Controlling access to print spools handling reports, • Printing reports under secure, controlled conditions, • Arranging for secure disposal, • Requiring authorization for online access, • Logging evidence that a recipient has received the report.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Business process control assurance involves evaluating controls at the process and activity level. These may be a combination of management, programmed and manual controls. Among the elements that should be considered in reviewing business process control assurance are: • Process maps • Process controls • Assessing business risks within the process • Benchmarking with best practices • Roles and responsibilities • Activities and tasks • Data restrictions The controls need to be evaluated for both the general application and the specific business process owner. The latter include factors such as establishing and enforcing proper security procedures, segregation of duties and periodic review of access.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management To acquire an understanding of an application's development, the following documents should be reviewed as part of auditing the application controls: • System development methodology documents, including cost-benefit analysis and user requirements • Functional design specifications, which provide a detailed explanation of the application. The key control points should be noted. • Program changes, with evidence for authorization for any change, along with cross-reference to the source code • User manuals, to determine how the user is utilizing the application. Often control weaknesses can be noted from the review of this document. • Technical reference documentation, including any vendor-supplied technical manuals for purchased applications in addition to any in-house documentation. Access rules and logic usually are included in these documents.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Data integrity testing looks at the accuracy, completeness, consistency and authorization of data held in a system. It is similar to the testing used for input control. Data integrity tests are designed to detect failures in input or processing controls. To ensure the integrity of the accumulated data in a file, the data need to be checked regularly. Commonly, when comparing against authorized source documentation, only a portion of the file is checked at a time. Because the entire file is regularly checked in cycles, the control technique is termed cyclical checking.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Data integrity is commonly tested with referential integrity tests. Referential integrity tests define certain relationships between entities in a database that the database management system (DBMS) must maintain. Whenever two or more relations are related through referential constraints (primary and foreign key), the references must be kept consistent when there are updates, deletions or insertions.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management An alternative to the SDLC approach is "agile development. " This strategy uses smaller, time-limited subprojects or iterations, so that each iteration provides a basis for planning the next iteration. This approach features: • Planning the development in detail only for an iteration rather than planning subsequent development phases far out in time, • Replanning the project at the end of each iteration including reprioritizing requirements, identifying any new requirements and determining within which release delivered functionality should be implemented, • Relying to a greater extent on tacit knowledge (what is brought to the project by users), in contrast to external knowledge that is captured in project documentation, • Methods of promoting teamwork and disseminating tacit knowledge so that teams (composed of both business and technical members) can readily discuss progress and issues during daily, time-limited meetings.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management The "agile development" approach features, cont. : • Sometimes using pair-wise programming, in which two individuals code the same part of the system, as a way of sharing knowledge and as a quality check • Changing the project manager's role to primarily focus on facilitation and serving as an advocate for the project, rather than on project planning, allocating tasks and monitoring progress. • Planning and control responsibilities devolve to the team. This adaptive approach to requirements does not concentrate on managing to a requirements baseline. Its focus is to quickly prove an architecture by building actual functionality, instead of formally defining software and data architecture in the early stages with increasingly more detailed models and descriptions. Rather than focus on defined and repeatable processes, agile development performs and adapts its development based on frequent inspections. It assumes limits to defect testing, but uses a frequent-build test cycle to validate functions and correct problems in the next subproject before incurring much time and cost.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Question: Think About It What are the benefits of agile software development practices?

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: The benefits of agile development practices include: • Providing usable functionality sooner. Delivering software incrementally gives users the benefit of the functionality sooner. • Implementing the most important features first. Users choose the delivery order. • Receiving early and frequent feedback. Each iteration provides input for the next one and misunderstandings can be identified earlier. • Achieving measurable progress. Frequent delivery of working code means progress is measurable. • Mitigating risks: The risk of building (or doing) the wrong thing The risk of building the right thing, but doing it poorly The risk of being stuck in an endless cycle of design updates and reviews because of changing requirements or high levels of complexity

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Prototyping (also called heuristic or evolutionary development) creates a system using controlled trial-and-error procedures. This reduces risks associated with system development by allowing the developer and user to assess and respond to risks at each step. In this way, it combines the systematic step process of SDLC with an iterative process. Prototyping can also trim the time required to deploy systems. It usually focuses initial development on the reports and screens – the elements of the system most visible to end users, and in this way provides a high-level view of a working model of the proposed system within a short time. Prototyping employs faster development tools (such as fourth-generation techniques) that may include: • Nonprocedural languages for database query, • Report generation, • Screen interaction and definition, • Configuration management.

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Question: Think About It What are some of the disadvantages of prototyping?

CISA REVIEW Chapter 3 – Systems Infrastructure and Lifecycle Management Answer: The disadvantages of prototyping include: • Having to expand the prototype to handle transaction volumes, client-server network connectivity, and backup and recovery procedures, and provide for auditability and control before implementation, • Using prototyping alone for a large system may result in the same problems encountered in conventional development efforts (such as poor quality, poor maintainability and low user acceptance) unless a design strategy for the system is created first, • During development, functions or extras may be added that are not part of the initial requirements document and can result in a functionally rich but inefficient system, • Changes in design and requirements occur rapidly resulting in change control weaknesses. Because changes are not well documented or approved, the problem can escalate to a point at which the system is not maintainable.