Risk Management Principles of risk Types of risk

  • Slides: 32
Download presentation
Risk Management: Principles of risk, Types of risk and Risk strategies by Erlan Bakiev,

Risk Management: Principles of risk, Types of risk and Risk strategies by Erlan Bakiev, Ph. D.

IT Security Metrics A Practical Approach to Measuring Information Security: Measuring Security at the

IT Security Metrics A Practical Approach to Measuring Information Security: Measuring Security at the System Level

Introduction

Introduction

IT Security Metrics Training Audience: Federal IT security personnel with GISRA reporting responsibilities Goal:

IT Security Metrics Training Audience: Federal IT security personnel with GISRA reporting responsibilities Goal: To train Federal IT security personnel how to develop metrics that they can use immediately to assist with GISRA reporting Duration: 3 hours

Objectives After completing this workshop, you will be able to: • Identify why metrics

Objectives After completing this workshop, you will be able to: • Identify why metrics are important for IT security • Identify the relationship among GISRA, NIST SP 800 -26, and IT Security Metrics • Describe metrics development process • Apply metrics development process by completing a Metrics Form for one of the OMB GISRA reporting requirements for FY 02 • Identify metrics-related Roles and Responsibilities • Describe how to implement a Metrics Program

Metrics Development

Metrics Development

In this section, you will: • Learn the definition and characteristics of IT Security

In this section, you will: • Learn the definition and characteristics of IT Security Metrics • Identify the difference between Performance Goals and IT Security Metrics • Learn the seven-step IT Security Metrics Development Process • Discover the types of information and insights that can be gained from IT Security Metrics • Complete three examples of IT Security Metrics

What are IT Security Metrics? IT Security Metrics are tools that facilitate decision making

What are IT Security Metrics? IT Security Metrics are tools that facilitate decision making and accountability through collection, analysis, and reporting of relevant performance data. • Based on IT security performance goals and objectives • Quantifiable • Obtainable/feasible to measure • Repeatable • Provide relevant performance trends over time • Useful in tracking performance and directing resources

Why Measure IT Security? Categories Regulatory Requirements • Government Information Security Reform Act (GISRA)

Why Measure IT Security? Categories Regulatory Requirements • Government Information Security Reform Act (GISRA) • Clinger-Cohen Act Benefits • Satisfy regulatory requirements • Government Paperwork Reduction Act (GPRA) Financial • Measure successes and failures of past and current security investments • Enable investment targeting to identified areas in need • Ensure best value from security • Justify future investments Organizational • Improve accountability to stakeholders • Ensure appropriate level of mission support • Determine IT security program effectiveness • Improve customer confidence • Build confidence in leadership • Demonstrate improvement to stakeholders • Play key role in initiating improvement actions based on performance trends • Enable relevant, realistic, appropriate security procedure modification

IT Security Metrics should support IT security goals and objectives IT Security Performance Goals

IT Security Metrics should support IT security goals and objectives IT Security Performance Goals identify desired results of system security program implementation. IT Security Performance Objectives enable accomplishment of goals by: • Identifying strategic practices, defined by security policies, procedures, and controls • Directing consistent implementation of policies and procedures across the organization IT Security Metrics monitor accomplishment of goals and objectives by: • Quantifying the level of implementation of security control objectives and techniques for a system and the effectiveness and efficiency of the controls within the organization • Using analysis of collected IT Security Metrics to determine adequacy of security activities and make appropriate business decisions

Exercise: Performance Goal or IT Security Metric? . Statement Program Officials understand the risk

Exercise: Performance Goal or IT Security Metric? . Statement Program Officials understand the risk to systems under their control and determine the acceptable level of risk. . Percentage of system security plans that are updated annually. Duties are separated to ensure least privilege and individuals accountability. Percentage of systems with automated virus updating. Data integrity and validation controls are used to provide assurance that the information has not been altered and the system functions as intended. Performance Goal IT Security Metric

Metrics development is a seven step process

Metrics development is a seven step process

The focus of the metrics program depends on IT security program maturity

The focus of the metrics program depends on IT security program maturity

Stakeholders and Interests • Anyone within an organization is an IT security stakeholder, though

Stakeholders and Interests • Anyone within an organization is an IT security stakeholder, though some functions have a greater stake than others: – – – CIO Program Manager/System Owner Security Program Manager Resource Manager Training/Human Resources Personnel • Each stakeholder needs a set of metrics that provides a view of the organization’s IT security performance within their needs, for a total of no more than 10 -20 metrics per stakeholder • Many IT Security Metrics can be created to measure each aspect of the organization’s IT security. Selecting the most critical elements of the organization’s IT security program during metrics prioritization will make the program manageable and successful

IT Security Performance Goals and Objectives IT security performance goals and objectives are expressed

IT Security Performance Goals and Objectives IT security performance goals and objectives are expressed in the form of high level policies and requirements in many laws, regulations, policies, and guidance that describe the dimensions of an effective IT security program: • Clinger Cohen Act • Presidential Decision Directives 63 • Government Information Security Reform Act (GISRA) • OMB Circular A-130, Appendix III • Critical Elements within NIST Special Publication 800 -26 • Federal Information Security Compliance Audit Manual (FISCAM)

IT Security Policies, Guidance, and Procedures Some Federal guidance and agency-specific policies and procedures

IT Security Policies, Guidance, and Procedures Some Federal guidance and agency-specific policies and procedures provide more detailed information specific to the agency: • NIST SP 800 -12, 800 -14 • Agency-specific policy and guidance • Subordinate questions within NIST Special Publication 800 -26

System Security Program Implementation System Security implementation includes: • Processes and procedures in place

System Security Program Implementation System Security implementation includes: • Processes and procedures in place • Existing capabilities • Areas for improvement • Existing metrics • Existing data sources that can be used to derive metrics data These may be documented in the following sources: • System Security Plans • OMB Plan of Actions and Milestones (POA&M) reports • Latest GAO and IG findings • Tracking of security-related activities

Metrics can describe three aspects of IT security program operations and management 7 Level

Metrics can describe three aspects of IT security program operations and management 7 Level of Implementation: Most organizations are new to measuring IT security with performance metrics. They will begin by measuring level of implementation of its security policies and procedures. Instituting a metrics program is the first step to process maturity. Business Impact can be measured through correlation analysis once an organization’s processes are selfregenerating and measurement data gathering is transparent. Security Program Effectiveness and Efficiency: As an organization’s process maturity increases and performance data becomes more readily available, metrics will focus on program efficiency and effectiveness.

It is important to record the specifics of each metric for the purposes of

It is important to record the specifics of each metric for the purposes of data analysis and possible metric reuse Metric Describes the overall functionality obtained by collecting the metric Purpose Defines the metric by describing the quantitative measurement(s) provided by the metric Frequency Implementation Evidence Formula Frequency Target Specific questions that will need to be answered via survey or through automatic data gathering to be able to calculate the metric Proposes the periods for collection of data to be used for measuring changes over time. Suggested time periods are based on likely updates occurring in the applicable process Formula Describes the calculation to be performed that results in a numeric expression of a metric Data Source Lists the location of the data to be used in calculating the metric Indicators Provides information about the meaning of the metric and its performance trend. Proposes possible causes of trends, identified through measurement, and points at possible solutions to correct observed shortcomings

Metrics can help identify causes of poor performance, including: Areas Examples Resources Insufficient human,

Metrics can help identify causes of poor performance, including: Areas Examples Resources Insufficient human, monetary, or other resources can be causing negative performance trends Training Lack of appropriate training for the personnel installing, administering, maintaining, or using the systems System Upgrades Security patches that have been removed during the operating system upgrades Configuration Management Practices New or upgraded systems that are not configured with all required security settings and patches Software Compatibility Security patches or upgrades that are incompatible with software applications supported by the system Awareness and Commitment Lack of management awareness and/or commitment to security Policies and Procedures Lack of policies and procedures that are required to ensure existence, use, and audit of required security functions Architectures Poor system and security architectures that make systems vulnerable Inefficient Processes Inefficient planning processes that influence the metrics (including communication processes necessary to direct organizational actions)

How does NIST SP 800 -26 relate to metrics? Performance Goal Critical Element: 13.

How does NIST SP 800 -26 relate to metrics? Performance Goal Critical Element: 13. 1. Have employees received adequate training to fulfill their security responsibilities? Subordinate Questions: Performance Objectives Implementation Evidence 13. 1. 1 Have employees received a copy of the Rules of Behavior? 13. 1. 3 Is there a mandatory annual refresher training? 13. 1. 4 Are methods employed to make employees aware of security, i. e. , posters, booklets? 13. 1. 5 Have employees received a copy of or have easy access to agency security procedures and policies? Rules of Behavior are included in training Employees sign employee agreements stating that they have read and understood rules of behavior New employee training is conducted Policies and Procedures discussion included in training Annual refresher training is conducted % of employees who received annual refresher training Metrics % of employees who signed employee agreements % of new employees who underwent security awareness training . . .

Metrics Development Criteria: What is a Good Metric? Based on IT security performance goals

Metrics Development Criteria: What is a Good Metric? Based on IT security performance goals and objectives: NIST SP 800 -26 Critical Elements and Subordinate Questions are used to derive performance goals and objectives Quantifiable: Metrics should yield quantitative rather than qualitative information to increase the objectivity and validity of data Obtainable/Feasible to measure: Metrics data should be available or easily collected through interviewing or by accessing data repositories. If a metric requires significant modification of agency processes or implementing a new tool, data collection may not be feasible at this time Repeatable: Measurements should be able to be repeated in a standard way at predetermined intervals to identify trends or identify if positive changes have occurred as a result of corrective actions Provide relevant performance trends over time: Repeated measurements reveal change in a timely manner Useful in tracking performance and directing resources: Metrics should be useful to stakeholders and should yield information that is important in financial decision making

Metrics Program Implementation

Metrics Program Implementation

In this section, you will: • Receive an introduction to the IT Security Metrics-related

In this section, you will: • Receive an introduction to the IT Security Metrics-related roles and responsibilities • Learn the steps involved in IT Security Metrics program implementation by learning the process and following an example through the process

Multiple success factors can influence quality and sophistication of IT Security Metrics Ensure that

Multiple success factors can influence quality and sophistication of IT Security Metrics Ensure that IT Security Metrics Program is manageable: • Use no more than 10 -20 metrics at a time, based on current priorities • Phase old metrics out and phase new metrics in when performance targets are reached or when requirements change Ensure acceptable quality of data: • Data collection methods and data repositories should be standardized • Events must be reported in a standard manner throughout the organization and the results of such reports need to be stored in the data repository

Multiple success factors can influence quality and sophistication of IT Security Metrics Obtain organizational

Multiple success factors can influence quality and sophistication of IT Security Metrics Obtain organizational acceptance: • Metrics need to be validated with organization’s stakeholders within headquarters and in the field • Metrics should be vetted through appropriate approval channels Ensure that metrics are useful and relevant: • Useful data should be collected • Not all data are useful

Metrics-related roles and responsibilities are dispersed throughout an organization Responsibility for Organizational Acceptance of

Metrics-related roles and responsibilities are dispersed throughout an organization Responsibility for Organizational Acceptance of Metrics Program Responsibility for Metrics Data Collection and Data Accuracy

Each organization will implement a metrics program specific to its needs • Tailor to

Each organization will implement a metrics program specific to its needs • Tailor to organization and business processes • Identify IT Security Metrics-related stakeholder roles and responsibilities • Lay out required infrastructure changes, such as creation of web-based data collection tools and of new data repositories • Identify required modifications of the current data sources • Define data reporting formats

Output from standard security activities can be used to quantify IT security performance •

Output from standard security activities can be used to quantify IT security performance • • • Incident Handling Testing Network Management Audit Logs Network and System Billing Configuration Management IT Security Metrics data collection must be as transparent and non-intrusive as possible. Contingency Planning Training Certification and Accreditation

IT Security Metrics Program Implementation Process • Identify stakeholders • Determine goals / objectives

IT Security Metrics Program Implementation Process • Identify stakeholders • Determine goals / objectives • Review existing metrics • Develop new metrics • Identify data collection methods and tools • Collect metrics • Analyze collected data • Conduct gap analysis - Identify gaps between actual and desired performance • Identify reasons for undesired results • Identify areas requiring improvement • Determine range of corrective actions • Select most appropriate corrective actions • Prioritize corrective actions based on overall risk mitigation goals • Develop cost model - Project cost for each corrective action • Perform sensitivity analysis • Develop business case • Prepare budget submission • Track progress • Report as required • Management • Operational • Technical • Budget allocated • Available resources prioritized • Resources assigned

Process Implementation Example Lack of IT security refresher training may be causing weak passwords,

Process Implementation Example Lack of IT security refresher training may be causing weak passwords, identified by a password cracker that is run regularly. Employees should be required to take annual IT security refresher training as part of their annual review process. Since annual refresher training has ceased, the number of weak passwords has increased by 50%. Since the training was re-instituted, the percentage of weak passwords has decreased by 40% Annual refresher training, an operational control, is instituted. A budget submission detailing metrics findings related to annual IT security refresher training was submitted, and funding received. Only 5% of employees receive annual IT security refresher training.

Summary • Discussed why Metrics are important for IT security • Obtained understanding of

Summary • Discussed why Metrics are important for IT security • Obtained understanding of the relationship between GISRA, NIST SP 800 -26, and IT Security Metrics • Described the Metrics Development Process • Created metrics to be implemented at a system level through applying metrics development process • Discussed metrics-related Roles and Responsibilities • Described how to implement a Metrics Program