Optimizing Performance Management Gina Fisk LANL Senior Cyber

Optimizing Performance Management Gina Fisk, LANL Senior Cyber Security Manager gina@lanl. gov � n Adaptive Metrics n n Fitness Functions n n n Develop metrics that determine how well we are adapting to our ever-changing environment. Identify dependencies and requirements for optimum productivity around the Laboratory. Measure the impact of a localized failure of one entity across the entire organization. Balanced Score Card n n Review our program from a balanced perspective. Provide metrics by which we can manage. UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA Strategy Measures Budgeting Initiatives

Starting Point – Remove the Clutter n Remove metrics that we can’t use to manage our information security program. n n How many customers called our help desk. How many connections were deflected by our firewall. How many times our network was scanned, etc. Bin the remaining metrics into the BSC framework for a Phase I BSC. n n Financial. Customer. Internal Processes. Learning and Growth. UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA 2

Determine Impacts of Failure n Conduct IT Impact Analysis n Determine the cost to an organization if various IT services failed for variable lengths of time. n n Calculate Impact Rating for each IT Service. n n n Network, Email, local storage, etc. 1/n, where n is the average number of days until an organization has lost 100% of productivity. Calculate the Daily Monetary Impact of the Loss of that IT Service for an organization. Calculate the overall productivity cost for the Laboratory as a whole based on that loss. UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA 3

Focus Areas n From IT Impact Analysis results, identify IT Services with largest impacts to productivity. n n n n Loss Loss of of of Accreditation of systems local network access Email Oracle Internet access Goals that the CIO and CISO had set for the organization in the Strategy Map. Develop metrics based on these focus areas and develop Phase II of the BSC. UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA 4

Information Security Strategy Map Maximize mission enablement by balancing risk and value (F 1) Operational Excellence F 2. Maximize operational efficiency Competitive Advantage F 3. Minimize IT enterprise risk F 4. Facilitating acquisition of new business through best-in-class IT security execution Competency C 1. “Understand consistently deliver what I need” Contribution C 2. “Keep me out of security and compliance trouble” C 3. “Establish a positive reputation which will help me with my customers” C 4. “Become a trusted partner by helping me solve my challenging problems” Achieve Operational Excellence Create and Support Internal Programs and External Partners Deliver Innovative Security Solutions IP 1. Streamline compliance program to achieve 100% of scheduled accreditations IP 2. Optimize operations to reduce KTLO by 10% per BU IP 3. Enhance performance through implementation and management of service agreements IP 4. Mature IT governance processes and increase partner participation IP 5. Build a structured, transparent and collaborative regulator relationship IP 6. Promote transparency and performance through holistic metrics program IP 7. Propose and deliver business -enabling information security solutions IP 8. Mature IT risk program to drive security, portfolio, and governance decisions IP 9. Enhance red network monitoring and vulnerability management LG 1. Attract, develop, and retain highly skilled security professionals LG 2. Develop risk-focused and customer-centric culture UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA FINANCIAL PERSPECTIVE LG 3. Align employee training with strategic initiatives CUSTOMER PERSPECTIVE INTERNAL AND PROCESS PERSPECTIVE LEARNING AND GROWTH PERSPECTIVE 5

Balanced Score Card Note: BSC target performance scores are represented here for explanatory purposes only Customer (C 1 -C 4) Financial (F 1 -F 4) On-time rate of Security unit Enterprise costs accreditation risk rating s Business impact of incidents Lower unit costs <25 hrs/Q 100% on time Maintain. 3 rating Communicat Customer Compliance ion Support Projects on- Cyber PBI time/budget ratings >80% survey scores <10% >95% green variance >70% survey scores >80% survey scores Program Input Time per Customer accreditation Satisfaction >90% governance participatio n >95% CA/avg times >80% survey scores Target Initiative Internal Processes (IP 1 -IP 7) Learning and Growth (LG 1 -LG 3) Training Planned role Attrition roadmap rotations reduction <10% schedule variance >=1/Q Reduced attrition rate Strategic training X X CSIPP: AOE: Opex AOE: SLA DISS: AOP DISS: BP unplanned reduction performance risk mapping tied to risk work >50% training mapped to initiatives X X >=2. 5% Q/Q <10% variance <=3/Q >=80% >=30% key processes DISS: Red capabilities Positive trend Target Initiative Hits target. Initiative on track Short of target. Initiative recoverable UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA Failed process. Initiative not recoverable Target not defined. No initiative 6

n n Fitness functions measure the overall health of an organization by measuring not only performance, but also the performance of those organizations on which we are dependent to achieve our goals. If the performance of one of the dependencies fails, there are ramifications throughout the entire organization. Using the fitness scores of dependent organizations, we can measure the impact of a localized failure of one entity across the entire organization, providing valuable measurements of the actual cost of security incidents, network outages, etc. We can trend these scores to evaluate performance at various levels of the organization. UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA Contracts Science and Engineering Production Cycles Backups and Storage Visualization Services Reliant Organizations n Publications Scientific Computing Network Services Identity Management System Administration Dependencies Fitness Functions Patents Core Services Security Infrastructure C&A Physical Infrastructure 7

Example Fitness Function Framework n n n Fiscal Responsibility (weight: 20%) n Milestones and deliverables (quality, timeliness) n Expenditures (percentage over budget) Customer Productivity (weight: 15%) n Services maximize productivity around organization (uptime, etc) Customer Orientation (weight: 15%) n Responsiveness to the customer (SLAs, etc) Improving Security (weight: 15%) n Progress made toward improving security against our current threat environment (hardening tools, etc) Institutional Responsibilities (Weight: 20%) n PBI deliverables and reporting (quality, timeliness) n CAP deliverables and reporting (quality, timeliness) n Metrics reporting (quality, accuracy) Goal-Based Initiatives (weight: 10%) n Progress made against organizational goals. UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA 8

Fitness Function Example Note: Fitness scores are represented here for explanatory purposes only n n n n Fiscal Responsibility (weight: 20%). 89 n Timeliness of deliverables and milestones: . 83 n % of projects +/- 10% of budget allocation: . 95 Laboratory Productivity (weight: 15%). 98 n Uptime of service: . 98 Customer Orientation (weight: 15%). 89 n Customer Satisfaction Rating. 89 Improving Security (weight: 15%) **. 56** n Progress made toward improving security against our current threat environment (hardening tools, etc). 56 Institutional Responsibilities (Weight: 25%). 68 n PBI deliverables and reporting (quality, timeliness). 90 n CAP deliverables and reporting (quality, timeliness). 75 n Metrics reporting (quality, accuracy). 40 Goal-Based Initiatives (weight: 10%). 98 n Progress made against initiatives. . 98 FITNESS SCORE: . 806 UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA 9

Fitness Score Trends n Fitness scores allow us to watch for trends and to manage by our metrics. n See how major changes affect our performance from month to month. n n n Change in Management Change of Platform Change of Vendor, etc. UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA 10

Adaptive Metrics n n n IT Impact Analysis provide us with costs of the failures of IT Services. We have the data on our ever changing threat environment. The fitness functions allow us include “moving target” metrics, which change each month, to measure our performance against our current threat environment. UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA 11

**Adaptive Metric Example Note: Threats and budgets represented here for explanatory purposes only n “Improving Security (weight: 15%)” n n Identify top threats for the month. n Phishing, Windows vulnerability, Oracle vulnerability. Calculate cost of failure of these services across the organization per month. n Email: $200 K, Windows: $500 K, Oracle: 800 K n Overall Budget: 10% in jeopardy Review % of security effort we are placing on these areas ($$$ spent). n Email: 5%, Windows: 31%, Oracle: 20% Weight the fitness function by how responsive we are to these areas. n 56% of our budget is spent on our top threat areas. UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA 12

Managing by the Metrics n n n Our budget, metrics, and initiatives are actionable and directly tied to our goals. Our use of the Balanced Score Card helps us ensure uniform management of our business. Our use of the Fitness Functions help us trend our metrics effectively and monitor the major changes. n n We can trend our components individually or as a whole, organizationally or institutionally. Our use of Adaptive Metrics keep our outlook fresh and defendable. UNCLASSIFIED Operated by Los Alamos National Security, LLC for NNSA 13