SP i RE security Security Measures Metrics Pete

SP i RE security Security Measures & Metrics Pete Lindstrom, CISSP Research Director Spire Security, LLC www. spiresecurity. com petelind@spiresecurity. com © 2004 Spire Security, LLC. All rights reserved.

Security Metrics I Security Metrics (Part 1): Building the Framework There are obvious benefits to charting and quantifying the success of your security program. But where do you begin? This session -- part 1 of a 2 -part miniworkshop -- outlines a practical approach to security metrics that links standard business practices with security functions. Find out from Information Security magazine contributing editor, Pete Lindstrom, Research Director for Spire Security, how to build a rock-solid foundation based on a model known as the "Four Disciplines of Security Management. " Then learn about the elements of a cohesive security metrics program from a functional and resource-usage perspective. Plus, you leave with a solid understanding of the relative utility metrics for productivity, process efficiency, cost effectiveness and risk management. © 2004 Spire Security. All rights reserved. 2

What is the Four Disciplines Model? ª A way to think about security o High-level without losing clarity o Detailed enough for technical folks o Identifies relationships ª A taxonomy of objectives, functions, activities, and products. ª A framework for security measurement. © 2004 Spire Security. All rights reserved. 3

Introducing the Four Disciplines Identity Mgt: Managing Users and other sources Trust Mgt: Designing security policy and process 2 3 1 Threat Mgt: Monitoring activities and events 4 © 2004 Spire Security. All rights reserved. Vuln. Mgt: Hardening the systems 4

1. Harden Systems © 2004 Spire Security. All rights reserved. 5

Vulnerability Mgt Functions ª Evaluate and harden configurations o By platform ª Identify and remediate vulnerabilities o Software bugs ª Configure firewalls / other access control ª Reduce/filter anomalous traffic © 2004 Spire Security. All rights reserved. 6

2. Identify/Manage Users © 2004 Spire Security. All rights reserved. 7

Identity Management Functions ª Validate user information ª Create/modify user accounts and privileges ª Disable/delete user accounts ª Change/reset passwords ª Validate sessions ª Authorize access © 2004 Spire Security. All rights reserved. 8

3. Design/Strengthen Processes © 2004 Spire Security. All rights reserved. 9

Trust Management Functions ª Create/modify user policies ª Create/modify system policies - technical baselines ª Design security architecture ª Design/implement controls to prevent sniffing or copying data. ª Design/implement controls to prevent modifying data. © 2004 Spire Security. All rights reserved. 10

4. Monitor Environment © 2004 Spire Security. All rights reserved. 11

Threat Management Functions ª Identify anomalous activities o Monitor network and components o Aggregate alerts and logs o Collect physical information ª Manage/resolve incidents ª Incident response - take corrective action ª Conduct forensic analysis of systems/data © 2004 Spire Security. All rights reserved. 12

Putting It All Together © 2004 Spire Security. All rights reserved. 13

Q 1: Most Important? Which Discipline is most important to a strong security program? 1. Vulnerability Management (firewalls, vuln assess, patch) 2. Identity Management (provision, acct mgt, authent. ) 3. Trust Management (policies, tech guides, crypto) 4. Threat Management (monitor, incident, forensics) © 2004 Spire Security. All rights reserved. 14

Q 2: Most Time? Which Discipline does your organization spend the most time on? 1. Vulnerability Management (firewalls, vuln assess, patch) 2. Identity Management (provision, acct mgt, authent. ) 3. Trust Management (policies, tech guides, crypto) 4. Threat Management (monitor, incident, forensics) © 2004 Spire Security. All rights reserved. 15

Fundamental Security Elements Activities: Four Disciplines Time: Hr/Day Month/Yr Resources: User accts, systems, apps © 2004 Spire Security. All rights reserved. People: Departments Admins Costs: Salaries, Consulting HW, SW, Maint. 16

Types of Metrics ª Process Effectiveness – doing things right. (measure quality) ª Staff Productivity – people doing more things. (measure volume) ª Cycle Time – transaction time. (measure process efficiency) ª Staff Efficiency – people doing things faster. (people / transaction / time) ª Cost Effectiveness – transaction costs. (cost / activity) © 2004 Spire Security. All rights reserved. 17

Process Effectiveness Metrics “doing things right” Key Elements: • Activities • errors Examples: • Acct request errors • Remediation errors • False alarm rate • Policy exceptions © 2004 Spire Security. All rights reserved. error rates 18

Process Effectiveness ª Measure quality by identifying error rates of activities ª Identity Management o User account request errors ª Vulnerability Management o Vulnerabilities not remediated ª Threat Management o Improper incident management ª Trust Management o Policy violations © 2004 Spire Security. All rights reserved. 19

Staff Productivity Metrics “people doing more things” Elements: • People • Activities Examples: • Accts person • Vulns person • Patches person © 2004 Spire Security. All rights reserved. 20

Staff Productivity ª Productivity and workload for all manual activities (activities/people) ª Identity Management o o o Requests per administrator Account disablements per admin Password resets per admin ª Vulnerability Management o Vulnerabilities resolved per administrator ª Threat Management o Incidents person ª Trust Management o Policy changes person © 2004 Spire Security. All rights reserved. 21

Cycle Time Metrics avg “time to perform activity x” Elements: • Time • Activities Examples: • Accts per month • Vulns fixed per month • Patches per month © 2004 Spire Security. All rights reserved. 22

Cycle Time ª Process efficiency ª Identity Management o User account request time to complete ª Vulnerability Management o Remediation time to complete ª Threat Management o Incident response time to complete ª Trust Management o Policy creation time to complete © 2004 Spire Security. All rights reserved. 23

Staff Efficiency Metrics “people doing things” quicker Admins by Department 2000 Hours per FTE © 2004 Spire Security. All rights reserved. Elements: • People • Activities • Time Examples: • Accts person/hr • Vulns person/hr • Patches person/hr 24

Staff Efficiency ª Combines staff productivity and cycle time metrics. ª Identity Management o User account requests completed person per day/week/month ª Vulnerability Management o Vulnerabilities remediated person per day/week/month ª Threat Management o Incidents closed person per day/week/month ª Trust Management o Policies reviewed person per day/week/month © 2004 Spire Security. All rights reserved. 25

Cost Effectiveness Metrics Cheaper transactions Elements: • Activities • Costs Examples: • Cost per acct • Cost per vuln fixed • Cost per patch © 2004 Spire Security. All rights reserved. 26

Cost Effectiveness ª Dollars/activities; dollars/resources; dollars/demographics ª Identity Management o o ª Vulnerability Management o o ª Cost per vulnerability Cost per system setting Threat Management o ª Cost per request Cost per password reset Cost per incident Trust Management o o Cost per policy Cost per project © 2004 Spire Security. All rights reserved. 27

When to Use Metrics ª Process Effectiveness o Six Sigma ª Staff Productivity o ROI / promotions ª Cycle Time o Balanced Scorecard ª Staff Efficiency o ROI ª Cost Effectiveness o o Activity-based costing ROI/TCO © 2004 Spire Security. All rights reserved. 28

Q 3: Most Useful? Which metric type is most useful to your security program? 1. Process Effectiveness 2. Staff Productivity 3. Cycle Time 4. Staff Efficiency 5. Cost Effectiveness © 2004 Spire Security. All rights reserved. 29

Conclusions ª Security functions are spread throughout organizations. ª You can’t improve security until you measure it. ª Ultimately, security is a business operation that should be run like a business operation. © 2004 Spire Security. All rights reserved. 30

SP i RE security Agree? Disagree? Pete Lindstrom petelind@spiresecurity. com www. spiresecurity. com © 2004 Spire Security, LLC. All rights reserved.