Security Metrics That Dont Suck Dr Mike Lloyd

Security Metrics That Don’t Suck Dr Mike Lloyd CTO

Problems in security metrics • Security is the absence of something • Can’t report how often you were NOT on the cover of WSJ 2

How fast is your treadmill? • Many people start with process counting • These measure busyness • Not business • How do you show gains? • Just get busier? 3

Really bad metrics • An extreme case of measuring busyness • Bad metric → bad behavior → bad outcome 4

C o p y r i g h t Security metrics 1. 0 • On the hamster wheel of pain • (thanks, Andrew Jaquith!) R e d S e a l N e t w o r k • Oh look, “stuff” happens … 5

Management metrics • Metrics close the control loop • Ops has availability • Security needs risk • Focus on outcomes • How easily could a breach occur? • How effective is our spend? • Are we making it harder to break in? 6 Availability Operations Risk Security

Actors in the play • Good measurements tell stories • Which story depends on who’s on stage 1. 2. 3. 4. The team member The CISO The CFO Any guesses? 7

C o p y r i g h t R e d S e a l N e t w o r k Within security • The team members and the CISO • Should we focus on network or endpoints? • What are the top 10 risk contributors? • Are we being effective? 8

Out to the business • The CISO talking to the CFO • • Look, we are being effective! Show reduction in risk Correlate with expenditures Ask for more money 9 Dept of State, Feb 2011

About that “asking for money” bit … • The CFO wants ROI • I spend $X, I save $Y • We tell FUD stories • “Look at Sony – want that? ” • We’re sick of it, so are they • You have an unexpected ally • Thought about your company’s insurance agent? Not an actual agent – real ones are much nicer 10

Insurance brings focus • Data Breach Insurance is available • Pay to transfer risks • They see many breaches • You only see yours • They learn what works • CFO can define “good security” as “that which reduces my insurance premiums” • Measure your posture, negotiate a discount • If we do this right, we could actually measure what we all do for a living! 11

Enough why – time for what

Resources • Assets you need to protect • Everyone has some examples • PII, regulatory assets, IP, etc • Some truly “mission critical” • Financial, energy, government, military • Knowledge of vulnerabilities • Bad guys exploit them, so you scan • Counter-measures • It starts with the firewall 13

Simulate attacks before they happen • We want to know our defensive posture • That involves finding the weak points • Attack a model of the network • Measure ease of compromise • Use standards where possible 14 Copyright Red. Seal Networks, Inc. All rights reserved

Risk from Network-Based Attacks Low Risk High Risk Blocking Rule Blocking ACL Pivot Attack Copyright Red. Seal Networks, Inc. All rights reserved Pivot Attack

Sample attack chain – Before Internet DMZ Main Site 16 Copyright Red. Seal Networks, Inc. All rights reserved

Step 1 – Vulnerabilities exposed in DMZ § Attackers can reach these Internet-facing servers Copyright Red. Seal Networks, Inc. All rights reserved 17

Step 2 – Some attack paths sneak in § Just a few pivot attacks are possible Copyright Red. Seal Networks, Inc. All rights reserved 18

Step 3 – Attack fans out § An attacker can get in if they find this before you fix it Copyright Red. Seal Networks, Inc. All rights reserved 19

Penetration test results • Sample result: • External attackers can reach red hosts • Then pivot to attack yellow hosts • But no attack combination reached green hosts Copyright Red. Seal Networks, Inc. All rights reserved 20

2 1 Turning attacks into measurements

Rolling up the scores

Dashboards for outbound proof How easily can attackers get in? How big is my attack surface? How much is non-compliant? 23

Dashboards for internal questions Are investments working? Where do we need to improve? 24

C o p y r i g h t R e d S e a l N e t w o r k Conclusions • Defensive posture CAN be measured • This drives to better outcomes • Measure posture => improved posture • It helps the CFO “get it” • You can sleep better • Demonstrate effectiveness, not busyness 25

Thank you.

Insurance and the search for ALE • As we’ll show, you can measure POSTURE • How easily could someone break in? • True risk tradeoffs require incidence data • Sure, we know a vulnerability is bad … • But how often does it cause losses? • Risk wonks call this Annualized Loss Expectancy 27

Good ALE is hard to find • Bad breaches are common • But for any one company, big ones are rare • Similar to car accidents • Roughly one accident person per lifetime • How do you get a wide sample of accidents? • Who can test effects of a safety measure? • Insurers can! 28

Another complex arena: chess • Are computers or humans best? • Kasparov: “wrong question”! • How to play the best chess? • Human-computer teams • • Humans are good at strategy Computers don’t get tired Security faces related problems Attackers have adopted automation • They will find whatever you miss 29 Copyright Red. Seal Networks, Inc. All rights reserved