Defining Security Metrics Security Planning Susan Lincke Security

Defining Security Metrics Security Planning Susan Lincke

Security Planning: An Applied Approach | 12/16/2021 | 2 SABSA High-Level Framework

Security Planning: An Applied Approach | 12/16/2021 | 3 Gap Analysis The difference between where you are and where you want to be: (For example: # malware infections/month Rate of finding illegal software, hardware Security awareness training averages

Security Planning: An Applied Approach | 12/16/2021 | 4 SEI/COBIT Level 4 Monitoring: Includes Metrics inform management (and independent auditors) of the effectiveness of the security program Monitoring achievement of control objective may be more important than perfecting security procedures

Security Planning: An Applied Approach | 12/16/2021 | 5 Which metrics to use? Business-Driven Addresses specific business risks • Inherent industry risks Tailored to organization Measures adherence to control objectives Technology-Driven Addresses recent threats observed by CERT • CERT: Computer Emergency Readiness Team Addresses recent forensic data

Security Planning: An Applied Approach | 12/16/2021 | 6 Monitoring Function: Business-Driven Metrics Executive mgmt is interested in Strategic risk, budget, policy. Metrics Review every 6 months-1 year Metrics Tactical Metrics Determine effectiveness of security program: risk changes, compliance, incident response tests. Review quarterly to half-year Operational Metrics Technical details: E. g. , firewall, logs, IPS, vulnerability tests. Review weekly. Automate statistics.

Security Planning: An Applied Approach | 12/16/2021 | 7 Monitoring Function: Business-Driven Metrics Project Plan or Budget Metrics Strategic Risk performance Metrics Disaster Recovery Test results Audit results Regulatory compliance results Metrics Tactical Metrics Policy compliance metrics Exceptions to policy/standards Changes in process or system affecting risk Incident management effectiveness Operational Vulnerability Scan results Metrics Server config. standards compliance IDS monitoring results Firewall log analysis Patch mgmt status

Security Planning: An Applied Approach | 12/16/2021 | 8 Which metrics? Step 1: What are the most important security areas … threats …. regulation … to monitor in your organization? Step 2: Which metrics make the most sense to collect. Can they be automated? Step 3: Consider the 3 perspectives: strategic, tactical, operational metrics, relative to 3 audiences.

Security Planning: An Applied Approach | 12/16/2021 | 9 Monitoring Function: Metrics Risk: The aggregate ALE % of risk eliminated, mitigated, transferred # of open risks due to inaction Cost Effectiveness: What is: Cost of workstation security per user Cost of email spam and virus protection per mailbox Operational Performance Time to detect and contain incidents % packages installed without problem % of systems audited in last quarter Organizational Awareness: % of employees passing quiz, after training vs. 3 months later % of employees taking training Technical Security Architecture # of malware identified and neutralized Types of compromises, by severity & attack type Attack attempts repelled by control devices Volume of messages, KB processed by communications control devices Security Process Monitoring: Last date and type of BCP, DRP, IRP testing Last date asset inventories were reviewed & updated Frequency of executive mgmt review activities compared to planned

Security Planning: An Applied Approach | 12/16/2021 | 10 Monitoring Function: Metrics cont’d Security Management Framework: Completeness and clarity of security documentation Inclusion of security in each project plan Rate of issue recurrence Compliance: Rate of compliance with regulation or policy Rate of automation of compliance tests Frequency of compliance testing Secure Software Development: Rate of projects passing compliance audits Percent of development staff certified in security Rate of teams reporting code reviews on high-risk code in past 6 months Incident Response Metrics # of Reported Incidents # of Detected Incidents Average time to respond to incident Average time to resolve an incident Total number of incidents successfully resolved Total damage from reported or detected incidents Total damage if incidents had not been contained in a timely manner

Security Planning: An Applied Approach | 12/16/2021 | 11 Workbook: Metrics Selected What are the most important areas to monitor in your organization? Lunatic gunman FERPA Violation Category Major Risks: Metric Operational Web Availability Calculation & Collection Method Information Tech. Group Period of Reporting 1 year Incident Response totals 6 months % employees passing FERPA quiz Annual email requesting testing 1 year % employees completing FERPA training Two annual trainings with sign-in. Performance review 1 year # Hours Web unavailable Incident Response form 6 months # brute force attacks Incident Response form 1 month # malware infections Incident Response form 1 month Strategic Cost of security/terminal Cost of incidents Tactical Cracking Attempt

Security Planning: An Applied Approach | 12/16/2021 | 12 SANS-Recommended Critical Controls for Effective Cyber Defense TECHNOLOGY-DRIVEN METRICS

Security Planning: An Applied Approach | 12/16/2021 | 13 Creating a baseline configuration of network

Security Planning: An Applied Approach | 12/16/2021 | 14 Noticing inappropriate ‘additions’ to the network New PC New AP New wireless

Security Planning: An Applied Approach | 12/16/2021 | 15 Checking the security configuration of network Patched? Legal software? Firewall on & security configured? Antivirus on and patched? Limit USB access? WPA 2 AES, EAP/ TLS? Monitor Network? Withstands attacks? SQL, buffer overflow, cross-site scripting, clickjacking, …

Security Planning: An Applied Approach | 12/16/2021 | 16 Noticing inappropriate actions New sys admin or user acct Transfer of confidential data or illegal packets Detect new network service

Security Planning: An Applied Approach | 12/16/2021 | 17 SANS: Critical Controls for Effective Cyber Defense Typical SANS Metric: Temporarily install unauthorized software, hardware or configuration on a device. It should be: • found within 24 hours (or best: 2 minutes) • isolated within one hour confirmed by alert/email • reported every 24 hours until issue is resolved.

Security Planning: An Applied Approach | 12/16/2021 | 18 SANS Critical Control 1: Inventory of Authorized Devices Ensure all devices (with IP address) on network are known, configured properly, and patched. Scan network daily or use DHCP reports or passive monitoring. Compare results with baseline configuration. Metric: Temporarily install unauthorized device.

Security Planning: An Applied Approach | 12/16/2021 | 19 SANS: Critical Control 2: Inventory of Authorized Software Ensure all software is approved and recently patched • Whitelist defines the permitted list of software. • Blacklist defines illegal software (e. g. , IT tools). • Endpoint Security Suites (ESS) contain antivirus, antispyware, firewall, IDS/IPS, s/w white/blacklisting. Metric: Temporarily install unauthorized software on a device.

Security Planning: An Applied Approach | 12/16/2021 | 20 SANS Critical Control 3: Secure Configurations for Hardware & Software All devices are hardened using recommended security configurations • Illegal software list exists, includes Telnet, VNC, RDP • New software is quarantined and monitored. • Imaged software is maintained in an updated state. Build secure images, and use configuration checking tools daily. Metric: Temporarily attempt to change a set of random configurations.

Security Planning: An Applied Approach | 12/16/2021 | 21 SANS Critical Control 4: Continuous Vulnerability Assessment Run vulnerability scans on all systems at least weekly, preferably daily. Problem fixes are verified through additional scans. • Vulnerability scanning tools (updated) for: wireless, server, endpoint, etc. • Automated patch management tools notify via email when all systems have been patched. Metric: If the scan does not complete in 24 hours, an email notification occurs.

Security Planning: An Applied Approach | 12/16/2021 | 22 SANS Critical Control 5: Malware Defense Antivirus/antispyware is always updated • Run against all data: shared files, server data, mobile data. Additional controls: blocking social media, limiting external devices (USB), using web proxy gateways, network monitoring. • Endpoint security suites report tool is updated and active on all systems Metric: For install of benign malware (e. g. , security/hacking tool), antivirus prevents installation or execution or quarantines software • Sends an alert/email within one hour indicating specific device and owner

Security Planning: An Applied Approach | 12/16/2021 | 23 SANS Critical Control 6: Application S/W Security New application software is tested for security vulnerabilities: • Web vulnerabilities: buffer overflow, SQL injection, cross-site scripting, cross-site request forgery, clickjacking of code, and performance during DDOS attacks. • Input validated for size, type • No system error messages reported directly to user Automated testing includes static code analyzers and automated web scanning. Configurations include application firewalls and hardened databases. Metric: An attack on the software generates a log/email within 24 hours (or less). Automated web scanning occurs weekly or daily

Security Planning: An Applied Approach | 12/16/2021 | 24 SANS Critical Control 7: Wireless Device Control Wireless access points are securely configured with WPA 2 protocol and AES encryption. • Extensible Authentication Protocol-Transport Layer Security (EAP/TLS) provides mutual authentication. • Only registered, security-approved devices are able to connect Wireless networks are configured for the minimum required radio footprint. Metrics: Wireless intrusion detection systems detect available wireless access points and deactivate rogue access points within 1 hour Vulnerability scanners can detect unauthorized wireless access points connected to the Internet.

Security Planning: An Applied Approach | 12/16/2021 | 25 SANS Critical Control 8: Data Recovery Capability Backups are maintained at least weekly and more often for critical data. Backups are encrypted and securely stored. Multiple staff can perform backup/recovery. Metric: Test backups quarterly for a random sample of systems. This includes operating system, software, and data restoration.

Security Planning: An Applied Approach | 12/16/2021 | 26 SANS Critical Control 9: Security Skills Assessment Security awareness training: required for end users, system owners Security training: necessary for programmers, system, security and network administrators Metric: Test security awareness understanding • Periodically test social engineering tests via phishing emails and phone call • Employees who fail a test must attend a class

Security Planning: An Applied Approach | 12/16/2021 | 27 SANS Critical Control 10: Secure Network Configurations A configuration DB tracks approved configurations in config. mgmt. for network devices: firewalls, routers, switches. Tools perform rule set sanity checking for Access Control Lists. Two-factor identification is used for network devices. Metric: Any change to the configuration of a network device is reported within 24 hours

Security Planning: An Applied Approach | 12/16/2021 | 28 SANS Critical Controls 11. Control of Network Ports, Protocols and Services: Default Deny packets. Periodically review for restriction Metric: Measure time to recognize added network service 12. Controlled Administrative Privilege: Minimal elevated privileges Passwords are complex, changed periodically, 2 -factor Metric: Measure time to recognize new sys admin

Security Planning: An Applied Approach | 12/16/2021 | 29 SANS Critical Controls 13. Boundary Defense: Use firewall zones to filter incoming and outgoing traffic. Blacklist & whitelist network addresses 14. Analysis of Security Audit Logs: Server logs are write-only and archived for months. Firewalls log allowed and blocked traffic. Unauthorized access attempts are Metric: Measure time to recognize unauthorized packets logged Metric: Measure time to recognize no log space

Security Planning: An Applied Approach | 12/16/2021 | 30 SANS Critical Controls 15. Need to Know Access: Prevent exfiltration of data (e. g. , to competitors) Classify data Use restrictive firewall configurations Log access to confidential data Metric: Measure time to recognize unauthorized access 16. Account Monitoring and Control: Terminated accounts -> removed Expired password/ disabled/ locked out accounts, -> investigated Failed logins -> lockouts Inactivity -> locked sessions Unusual time access -> alert Data exfiltration recognized by keywords. Metric: Measure time to recognize new/ changed user accounts

Security Planning: An Applied Approach | 12/16/2021 | 31 SANS Critical Controls 17. Data Loss Prevention: 18. Incident Response: Prevent exfiltration of proprietary or confidential info • Encrypt mobile and USB devices • Disable USB Incident Response Plan defines who does what for various conditions IRP includes contact information for third party contractors Metric: Measure time to recognize transfer of confidential data file

Security Planning: An Applied Approach | 12/16/2021 | 32 SANS Critical Controls 19. Secure Network Engineering: Separate zones exist: DMZ, middleware, private network • DMZ accessed through proxy firewall • DMZ DNS is in DMZ; internal DNS is in internal zone, … Emergency config. for restricted network is ready for quick deployment. 20. Penetration Tests: Penetration tests = vulnerability tests + attacker tests Red Team exercises test incident response team reactions Metric: Measure false positive, false negative, true positive rate

Security Planning: An Applied Approach | 12/16/2021 | 33 Question The difference between where an organization performs and where they intend to perform is known as: 1. Gap analysis 2. Quality Control 3. Performance Measurement 4. Benchmarking

Security Planning: An Applied Approach | 12/16/2021 | 34 Question The MOST important metrics when measuring compliance include: 1. Metrics most easily automated 2. Metrics related to intrusion detection 3. Those recommended by best practices 4. Metrics measuring conformance to policy

Security Planning: An Applied Approach | 12/16/2021 | 35 Question SANS recommends that an initial maximum allowable time to detect a problem in a network or server configuration is: 1. Two minutes 2. One hour 3. One day 4. One week