Containment Eradication Recovery Lessons Learned Incident Response Where

Containment, Eradication, Recovery, Lessons Learned Incident Response

Where we’re at • We’ve hopefully done lots of planning beforehand • We’ve found and verified the presence of an incident • Determined there’s enough evidence to declare an incident • Before we decide how to contain the incident, let’s characterize the incident (CSIRT Case Classification) Category Criticality Sensitivity Incident Response We know what’s affected Have a good idea how they got in 2

Category Denial of Service • DOS or DDOS • Compromised Information Disclosure of sensitive corporate information or intellectual property • Compromised Asset Compromised host, network device, user account, application Malware, root account, rootkit, etc. • Unlawful Activity Theft, Fraud, Computer Safety Internal Hacking Suspicious activity originating from inside the network, excluding malware • External Hacking Suspicious activity originating from outside the network, excluding malware • Email Spoofed email, spam, phishing • Malware • Policy Violation Incident Response • 3

Criticality • Incident affecting critical systems Potential to impact revenue or customers Active hacking, compromised critical asset, virus outbreak, etc. Initial response time: 60 minutes • Incident affecting non-critical systems • Possible incident affecting non-critical systems Employee investigations not time sensitive, longer-term investigations Email, forensics request, policy violation Initial response time: 48 hours Incident Response Not revenue or customer impacting Time-sensitive employee investigations Policy violations, unauthorized access, compromised asset, inactive hacking Initial response time; 4 hours 4

Sensitivity • Helps define the “need to know” and who should be in the loop • Level 1: Extremely sensitive Forensics request, destruction of property, unlawful activity, compromised information, etc. Informed: CSIRT, Management Computer Security Incident Response Team • Level 2: Sensitive • Level 3: Not sensitive Denial of service, virus, email Informed: Employees can be informed as well of isolated infections Incident Response Hacking, unauthorized access Informed: CSIRT, Management, System owners, Operations 5

Inform appropriate entities • Thorough planning should help in identifying who should be informed • A senior manager should sponsor the IR team CISO, CIO, Legal counsel, etc. Notify them when declaring an incident • Local or organizational incident-handling team • Impacted business unit • Start tracking the incident Utilize an incident tracking system Incident Response Email Phone call or visit for the critical incidents 6

Containment • Goal is to keep the problem from getting worse • A number of ways to effectively contain the incident • Specific strategy details will vary based on The type of incident The goals of the containment strategy Incident Response Shut down the system Disconnect the system from the network Disable certain affected functions of the system 7

• Potential damage or theft of data or resources • Evidence preservation requirements • Availability of services • Effort needed to implement • Feasibility • Effectiveness of the strategy • Duration of the strategy Temporary solution? More permanent? Incident Response Containment method decision criteria 8

Costs of containment • Do you want to tip your hand? Some organizations may setup a sandbox and monitor the attacker Some organizations may determine leaving everything online with stringent monitoring is a better short-term solution Further determine what’s happening, what is affected Dangerous solution – could lead to further compromise Might cause additional damage Removing internet access from some malware could cause it to behave differently Malware could start encrypting files when it loses connectivity Making changes could damage evidence May need evidence for determining the full scope of the incident May need evidence for legal purposes Incident Response • 9

Short term containment • Keep a low profile Avoid tipping your hand to the attacker Don’t start looking for the attacker from the compromised machine with obvious methods Ping, traceroute, nslookup • Prevent more damage caused by the attacker • Possible short term actions Disconnect the network cable Pull the power Isolate the switch port, or put it on an “infected VLAN” Apply network-based firewalls to isolate the affected machine • If you remove the system or disable it, be sure to inform the business unit Incident Response Destroys volatile memory, could damage drive 10

• Especially in the case you need to preserve for legal proceedings, documentation is essential • Talk with legal professionals to be sure your evidence can be admissible in court • Account for evidence with a chain of custody • Keep an evidence log Identifying information - location, model number, hostname, IP address, etc. Name and title of individuals who collected the evidence Time and date Evidence storage locations • NIST guide on evidence preservation NIST SP 800 -86, Guide to Integrating Forensic Techniques into Incident Response Gathering evidence 11

Create forensic images • The quicker you can create a forensic image of the affected systems, the better • Image the memory and filesystem You’ll lose data if you turn off the system • Volatility and Memoryze can capture memory • Bit-by-bit images to get all filesystem data is best • Create hashes of the original and images • Not all incidents will allow for full backups and analysis More time sensitive incidents will require live forensics Incident Response As well as provide some analysis capabilities – more on that later 12

Determine how to continue • Analyze logs and other forensic information • How far did the attacker go? Be sure to check neighboring systems for compromise Lateral movement • Determine longer term containment strategies • Move into long term containment phase after backups and evidence have been acquired Incident Response Ultimately a business decision informed by the incident handler’s input 13

Long term containment • Can begin making changes to the system once forensic artifacts have been obtained • If the system can be kept offline, move to the eradication phase • If the system must be kept online, perform long term containment options • Long term containment may be long term, but it should still be temporary Running in production only until a clean replacement system is ready Incident Response Patch the system and network Change passwords Apply firewalls Use IPS Shutdown backdoor processes used by the attacker 14

Eradication • The goal of eradication is to actually get rid of the attacker Containment should stop the bleeding Eradication is actually removing the attacker’s artifacts • Re-format and re-build the machine Sure way to remove all of the attacker’s artifacts • Restoring from system backups • Remove all the artifacts manually from the existing system Be sure you don’t miss anything Incident Response Be sure it’s a clean backup! 15

Improve your defense • One of the most important, sometimes overlooked portions of the eradication phase • Rebuilding is great, removing malicious software is great • You MUST fix the vulnerabilities • Put additional defensive measures in place Firewalls Move the system to a new IP address Change DNS names Apply patches Harden the system Apply monitoring solutions Incident Response But what is stopping it from happening again? 16

Assess the system • Before you put a rebuilt system into production, perform a vulnerability analysis System Network Related vulnerabilities Scan the entire existing network in addition to the rebuilt system • Vulnerability scanners can help • Be sure to not put a vulnerable system back into production! • Attackers will often leverage the same vulnerability across multiple systems Look everywhere Incident Response Nessus, Open. VAS, Ne. Xpose, etc. 17

Recovery • The goal of recovery is to bring the impacted systems back online to production • Validate the system is back online and running normally • Typically will have the business unit assist in testing functionality • Restoring in off-hours is best Incident Response Easier to monitor Might be over-ruled, the business often will want to restore asap 18

Recovery - monitoring • The other major portion of the recovery phase is continued monitoring • Continually look for evidence of the intrusion On the recovered systems On the entire network • Check regularly for re-compromise Script this! Look for the specific indicators Utilize network and host-based IPS/IDS Add indicators of the compromise Incident Response • 19

• Learning and improving • Develop a follow up report • Start as soon as possible, use notes compiled throughout the IR • Report compiled by the on site team and lead incident handler • Have all parties review the report • Hold a meeting within a few days of the end of the incident • Use what you learned to improve Fix your processes Fix your technology Incident Response Lessons Learned 20

• What happened? • How well did the response work? • Were documented procedures adequate? • What other information could have been used sooner? • What hurdles were there? • What would you do differently next time? • Could information sharing have been improved? • What can we do next time to prevent any problems? • How can we detect similar incidents in the future? • Are any additional tools or resources needed? Incident Response Questions to address 21