Incident Response and Digital Forensics Denver Chapter of

Incident Response and Digital Forensics – Denver Chapter of ISACA

Outline § § Introduction The Incident Response Process § Preparation § Identification § Containment § Eradication § Recovery § Lessons Learned The Attacker Process § Reconnaissance § Scanning § Exploitation § Keeping Access § Covering Tracks Conclusion

Introduction § Fish. Net Security § Started in 1996, largest Info. Sec VAR § Support, Consulting, Training, Products § Locations – Boston, Omaha, Kansas City, St. Louis, Dallas, Minnesota § Assessment Team: Policy, Network, Web. App and DB, Wireless, IR and Digital Forensics

Introduction § Trey Keifer - (trey. keifer@fishnetsecurity. com) § Level II Security Engineer § Sr. Incident Response Analyst § Coordinator of Incident Response Program § SANS Certified Incident Handler (GCIH)

Incident Response and Digital Forensics § One of the least practiced, most stressful, highly scrutinized areas of Information Security. § Every incident is unique and can incorporate many different areas of the affected organization. § Incident analysts must be able to think quickly, remain calm and consider all possibilities.

Common Incident Types § Economic Espionage § Intellectual Property Theft § Unauthorized Access § Stolen Passwords and Data § Unauthorized Use § Inappropriate E-Mail and Web Habits § Malicious Code § Worms with Backdoors (Sasser) § Insider Threats

6 Steps of the Incident Handler Methodology § Preparation § Identification § § Containment Eradication Recovery Lessons Learned

Preparation: § The key to a successful response is preparation. § § Form a strategy. Design a procedure. Gather Resources. Practice, practice.

Preparation: § Identify the “Core Team” § § § § Technical (IT, Info. Sec and System Owners) Management Legal Department Forensics Public Relations Human Resources Physical Security and Maintenance Telecommunications

Preparation: § Organizing Individuals § All members of the CSIRT team should know their role and how they will interact with the other members. § Outsourced or “third party” members should have contracts in place. § Contacts for Law Enforcement should be known and situations for their involvement discussed.

Preparation: § Develop a Procedure § Incident response can be a high-stress time. A well documented procedure, that is easy to follow, can greatly reduce the anxiety. § Develop a call tree and notification procedures § Brainstorm likely scenarios. § Identify general information needed in most scenarios ahead of time. § Make checklists and forms for as much as possible.

Preparation: § Communication is incredibly important during an incident. Not only the people involved, but the method which it is done. § Updates should be frequent. § Out-of-Band Communications are very important. § Faxes § Cell Phones § Be careful with the Blackberry’s

Preparation: § Access Rights § The incident response team must have access to systems without the administrators authorization. § Controversial Issue § User Accounts, Passwords and Encryption keys § Third-party storage methods are available

Preparation: § Policies § Protect the organization from legal liability and allow investigators to do their job. Warning Banners are readily displayed. Search policy is detailed in employee manual. Human Resources and Legal have signed off. Employees have acknowledged knowing their expectations on privacy. § Beware of international laws (European Privacy Directive) § §

Preparation: § Gathering Resources § Incident analysts should have all information ready and be able to respond to the incident. § Procedures, Checklists and Forms are ready. § Access credentials are available or individuals with them are known. § System information, network diagrams, software and intellectual property are documented thoroughly.

Preparation: § Training § SANS Institute and GIAC Certifications § Track 4: Incident Response and Hacker Techniques § Track ? ? : Digital Forensics § Vendor Training § Guidance Software § Access Data § Partners § Incident Response Scenarios

Identification: “Incidents can’t always be prevented, but must always be detected. ”

Incident: Intentional or Unintentional § Multiple failed logins to the domain administrator account. § Administrator credentials were cached on a users workstation and they are attempting to login. § Someone is actively attempting to brute-force the account.

Identification: § Goals § Determine Scope § Identify what systems, people and informational assets are involved in the event. § Preserve Evidence § Protect the facts of the incident while determining the scenario.

Identification: Suspicious Events § Unexplained Occurrences § New Accounts or Files § File Modifications § IDS Triggers § Firewall Entries § Accounting Discrepancies § Poor Performance/Unresponsive services § System Instability

Identification: Passive Identification § Sniffers and Traffic Analysis § Cyclical Buffers allow full recording of events at the packet level to a point, depending on size and utilization. § Target machine evidence is still preserved. § Assist in determining new attacks for which signatures have not yet been written.

Identification: Passive Identification § Intrusion Detection Systems § Least invasive method § Target machine evidence is preserved § Logs must still be protected § Write-Once, Read-Many Media

Identification: Passive Identification § Tripwire-style File Modification § A hash of the file is taken and stored in a secure database. Any modification to that file results in a change of the hash. § Very indicative of a successful compromise. § Can be noisy during patching and must be tuned after every software upgrade.

Identification: Honey. Pots and Honey. Tokens § Specific systems or accounts with additional logging and notification to alert on suspicious activity. § Operators must be careful of entrapment. § Systems have to be secured and heavily monitored. § Systems cannot invite intruders – § No “hackme” accounts § No “Salary Database” systems

Identification: Chain of Custody § Evidence must be accounted for from the time it is collected until the time it is submitted to the court. § Each piece of evidence must be under the control of one, identifiable person at all times. § A change in control of the evidence must be recorded. § Evidence in storage must be protected from contamination. (ie… sealed and secured)

Containment Now that the events have been identified as an incident and a chain-of-custody for evidence has been established, we will take the first step into system modification by beginning our containment.

Containment: § Vendor Coordination § Work closely with your vendors and know how to open security-related tickets with high priority. § ISPs can prevent some Denial of Service situations. § They are more familiar with attacks because they have seen them with other clients and are up-to-date on advisories. § Additional people working towards identification, containment and recovery. § We are used to the pressure!

Containment: § Identifying the Trust Model § The trust model identifies not only the technology, but also the people that are involved in the incident. § What connectivity does the network or system have to other areas in the organization? § What information is contained within it? § Who needs to be involved and to what extent?

Containment: § Documentation Strategies § Documentation should be collected from most volatile to least volatile and least invasive to most invasive. § Volatile evidence includes RAM, running processes and active connections. § Be careful of running system commands from anything but recovery media.

Containment: § Should we Quarantine? § Changes to a system may be easily observed by an active attacker. § Rootkits may identify a pulled network connection or extensive system modification and protect the attacker. § Some exploits are entirely memory resident and will disappear when the power is pulled.

Containment: § Initial Analysis § § § Keep a low profile Never analyze the original Make frequent updates to CSIRT Acquire log files Stick to the facts and avoid blame Consider all possibilities but keep it simple

Containment: § Backups § Numerous backups allow both investigation and preservation of evidence. § § § Different strategies exist and depend on the situation. Original is kept as evidence Backup 1 – Placed back in production Backup 2 – Forensic Analysis Backup 3, 4, etc… separate copies for analysis

Containment: § Digital Forensics § Numerous separate analysis all yield the same results. § § Requires specialty hardware, software and training. Bit by Bit copying and analysis of data. Recovery of deleted data. Identification of altered system files (trojans) and binaries in a safe environment.

Containment: § Digital Forensics: Hardware Write Blockers § No modification to the data itself, we want to observe and duplicate only. § Hardware device or driver between acquisition machine and target system. § May use NIC, USB, Fire. Wire or IDE/SCSI channels. § Intercepts write commands and gives logical return results. § Allows browsing of the filesystem during acquisition.

Containment: § Digital Forensics: Forensic Software § Allows quick and efficient analysis of the information contained on the device. § Guidance Software’s En. Case used by law enforcement. § Linux Forensics CD’s are coming along in maturity. (still must use write blockers!!!) § Scripts allow quick searching of keywords in files and deleted data. § Hash comparisons verify original files, known dangerous applications and aid the examiner in avoiding the bad stuff.

Containment: § Digital Forensics: What are we looking for? § Many areas of interesting data are forgotten about. Cached web content Email Files (PST’s) Recoverable Deleted Files Specific Incidents: CAD drawings, Engineering diagrams, Pornography § Known file signatures of hacking tools, backdoors, etc… § §

Containment: § Digital Forensics: Other devices? § May not be able to submit as evidence in court, but can assist the Incident Handler in their investigation. § Personal Organizers (PIMs): Blackberry, Palm Pilots, IPAQ’s. § SIM Cards/Cell phones § USB Tokens/Flash Drives

Containment: § Digital Forensics: Not Perfect! § Some tools have been written specifically to defeat forensics software. § Do. D: 7 -Pass, random-write method for secure deletion of magnetic media. (Rainbow Method) § Windows: Eraser § Unix: Wipe

Containment: § Slowing the Attack § § § Change passwords and access rights. Change hostnames and IPs. Null Route suspicious traffic. Block IPs or Networks. Apply Patches to similar systems. Shutdown services.

Eradication Once an incident has been contained we attempt the total removal of malicious applications from a system or network.

Eradication: § Remove or Restore § The decision of whether to remove malicious files or restore from backups is a difficult task. § Rootkits almost always demand a rebuild. § Verification of backups is a must. § Patches may not be available and a total change of architectures may be necessary.

Eradication: § Improve Defenses § Implement additional detection and protection methods and strengthen existing technologies and processes. § Apply firewall and router filters. § Perform “mini-assessments” using the same tools and techniques as your attackers. § Look for the same exploits and backdoors on multiple machines.

Recovery Once threat has been removed the organization must begin the process of returning the business to normal operation.

Recovery: § Returning to Operation § System owners make the final call on returning to production. § Owners depend on the systems and know their true value. § If a disagreement occurs on whether to return to production or not it should be documented by the analysts and the owner should acknowledge responsibility.

Recovery: § Monitoring § At this point in the process you should have enough information to identify the attack if it occurs again. § Create custom IDS signatures if possible. § Verify properation to baseline configurations. § Implement additional logging on network, hosts and applications.

Lessons Learned The lessons learned meeting provides a method for the organization to coordinate knowledge of an incident, suggest changes in procedures and policies for the future and justify the implementation of new safeguards.

Lessons Learned: § Recap Meeting § Should occur promptly after eradication of an incident while details are fresh in the team members heads. § Create a timeline of events. § Provide a consensus of notes and documentation. § Finalize facts for a final report.

7 Deadly Sins § § § § Failure to report/ask for help Incomplete/Non-Existent Notes Mishandling/Damaging Evidence Failure to create backups Failure to eradicate or contain Failure to prevent re-infection Failure to apply lessons learned

Attacker Methodology § § § Reconnaissance § Profiling the Target Scanning § Identifying Weaknesses Exploitation § Breaking the Law Keeping Access § Backdoors Covering Tracks § Staying out of Jail

Reconnaissance: § The target is profiled – § § Employee Information (name, numbers, titles) Systems Information (usenet postings, job listings) Process Information (vendors and transactions) Location Information (external networks, physical locations)

Scanning: § Port and Vulnerability scanners are run to identify vulnerable systems. § § Open Ports and Services Vulnerable Applications Default Usernames and Passwords Weak Encryption Implementations

Exploitation: § Execution of attack – usually the first point at which the law is broken. § Goals § § Gaining Access Elevating Access Extracting Information Denying Service (Do. S)

Keeping Access: § Addition of Admin-level User Accounts § Enabling of default, insecure services § Installation of “Backdoor” or “root kit” applications allowing the attacker to retain access despite system modifications. § Application Level § Traditional Rootkit § Kernel Level Rootkit

Covering Tracks: § Modification of system logs, applications and processes to prevent identification by administrators. § § Hiding files and Directories (… and alt-255 dirs) Changes in /var/log Changes in shell history Removal of events (windows)

Our Example Scenario § An attacker uses a “ 0 -day” exploit to infiltrate the target organization, install a backdoor and retrieve critical intellectual property for a competitor. § Normal security procedures alert the administrators to suspicious activity and the incident response plan is activated.

Attacker Perspective: Reconnaissance § Google and the corporate web site are used to identify the organizational structure of key personnel including HR managers and executive management. § Low-Profile, no data sent directly to organization. § Impossible to detect.

Attacker Perspective: Harvesting § Freely-available scanning tools are used to identify email addresses from the corporate website. § Same method as SPAM groups. § Many sites do not use generic web addresses.

Attacker Perspective: Exploitation § Attacker sends malicious application to email addresses obtained during scanning. § Users open emails (possibly through social engineering) and are immediately infected. § Attacker can be listening for connections from infected machines and have immediate control over systems.

Attacker Perspective: Keeping Access

Incident Timeline

Incident Timeline: Preparation § IR Team established and roles defined. § Daily procedures established for log analysis and identification. § Containment procedures are outlined in policy. (Restoration takes priority) § Roles and Responsibilities are defined

Incident Timeline: Identification § Bandwidth graphing shows abnormal usage § Passive sniffing identifies responsible host

Incident Timeline: Containment § No “watch and learn” policy, power is pulled from the host. § System is imaged using forensic tools and Hardware Write-Blockers which prevent alteration of data during backup. § Employee is interviewed to determine method of infection.

Incident Timeline: Eradication and Recovery § System is restored from the organizations hardened base image and patches are applied. (Analysis can continue through restore)

Incident Timeline: Lessons Learned § § § Social Engineering Awareness File attachment blocking Firewall Rule Revisions IDS Signature changes Patch Management Advisory Alert Services

Questions?