CSC 382 Computer Security Software Security Testing CSC

  • Slides: 34
Download presentation
CSC 382: Computer Security Software Security Testing CSC 382: Computer Security 1

CSC 382: Computer Security Software Security Testing CSC 382: Computer Security 1

Topics 1. 2. 3. 4. Why is Testing Important? White Box vs Black Box

Topics 1. 2. 3. 4. Why is Testing Important? White Box vs Black Box Testing Creating a Test Plan Fault Injection – Source-based Fault Injection – Environment Fault Injection – Input-based Fault Injection 5. Fuzz Testing CSC 382: Computer Security 2

Why is Testing Important? Navy shoots down Civilian Airliner. – 1988 – US Vicennes

Why is Testing Important? Navy shoots down Civilian Airliner. – 1988 – US Vicennes shot down Airbus 320. – Mistook airbus 320 for a F-14. – 290 lives lost. – Why: Software bug – “cryptic and misleading output displayed by the tracking software. ” CSC 382: Computer Security 3

Why is Testing Important? “I Love You” virus cost $8. 75 billion. 2 –

Why is Testing Important? “I Love You” virus cost $8. 75 billion. 2 – Result of Microsoft decision to automatically execute binary attachments in Outlook. CSC 382: Computer Security 4

Why is Testing Important? 1. Complexity – Continually increasing. – Windows 3. 1 (3

Why is Testing Important? 1. Complexity – Continually increasing. – Windows 3. 1 (3 mloc) to Windows XP (40 mloc) 2. Extensibility – Plugins. – Mobile code. 3. Connectivity – Network access. – Wireless networking. CSC 382: Computer Security 5

Software Complexity 5 -50 bugs per/kloc 8 – 5/kloc: rigorous quality assurance testing (QA)

Software Complexity 5 -50 bugs per/kloc 8 – 5/kloc: rigorous quality assurance testing (QA) – 50/kloc: typical feature testing System Lines of Code MS Word 95 MS Windows 3. 1 Boeing 777 2 million 3 million 7 million Space Shuttle Netscape MS Windows XP 10 million 17 million 40 million CSC 382: Computer Security 6

Software Connectivity • Internet connectivity nearly ubiquitous. – 802. 11 wireless networking. – Bluetooth

Software Connectivity • Internet connectivity nearly ubiquitous. – 802. 11 wireless networking. – Bluetooth wireless personal area networking. – Embedded devices often networked. • Can your cellphone get a virus? – Symbian Bluetooth virus released in 2004. • Can your automobile get a virus? – BT discovery reveals embedded BT devices in high-end cars. CSC 382: Computer Security 7

Testing Caveat "Testing can establish the presence of errors, but never their absence. "

Testing Caveat "Testing can establish the presence of errors, but never their absence. " – Dijkstra CSC 382: Computer Security 8

White and Black Box Testing White Box Testing – Testing guided by the source

White and Black Box Testing White Box Testing – Testing guided by the source code and design documentation. – May be approximated by decompiling or disassembling binary code. Black Box Testing – Testing without access to system code. CSC 382: Computer Security 9

Creating a Test Plan Base Test Plan on Threat Model 1. Decompose application into

Creating a Test Plan Base Test Plan on Threat Model 1. Decompose application into components. 2. Identify entry points for each component. 3. Use STRIDE classification to determine types of tests to perform for each entry point. 4. Prioritize testing effort based on risk evaluation for each threat. 5. Use threat trees and DFDs to guide implementation of specific tests. CSC 382: Computer Security 10

1. Decompose Application Threat Model provides list of application components. CSC 382: Computer Security

1. Decompose Application Threat Model provides list of application components. CSC 382: Computer Security 11

2. Identify Entry Points Threat model provides list of entry points. Classify entry points

2. Identify Entry Points Threat model provides list of entry points. Classify entry points by risk potential 5. Entry Point Characteristic Points Entry point component runs with high privilege +2 Entry point component written in C/C++ +1 Entry point component written in VHL language -2 Entry point accepts arbitrary-sized input +1 Recipient buffer is stack-based +2 Entry point has no or weak access control +1 Entry point has good access control -2 CSC 382: Computer Security 12

STRIDE 5 Threat Categorization Spoofing ex: Replaying authentication transaction. Tampering ex: Modifying authentication files

STRIDE 5 Threat Categorization Spoofing ex: Replaying authentication transaction. Tampering ex: Modifying authentication files to add new user. Repudiation ex: Denying that you purchased items you actually did. Information disclosure ex: Obtaining a list of customer credit card numbers. Denial of service ex: Consuming CPU time via hash algorithm weakness. Elevation of privilege ex: Subverting a privileged program to run your cmds. CSC 382: Computer Security 13

3. STRIDE Test Types Spoofing – Attempt to force application to use no authentication.

3. STRIDE Test Types Spoofing – Attempt to force application to use no authentication. – Attempt to force application to fall back to less secure authentication protocol. – Can security tokens by replayed? – Can security tokens be brute-forced? Tampering – Is it possible to modify and rehash security token? – Create invalid hashes, MACs, and signatures to verify that they are checked correctly. CSC 382: Computer Security 14

3. STRIDE Test Types Repudiation – Can you prevent logging or auditing? – Is

3. STRIDE Test Types Repudiation – Can you prevent logging or auditing? – Is it possible to create requests that create incorrect data in event log? EOF, newline. Information Disclosure – Attempt to access privileged data. Sniffer, file. – Kill process, then search disk for confidential data left behind due to unclean termination. – Cause application to fail. Do error messages reveal any useful information? CSC 382: Computer Security 15

3. STRIDE Test Types Denial of Service – Flood application with many requests. –

3. STRIDE Test Types Denial of Service – Flood application with many requests. – Can malformed input crash the application? – Can external influences (low memory, disk, or other resource limits) cause a failure? Elevation of Privilege – Can you execute data as code? – Can an elevated privilege be forced to invoke a shell? CSC 382: Computer Security 16

4. Prioritize Testing Effort Use risk evaluation from Threat Model. – Summing attack tree

4. Prioritize Testing Effort Use risk evaluation from Threat Model. – Summing attack tree values based on categories from DREAD evaluation system. Read File Get Password Search Desk Network Access Social Engineer CSC 382: Computer Security Physical Access Boot with CD Remove hard disk 17

Evaluate Risk with DREAD Damage Potential Extent of damage if vulnerability exploited. Reproducibility How

Evaluate Risk with DREAD Damage Potential Extent of damage if vulnerability exploited. Reproducibility How often attempt at exploitation works. Exploitability Amount of effort required to exploit vulnerability. Affected Users. Ratio of installed instances of system that would be affected if exploit became widely available. Discoverability Likelihood that vulnerability will be discovered. CSC 382: Computer Security 18

5. Test Implementation Attack Trees – Divide threat into testable conditions. Data Flow Diagrams

5. Test Implementation Attack Trees – Divide threat into testable conditions. Data Flow Diagrams – Show data moves from entry point to system component impacted by threat. – Indicates security mechanisms to test/bypass. CSC 382: Computer Security 19

Dynamic Analysis: Memory Checking Tools like purify and valgrind check C/C++ code memory access

Dynamic Analysis: Memory Checking Tools like purify and valgrind check C/C++ code memory access at runtime. – Use of unitialized memory. – Memory access after free()/delete. – Out-of-bounds memory access. – Access to inappropriate areas of stack. – Memory leaks. – Overlapping source and dest pointers in memcpy() and similar functions. CSC 382: Computer Security 20

Fault Injection: Modifying internal or external environment of an application in an attempt to

Fault Injection: Modifying internal or external environment of an application in an attempt to discover flaws. Source-based Fault Injection Modifies application source code to produce faults. Environment Fault Injection Modifies environment (memory, libraries, etc. ) to produce faults. Input-based Fault Injection Modifies expected input to produce faults. CSC 382: Computer Security 21

Source-based Fault Injection Inject new code at compile-time or runtime to force faults that

Source-based Fault Injection Inject new code at compile-time or runtime to force faults that would not normally occur. – Cause functions to return unexpected values. – Set internal data to values that trigger faults. CSC 382: Computer Security 22

Environment-based Fault Injection Intercept application calls to OS (requests for memory, disk space, etc.

Environment-based Fault Injection Intercept application calls to OS (requests for memory, disk space, etc. ) – Selectively deny resources. – Control system call responses. CSC 382: Computer Security 23

Environment-based Example Attacks Alter application files. – Create files using app’s expected filename. –

Environment-based Example Attacks Alter application files. – Create files using app’s expected filename. – Delete expected files. – Change permissions. – Replace files with links. – Use filenames longer than MAX_PATH. Block access to shared libraries. – Does application crash? – If it doesn’t crash, what failures occur due to lost functionality? CSC 382: Computer Security 24

Environment-based Example Attacks Limit memory, disk, and network resources. – How does app response

Environment-based Example Attacks Limit memory, disk, and network resources. – How does app response to allocation failures? – Does app leak confidential info to swap? – Does no network prevent auditing/logging? Manipulate registry key existence and permissions on MS Windows systems. CSC 382: Computer Security 25

Input-based Fault Injection Send unexpectedly bad input to application. Input should be construed broadly:

Input-based Fault Injection Send unexpectedly bad input to application. Input should be construed broadly: – Command line arguments. – STDIN and GUI inputs. – Files. – Network sockets and RPC interfaces. – Environment variables. – Registry values. CSC 382: Computer Security 26

Input-based Example Attacks Overly large input. Can you trigger a buffer overflow? Out of

Input-based Example Attacks Overly large input. Can you trigger a buffer overflow? Out of range values. Can you create an integer overflow? Unexpected characters. Can you supply any metacharacters that will be used by application in an unsafe manner? Check for default accounts. CSC 382: Computer Security 27

Input-based Example Attacks Modify input semantics. – How does system handle a parameter supplied

Input-based Example Attacks Modify input semantics. – How does system handle a parameter supplied multiple times? Does it check input once, then use the other version of parameter? – What does application do with unknown parameters/fields? Use non-canonical resource names to check for directory traversal flaws. CSC 382: Computer Security 28

Fuzz Testing Black-box input based testing technique. – Uses random data. – Easily automated.

Fuzz Testing Black-box input based testing technique. – Uses random data. – Easily automated. – If application crashes or hangs, it fails. Results of 1995 study 9. – 15 -43% of utilities from commerical UNIX systems failed. – 9% of Linux utilities failed. – 6% of GNU utilities failed. – 50% of X-Windows utilities failed. CSC 382: Computer Security 29

Browser Fuzz Testing Michael Zalewski’s mangleme CGI 14 – Generated streams of malformed HTML.

Browser Fuzz Testing Michael Zalewski’s mangleme CGI 14 – Generated streams of malformed HTML. – Most browsers crashed due to: • • NULL pointers. Memory corruption. Buffer overflows. Memory exhaustion. – Exception: MSIE CSC 382: Computer Security 30

SPIKE Fault Injection Tool Network protocol fault injection. – Stateless protocols. – Stateful protocols.

SPIKE Fault Injection Tool Network protocol fault injection. – Stateless protocols. – Stateful protocols. Scripting language makes it easy to convert sniffed network packets into tests. – Automatic length and checksum calculation. – Script permits user-defined varying of packets: • Permute characters. • Buffer-overflow attempts. CSC 382: Computer Security 31

Web Proxy Fault Injection Point browser at fault injection proxy. Fault injection proxy supports

Web Proxy Fault Injection Point browser at fault injection proxy. Fault injection proxy supports – Cookie viewing and modification. – Form parameters viewing and modification. – Automated SQL injection attempts. – Automated buffer overflow attempts. – Brute-force password guessing. Examples – SPIKE Proxy CSC 382: Computer Security 32

Key Points 1. The Importance of Testing. 2. Black and White Box Methods. 3.

Key Points 1. The Importance of Testing. 2. Black and White Box Methods. 3. Test Plan based on Threat Model: a) b) c) d) Entry points. Test types based on STRIDE. Prioritization based on DREAD. Implementation using Attack Trees. 4. Fault Injection a) Source-based Fault Injection b) Environment Fault Injection c) Input-based Fault Injection 5. Fuzz Testing CSC 382: Computer Security 33

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. Matt Bishop, Introduction to Computer Security, Addison-Wesley, 2005. Computer Economics, “Malicious Code Attacks Had $13. 2 Billion Economic Impact in 2001, ” http: //www. computereconomics. com/article. cfm? id=133, 2002. Wenliang Du and Aditya Mathur, “Vulnerability Testing of Software System using Fault Injection, ” COAST TR 98 -02, 1998. Mark Graff and Kenneth van Wyk, Secure Coding: Principles & Practices, O’Reilly, 2003. Greg Hoglund and Gary Mc. Graw, Exploiting Software: How to Break Code, Addison-Wesley, 2004. Michael Howard and David Le. Blanc, Writing Secure Code, 2 nd edition, Microsoft Press, 2003. Koziol, et. al, The Shellcoder’s Handbook: Discovering and Exploiting Security Holes, Wiley, 2004. Barton Miller, Lars Fredrickson, and Bryan So, “An Empirical Study of the Reliability of UNIX Utilities, ” ftp: //ftp. cs. wisc. edu/paradyn/technical_papers/fuzz. pdf, 1990. Barton Miller, et. al. , “Fuzz Revisited: A Re-examination of the Reliability of UNIX Utilities and Services, ” ftp: //ftp. cs. wisc. edu/paradyn/technical_papers/fuzz-revisited. pdf, 1995. John Viega and Gary Mc. Graw, Building Secure Software, Addison-Wesley, 2002. Jeffrey Voas and Gary Mc. Graw, Software Fault Injection: Inoculating Programs Against Errors, Wiley, 1999. David Wheeler, Secure Programming for UNIX and Linux HOWTO, http: //www. dwheeler. com/secure -programs/Secure-Programs-HOWTO/index. html, 2003. James Whittaker and Herbert Thompson, How to Break Software Security: Effective Techniques for Security Testing, Addison-Wesley, 2004. Michael Zalewski, “Web browsers – a mini-farce, ” Bugtraq mailing list posting, http: //www. securityfocus. com/archive/1/378632, 2004. CSC 382: Computer Security 34