Buffer Overflow Detection Stuart Pickard CSCI 297 June

Buffer Overflow Detection Stuart Pickard CSCI 297 June 14, 2005

Papers n n n “Stack. Guard: Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks. ” Crispin Cowan, Calton Pu, Dave Maier, Heather Hinton, Peat Bakke, Steve Beattie, Aaron Grier, Perry Wagle, and Qian Zhang. 7 th USENIX Security Symposium, January 1998, San Antonio, TX. “Buffer Overflows: Attacks and Defenses for the Vulnerability of the Decade. ” Crispin Cowan, Perry Wagle, Calton Pu, Steve Beattie, and Jonathan Walpole. SANS 2000, Orlando FL, March 2000. “Type-Assisted Dynamic Buffer Overflow Detection. ” K. S. Hlee and J. S. Chapin. 11 th Annual USENIX Security Symposium 2002.

The First Two Papers n n n G 1 was published in 1998 G 2 was published in 2000 Both papers focus on Static Buffer Overflow detection and prevention in the compilation stage. Lead to the development of software for the Immunix project. The project then became Immunix Corporation. On May 10, 2005 Immunix was bought by Novell and the products are being sold under the name of App. Armor

Quote from Press Release n “The new product, to be called Novell® App. Armor powered by Immunix, TM protects both the Linux operating system and applications from external or internal attacks, viruses, and malicious applications. ”

Buffer Overflow life Cycle 1. 2. 3. A malicious user finds the vulnerability in a highly privileged program. The user might be able to gain access to the system by overflowing a buffer and pushing code into the return address space of a function call. Fixes are then made by patching the code to prevent future attacks.

4 basic approaches to defending against buffer overflow attacks 1. 2. 3. 4. Write Code to fully check array bounds Non-Executable Buffers – prevent the data segment of the victim program’s address space from being executed Array Bounds Checking – Check all accesses and writes to an array. (Pruify) Perform integrity checks on code pointers before dereferencing them to detect an attack (not as strong as the others)

Problem with patches n n Fixes to buffer overflows with patches are attacking the problem at the source. The first two papers suggest solving the problem of buffer overflows by protecting the stack (The Destination of the Attack)

Brief Review of a Buffer Overflow (Stack Smashing) The injected attack code typically spawns a shell with root privileges. Stack. Guard will prevent the injected attack code from executing. The next slide discusses their method. If a program is written with a buffer overflow vulnerability, then the attacker can crash Stack Guard

Stack. Guard prevents change to the return address of a function on the stack by either preventing change to the address or by preventing the write to the return address. Stack. Guard is more effective when the return address cannot be altered, however there is more overhead. Stack. Guard can run in both modes.

How Stack. Guard is Implemented n n n The return address is unaltered IIF the canary word is unaltered. Reason: This only really works for buffer overflow attacks since the attacker needs to write in a linear sequential manner. Thus, if the return address is modified, then the canary word is also modified. Stack. Guard is implemented by modifying the open source gcc compiler functions: function_prologue and function_epilogue. The modifications include modified code that emits a canary word to the stack and checks that the canary word has not been modified before the function return.

Stack. Guard continued n n total changes are under 100 extra lines of code to the gcc compiler. Canary values need to be random Stack. Guard Example: http: //nsfsecurity. pr. erau. edu/bom/Stack. Guard. html

Mem. Guard: Preventing return address changes n n Mem. Guard will protect the return address by making it read-only when the function is called. It will then unprotect the return address when the function finishes and needs to return The protection and un-protection occur in gcc’s function_prologue and function_epilogue.

Stack. Guard n n Canary version is more efficient, however, Mem. Guard is more secure. Immunix project solution: Run Stack. Guard in Canary version until a treat is detected, then run with Mem. Guard.

Results

Final thoughts on Stack. Guard n n n Stack. Guard not only patches a broad range of existing faults, it patches faults that have not been discovered. It turns the problem of attackers gaining root access to a problem of mild degradation-of-service attacks (switch from Canary to Mem. Guard mode) This software gives software developers more time to fix these problems. Compare Stack. Guard to Purify and other array bounds checking for C. In regards to level of overhead, we have Canary Mode as most efficient then Mem. Guard, then Purify, then other array bounds checking implementations.

Point. Guard - Pointer Integrity Checking n n In the second paper, not only is Stack. Guard discussed, but the new (as of 2000) patch to the gcc compiler is Point. Guard is a generalization of Stack. Guard which places “canaries” next to all code pointers (function pointers) As of writing this paper, Point. Guard is not past the prototyping stage. Point. Guard will include a special canary storage class that forces canary protection onto arbitrary variables.

Questions? n n Stack. Guard – In Mem. Guard mode or Canary Mode Point. Guard Immunix Novell App. Armor

Third Paper – Type-Assisted Dynamic Buffer Overflow Detection n This paper first covers many of the other buffer overflow prevention techniques. Most notably, they describe Stack. Guard and how it is vulnerable to some attacks. They claim that Stack. Guarded software is vulnerable to attacks exploiting code pointers other than the return address. With this vulnerability Stack. Guard can be bypassed. They also claim that Stack. Guard can be bypassed by exploiting the heap memory objects.

Their Solution n Enable range checking on buffers at runtime by implementing an intermediary step during compilation that emits an additional data structure into the binary file. This structure describes the types of automatic buffers and static buffers. They generate a type table – a data structure that associates the address of each function with the information of the function’s automatic buffers. They implemented a prototype by extending the GNU C compiler on Linux.

Their Solution continued. . . n n The source files are not modified. Just the separate type tables are created. Range checking is done with the following steps: 1. Locate the stack frame of the buffer by chasing down the saved frame pointer. 2. Retrieve the return address of the next stack frame to find out who allocated the stack frame. 3. Locate the function who allocated the stack frame by comparing the return address with function addresses in the type table. 4. Locate the buffer of the function by comparing the buffer address with offsets in the table + frame pointer value, 5. The size of the buffer (or the size of a field if it is a struct variable) is returned

Limitations to their solution n 1. 2. There are two cases where they cannot determine the size of automatic buffers. stack buffers dynamically allocated with alloca(); variable-length automatic arrays

Results: C-library string functions

Conclusions n n The results from the last paper point to the fact that it might not be practical to keep track of buffer ranges during runtime. However, certain vulnerabilities that exist with the Stack. Guard solution, are prevented with Type-Assisted Dynamic Buffer Overflow Detection

Conclusions. . . n n The solution to buffer overflow attacks most likely lies in a combination of these techniques. Also, maybe it is unlikely that we can totally be guaranteed prevention of these kind of attacks.

References: n n “Stack. Guard: Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks. ” Crispin Cowan, Calton Pu, Dave Maier, Heather Hinton, Peat Bakke, Steve Beattie, Aaron Grier, Perry Wagle, and Qian Zhang. 7 th USENIX Security Symposium, January 1998, San Antonio, TX. “Buffer Overflows: Attacks and Defenses for the Vulnerability of the Decade. ” Crispin Cowan, Perry Wagle, Calton Pu, Steve Beattie, and Jonathan Walpole. SANS 2000, Orlando FL, March 2000. “Type-Assisted Dynamic Buffer Overflow Detection. ” K. S. Hlee and J. S. Chapin. 11 th Annual USENIX Security Symposium 2002. “Buffer Overflow demo: Embry-Riddle, NSF Scholarships for Service Grant. ” Susan Gerhart, Jan Hogle, Jedidiah Crandall 2002 http: //nsfsecurity. pr. erau. edu/bom/

Discussion