Machine Programming IA 32 memory layout and buffer

Machine Programming – IA 32 memory layout and buffer overflow CENG 331: Introduction to Computer Systems 7 th Lecture Instructor: Erol Sahin Acknowledgement: Most of the slides are adapted from the ones prepared by R. E. Bryant, D. R. O’Hallaron of Carnegie-Mellon Univ.

FF Linux Memory Layout ¢ C 0 BF Stack Upper 2 hex digits 80 of 7 F Red addre Hat ss v. 6. 2 ~1920 40 MB 3 F memor y limit 08 00 ¢ ¢ Heap ¢ DLLs Heap Data Text ¢ Stack § Runtime stack (8 MB limit) Heap § Dynamically allocated storage § When call malloc, calloc, new DLLs § Dynamically Linked Libraries § Library routines (e. g. , printf, malloc) § Linked into object code when first executed Data § Statically allocated data § E. g. , arrays & strings declared in code Text § Executable machine instructions § Read-only

Linux Memory Allocation Initially BF Stack 80 7 F Some Heap Linked BF Stack 80 7 F More Heap BF Stack 80 7 F Heap 40 3 F 40 DLLs 3 F Data 08 Text 00 40 DLLs 3 F Heap Data 08 Text 00

Text & Stack Example (gdb) break main (gdb) run Breakpoint 1, 0 x 804856 f in main () (gdb) print $esp $3 = (void *) 0 xbffffc 78 ¢ BF Stack 80 7 F Main § Address 0 x 804856 f should be read 0 x 0804856 f ¢ Initially Stack § Address 0 xbffffc 78 40 3 F Data 08 Text 00

Dynamic Linking Example (gdb) print malloc $1 = {<text variable, no debug info>} 0 x 8048454 <malloc> (gdb) run Program exited normally. (gdb) print malloc $2 = {void *(unsigned int)} 0 x 40006240 <malloc> ¢ Initially § Code in text segment that invokes dynamic linker § Address 0 x 8048454 should be read 0 x 08048454 ¢ Final § Code in DLL region Linked BF Stack 80 7 F 40 DLLs 3 F Data 08 Text 00
![Memory Allocation Example char big_array[1<<24]; /* 16 MB */ char huge_array[1<<28]; /* 256 MB Memory Allocation Example char big_array[1<<24]; /* 16 MB */ char huge_array[1<<28]; /* 256 MB](http://slidetodoc.com/presentation_image_h2/700c121245511aede9a4156b7c86a4df/image-6.jpg)
Memory Allocation Example char big_array[1<<24]; /* 16 MB */ char huge_array[1<<28]; /* 256 MB */ int beyond; char *p 1, *p 2, *p 3, *p 4; int useless() { int { p 1 p 2 p 3 p 4 /* } return 0; } main() = malloc(1 Some print <<28); /* << 8); /* statements 256 256. . . MB B */ */ */

Example Addresses $esp p 3 p 1 Final malloc p 4 p 2 beyond big_array huge_array main() useless() Initial malloc 0 xbffffc 78 0 x 500 b 5008 0 x 400 b 4008 0 x 40006240 0 x 1904 a 640 0 x 1904 a 538 0 x 1904 a 524 0 x 1804 a 520 0 x 0804 a 510 0 x 0804856 f 0 x 08048560 0 x 08048454 BF Stack 80 7 F Heap 40 DLLs 3 F Heap Data 08 Text 00

Internet Worm and IM War ¢ November, 1988 § Internet Worm attacks thousands of Internet hosts. § How did it happen? ¢ July, 1999 § Microsoft launches MSN Messenger (instant messaging system). § Messenger clients can access popular AOL Instant Messaging Service (AIM) servers AIM client MSN server MSN client AIM server AIM client

Internet Worm and IM War (cont. ) ¢ August 1999 § Mysteriously, Messenger clients can no longer access AIM servers. § Microsoft and AOL begin the IM war: AOL changes server to disallow Messenger clients § Microsoft makes changes to clients to defeat AOL changes. § At least 13 such skirmishes. § How did it happen? § ¢ The Internet Worm and AOL/Microsoft War were both based on stack buffer overflow exploits! many Unix functions do not check argument sizes. § allows target buffers to overflow. §

String Library Code § Implementation of Unix function gets § No way to specify limit on number of characters to read /* Get string from stdin */ char *gets(char *dest) { int c = getc(); char *p = dest; while (c != EOF && c != 'n') { *p++ = c; c = getc(); } *p = '