Dynamic Memory Allocation Basic Concepts Andrew Case Slides

Dynamic Memory Allocation: Basic Concepts Andrew Case Slides adapted from Jinyang Li, Randy Bryant and Dave O’Hallaro 1

Topics ¢ ¢ Basic concepts & challenges Implicit free lists 2

Why dynamic allocator? ¢ We’ve discussed how exactly systems handle two types of data allocation so far: § Data segments (global variables in. data/. bss) § Stack-allocated (local variables) ¢ Not sufficient! § What about data whose size is only known during runtime? § Need to control lifetime of allocated data? § E. g. a linked list that grows and shrinks 3

Dynamic Memory Allocation Holds local variables User stack Grown/shrunk by malloc library, holds dynamicallyallocated data Heap (via malloc) Uninitialized data (. bss) Initialized data (. data) Holds global variables Program text (. text) 0 4

Example usage: a simple linked list typedef struct item_t { Returns a pointer to a int val; item_t *next; block of size bytes, }item_t; memory item_t *head = NULL; void insert(int v) { item_t *x; x = (item_t *)malloc(sizeof(item_t)); if (x == NULL) { exit(1); } x->val = v; x->next = head; head = x; } void delete(int v) { item_t *x; if (head && head->val == v) { x = head; head = head->next; free(x); } } int main() { while (1) { if (fscanf(stdin, “%c %dn”, &c, &v)!=2) { break; } if (c == ‘i’) { insert(v); }else if (c == ‘d’) { delete(v); } } printlist(); } linux%. /a. out i 5 i 10 i 20 d 20 q list contents: 10 5 Returns memory block to allocator, void printlist() { item_t *x; x must come from previous x = head; to malloc printf(“listcalls contents: “); while (x!=NULL) { printf(“%d “, x->val); x = x->next; } } 5

Dynamic Memory Allocation Application malloc free realloc Dynamic Memory Allocator mmap sbrk Allocate or free data of arbitrary sizes Request for or give back large chunk of pagealigned address space OS Dynamic memory allocator is part of user-level library. Why not implement its functionality in the kernel? 6

Types of Dynamic Memory Allocator ¢ ¢ ¢ Allocator maintains heap as collection of variable sized blocks, which are either allocated or freed Two types of allocators… Explicit allocator (used by C/C++): application allocates and frees space malloc and free in C § new and delete in C++ § ¢ This class Implicit allocator (used by Java, …): application allocates, but does not free space § Garbage collection in Java, Python etc. 7

Challenges facing a memory allocator ¢ Applications § Can issue arbitrary sequence of malloc and free requests § free request must be to a malloc’d block ¢ Achieve good throughput § malloc/free calls should return quickly ¢ Achieve good memory utilization § Apps issue arbitrary sequence of malloc/free requests of arbitrary sizes § Utilization = sum of malloc’d data / size of heap 8

Challenges facing a memory allocator ¢ Constraints Allocators § Can’t control number or size of allocated blocks § Must respond immediately to malloc requests i. e. , can’t reorder or buffer requests Must allocate blocks from free memory § i. e. , can only place allocated blocks in free memory Must align blocks so they satisfy all alignment requirements § 8 byte alignment for GNU malloc (libc malloc) on Linux boxes Can manipulate and modify only free memory Can’t move the allocated blocks once they are malloc’d § i. e. , compaction is not allowed § § § 9

Good Throughput ¢ Given some sequence of malloc and free requests: § R 0, R 1, . . . , Rk, . . . , Rn-1 ¢ Goals: maximize throughput and peak memory utilization § These goals are often conflicting ¢ Throughput: § Number of completed requests per unit time § Example: 5, 000 malloc calls and 5, 000 free calls in 10 seconds § Throughput is 1, 000 operations/second § 10

Good Utilization ¢ Given some sequence of malloc and free requests: § R 0, R 1, . . . , Rk, . . . , Rn-1 ¢ Pk : Aggregate payload § malloc(p) results in a block with a payload of p bytes § After request Rk has completed, the aggregate payload Pk is the sum of currently allocated payloads ¢ Hk: Current heap size - Assume Hk is nondecreasing § ¢ i. e. , heap only grows when allocator uses sbrk Def: Peak memory utilization after k requests § Uk = ( maxi<k Pi ) / Hk 11

Challenge #1 ¢ Design an algorithm that maximizes throughput. ¢ Design an algorithm that maximizes usage. ¢ Designing an algorithm that balances both. 12

Assumptions made in these lectures ¢ Memory is word addressed (each word can hold a pointer) Allocated block (4 words) Free block (3 words) Free word Allocated word 13

Allocation Example p 1 = malloc(4) p 2 = malloc(5) p 3 = malloc(6) free(p 2) p 4 = malloc(2) 14

Fragmentation ¢ Poor memory utilization caused by fragmentation § internal fragmentation § external fragmentation 15

Internal Fragmentation ¢ ¢ Mallocates data from ``blocks’’ of certain sizes. Internal fragmentation occurs if payload is smaller than block size Block of 128 -byte 100 byte Payload ¢ Internal fragmentation May be caused by § Limited choices of block sizes § Padding for alignment purposes § Other space overheads 16

External Fragmentation ¢ Occurs when there is enough aggregate heap memory, but no single free block is large enough p 1 = malloc(4) p 2 = malloc(5) p 3 = malloc(6) free(p 2) p 4 = malloc(6) ¢ Oops! (what would happen now? ) Depends on the pattern of future requests § Thus, difficult to measure 17

Challenge #2 ¢ ¢ ¢ How do we know how much memory to free given just a pointer? How do we keep track of the free blocks? What do we do with the extra space when allocating a structure that is smaller than the free block it is placed in? How do we pick a block to use for allocation -- many might fit? How do we reinsert freed block into available memory? 18

Knowing How Much to Free ¢ Standard method § Keep the length of a block in the header field preceding the block § Requires an extra word for every allocated block p 0 = malloc(4) 5 block size data free(p 0) 19

Keeping Track of Free Blocks ¢ Method 1: Implicit list using length—links all blocks 5 ¢ 6 2 Method 2: Explicit list among the free blocks using pointers 5 ¢ 4 4 6 2 Method 3: Segregated free list § Different free lists for different size classes ¢ Method 4: Blocks sorted by size § Can use a balanced tree (e. g. Red-Black tree) with pointers within each free block, and the length used as a key 20

Topics ¢ ¢ Basic concepts Implicit free lists 21

Method 1: Implicit List ¢ ¢ Heap is divided into variable-sized blocks Each block has a header containing size and allocation status § some low-order address bits are always 0 § Instead of storing an always-0 bit, use it as a allocated/free flag § When reading size word, must mask out this bit 1 word header Format of allocated and free blocks Size Payload a Size: block size a = 1: Allocated block a = 0: Free block Payload: application data (allocated blocks only) Optional padding 22

Implicit List: Allocating in Free Block ¢ Allocating in a free block: splitting § Since allocated space might be smaller than free space, we might want to split the block 4 4 6 2 p malloc(3) 4 4 4 2 2 p free space being allocated space allocated split free space 23

Implicit List: Freeing a Block ¢ Simplest implementation: § Need only clear the “allocated” flag § Problem? § Leads to “false fragmentation” 4 4 2 2 p free(p) 4 malloc(5) 4 4 4 Oops! There is enough free space, but the allocator won’t be able to 24

Implicit List: Coalescing ¢ Join (coalesce) with next/previous blocks, if they are free § Coalescing with next block 4 4 4 2 2 p free(p) 4 4 6 logically gone Check if next block is free 4 4 6 2 How to coalesce with a previous block? 25

Implicit List: Bidirectional Coalescing ¢ Boundary tags § Replicate size/allocated word at “bottom” (end) of free blocks § Allows us to traverse the “list” backwards, but requires extra space 4 ¢ 4 4 4 6 6 4 4 What does this look like? § Doubly Linked List. Header Format of allocated and free blocks Boundary tag (footer) Size a Payload and padding Size a = 1: Allocated block a = 0: Free block Size: Total block size Payload: Application data (allocated blocks only) a 26

Constant Time Coalescing Block being freed Case 1 Case 2 Case 3 Case 4 Allocated Free 27

Constant Time Coalescing (Case 1) m 1 1 m 1 n 1 0 n m 2 1 1 n m 2 0 1 m 2 1 28

Constant Time Coalescing (Case 2) m 1 1 m 1 n+m 2 1 0 n m 2 1 0 m 2 0 n+m 2 0 29

Constant Time Coalescing (Case 3) m 1 0 n+m 1 0 m 1 n 0 1 n m 2 1 1 n+m 1 m 2 0 1 m 2 1 30

Constant Time Coalescing (Case 4) m 1 0 m 1 n 0 1 n m 2 1 0 m 2 0 n+m 1+m 2 0 31

Additional requirements: alignment Start of heap 8/0 16/1 32/0 16/1 0/1 Double word aligned Free blocks Unusable Headers: labeled with size in bytes/allocated bit Allocated blocks: shaded 32

Implicit List: Finding a Free Block ¢ First fit: § Search from beginning, choose first free block that fits: § Time taken? § ¢ Linear time for number of blocks (both allocated and free) Next fit: § Like first fit, except search starts where previous search finished § Faster than the first? Should be faster: avoids re-scanning unhelpful blocks § Some research suggests that fragmentation is worse § ¢ Best fit: § Search the list, choose the best free block: fewest wasted bits § Keeps fragments to a minimum (best utilization) § Speed? § Will usually run slower than either 33

Key Allocator Policies ¢ Placement policy: § First-fit, next-fit, best-fit, etc. § Trades off lower throughput for less fragmentation ¢ Splitting policy: § When do we go ahead and split free blocks? § How much internal fragmentation are we willing to tolerate? ¢ Coalescing policy: § Immediate coalescing: coalesce each time free is called § Deferred coalescing: try to improve performance of free by deferring coalescing until needed. Examples: § Coalesce as you scan the free list for malloc § Coalesce when the amount of external fragmentation reaches some threshold 34

Implicit Lists: Summary ¢ ¢ Implementation: very simple Allocate cost: § linear time worst case ¢ Free cost: § constant time worst case ¢ Memory usage: § will depend on placement policy § ¢ first-fit, next-fit or best-fit Not used in practice for malloc/free because of lineartime allocation § used in many special purpose applications 35