Portable Unobtrusive Garbage Collection for Multiprocessor Systems Damien

Portable, Unobtrusive Garbage Collection for Multiprocessor Systems Damien Doligez Georges Gonthier POPL 1994 Presented by Eran Yahav (yahave@math. tau. ac. il)

Portable, Unobtrusive Garbage Collection for Multiprocessor Systems A concurrent, generational garbage collector for a multithreaded implementation of ML - Doligez - Leroy (POPL 1993) On-the-fly garbage collection: an exercise in cooperation Dijkstra et al. (1978)

Overview l Motivation l Concurrent collection strategies l Concurrent collection constraints l The basic algorithm (Dijkstra) l Doligez-Leroy model l Doligez-Leroy concurrent collector

Concurrent GC l Known as a tough problem l Published algorithms contain simplifying assumptions that either: · impose unbearable overhead on mutators · require high degree of hardware/OS support l Other algorithms are buggy

“Stop the world” l all threads synchronically stop and perform GC l introduces sync. between independent threads Sync. GC T 1 T 2 T 3 T 4 Sync. GC

“Stop the world” l all threads synchronically stop and perform GC l introduces sync. between independent threads Sync. GC T 1 T 2 T 3 T 4

“Stop the world” - Mostly Parallel GC (Bohem et. al) l Uses virtual memory page protections l reduces duration of “stop the world” period l does not prevent synchronization between threads at “stop the world” points marking Sync. GC T 1 T 2 T 3 T 4

“Stop the world” - Scalable mark-sweep GC l Uses a parallelization of Bohem’s mostly parallel collector l reduces duration of “stop the world” periods l does not prevent synchronization between threads at “stop the world” points marking Sync. GC T 1 T 2 T 3 T 4

“Stop the world” - Real Time GC (Nettles & O’Toole) l Incremental copying collector l reduces duration of “stop the world” periods l does not prevent synchronization between threads at the swap point Sync. GC T 1 T 2 T 3 T 4

Concurrent collector l run the collector concurrently with user threads l use as little as possible sync between user threads and GC thread GC T 1 T 2 T 3 T 4

Concurrent Collection strategies l Reference counting l copying (relocation) l mark & sweep

Concurrent GC - Reference counting l Locks on reference counters 1 M 1 RC = 2 M 3 1+ heap

Concurrent GC - relocation l relocating objects while mutators are running M 1 ? from ? M 2 GC heap to

Concurrent GC - relocation l relocating objects while mutators are running l must ensure that mutators are aware of relocation · test on heap pointer deref · extra indirection word for each object · virtual memory page protections l significant run-time penalty

Concurrent GC - mark/sweep l Mark all threads roots l No inherent locks l Mutators may change trace graph during any collection phase Global variables Threads 1 Heap 2 3

Multiprocessors facts of life l Registers are local · impossible to track down machine registers of a running process l Synchronization is expensive · semaphores and synchronization are only available through expensive system calls

Unobtrusive? l No overhead on frequent actions: · move data between registers and memory · deref a heap pointer · fill a field in a new heap object l imposes sync. overhead only on reserve actions (for which it is unavoidable) l mutator cooperation with collector is done only at mutator’s convenience

Portable ? l No special use of OS synchronization primitives l no hardware support

Where all else fail l relocating GC algorithms break locality or impose large overhead l proposed incremental algorithms requires global synchronization l mark & sweep - collector working while mutators change trace graph - complicated but possible

The basic algorithm l Dijkstra et al. - “On the fly garbage collection” l published in 1978 l breaks locality l assumes fixed set of roots Global variables Threads 1 Heap 2 3 GC

Dijkstra’s collector Mark: for each x in Globals do Mark. Gray(x) Sweep: for each x in heap do if color[x] = white then append x to free list else if color[x] = black then color[x] white p white update ee mark sw gray sweep mark black allocate Scan: repeat dirty false for each x in heap do if color[x] = Gray then dirty true Mark. Gray(x. Sons) color[x] black until not dirty

Doligez-Leroy model l Damein doligez & Xavier Leroy at 1993 l a concurrent, generational GC for multithreaded implementation of ML l relies on ML properties: · compile time distinction between mutable and immutable objects · duplicating immutable objects is semantically transparent l does not stop program threads

Doligez-Leroy model l Do anything to avoid synchronization l trade collection “quality” for level of synchronization - allow large amounts of floating garbage l trade collection “simplicity” for level of synchronization - complicated algorithm (not to mention correctness proof)

Doligez-Leroy model Threads 1 Stacks Minor heaps Major heap 2 3 Global variables

Collection generations l Each thread treats the two heaps (private and shared) as two generations · private = young generation · shared = old generation l immutable objects are allocated in private heaps · does not require synchronization l mutable objects handled differently (later)

Minor collection l When private heap is full - stop and perform minor collection l copy live objects from private heap to shared heap (old generation) l after minor collection, whole private heap is free l can be performed in any time l synchronization is only required for allocation of the copied object on shared heap

Major collection l Dedicated GC thread l uses a variation of Dijkstra’s algorithm (mark & sweep) l does not move objects, no synchronization is required when accessing/modifying objects in shared heap l will be described later

Major and minor collection Threads 1 Stacks Minor heaps Major heap 2 3 GC Global variables

Copy on update l We assumed no pointers from shared heap to private heap Not reachable from thread’s roots Major heap

Copy on update l Copy the referenced object (and descendents) l similar to minor collection with a single root l simply does some of the minor collection right away Major heap

Copy on update l Until next minor collection, copying thread can access original and copied objects l immutable objects - semantically equivalent l what about mutable objects ? Major heap

Allocation of mutable objects l If copied - can update both objects separately l no equivalence of original and copied object l solution: always allocate mutable objects in the shared heap l requires synchronization (free list) l ML programs usually use few mutable objects l mutable objects have longer life span than average

The Concurrent collector l Adapted version of Dijkstra’s algorithm l naming conventions · mutator = thread + minor collection thread · collector = major collector l major collector only requires marking of mutator roots. l does not demand minor collections

Four color marking l White - not yet marked (or unreachable) l Gray - marked but sons not marked l Black - marked and sons marked l Blue - free list blocks Heap

Collection phases l Root enumeration l end of marking l sweeping

Root enumeration l Raise a flag to signal beginning of marking l shade globals l ask mutators to shade roots l wait until all mutators answered l meanwhile - start scanning and marking

Root enumeration Collector Mark: for each x in Globals do Mark. Gray(x) call mutator to mark roots wait until all mutators answered. . . Mutators Cooperate: if call to roots is pending then call Mark. Gray on all roots answer the call

End of marking l Repeatedly mark gray objects until no more gray objects remain Scan: repeat dirty false for each x in heap do if color[x] = Gray then dirty true Mark. Gray (x. Sons) color[x] black until not dirty

Sweeping l Scan heap l All white objects are free - set to blue and add to the free list l all black objects are reset to white l some object might have been set to gray since the end of marking phase - set to white

Invariants (1/2) l All objects reachable from mutator roots at the time mutator shaded its roots, or that become reachable after that time are black at the end of the marking phase Objects can become reachable by allocation and modification which are performed concurrently with the collection

Invariants (2/2) l gray objects that are unreachable at the beginning of the mark phase become black during mark, then white during sweep and reclaimed by the next cycle (floating garbage) l all white objects unreachable at the start of the marking phase remain white l No unreachable object ever becomes reachable again l there are no blue objects outside the free list

Concurrent allocation and modification l Mutators must consider collector status when performing modification or allocation of heap objects l first, lets consider modification of heap objects

Concurrent modification l Updating a black object could result in a reachable object that remains white at the end of marking l even worse - the set of roots is not fixed during collection l must shade both the new value and the old value

What happens if we don’t shade new value T 1 T 2 A Major heap B Root enumeration Mark T 1 root T 2 updates A T 2 pops

What happens if we don’t shade new value T 1 T 2 A Major heap B Root enumeration Mark T 1 root T 2 updates A T 2 pops

What happens if we don’t shade new value T 1 T 2 A Major heap B Root enumeration Mark T 1 root T 2 updates A End mark T 2 pops Mark T 2 root Sweep

What happens if we don’t shade old value T Major heap A B Root enumeration End mark Mark T root T pushes B

What happens if we don’t shade old value T Major heap A B Root enumeration End mark Mark T root T pushes B

What happens if we don’t shade old value T Major heap A B Root enumeration End mark Mark T root T pushes B Sweep T updates A

Concurrent Allocation l Assign right color to new objects l during marking - allocated objects are black · allocated are reachable · sons of allocated are reachable and will eventually be set to black l sweeping - white if already swept, gray otherwise · set to gray to avoid immediate deallocation

Synchronization l It is always safe to set an object to gray l setting many objects to gray is inefficient · will be only reclaimed on next cycle l allows us to avoid synchronization when race condition can end up making an object gray l used to test collector status without locking

Synchronization l Coloring of newly allocated block M S S S S S . 1 If phase = marking then. 2 Set object to black. 3 If phase = sweeping then. 4 Set object to gray. 5 Else. 6 If address(object) < sweep-pointer then. 7 Set object to white. 8 Else. 9 Set object to gray

Color transitions summary mark black gray sw e ep allocate blue sweep ate c o l al sweep allocate white mark update

Experimental results

Corrections l When shading old value - what old value do we shade ? l Another thread might “replace” old value, after we shade it, by a non-shaded value l this is corrected by adding another handshake all updates must end before we start marking

Summary l Doligez Leroy & Gonthier concurrent GC l does not stop program threads l four colors mark & sweep - white, gray, black and blue l relies on ML language properties, but can be extended for other languages

The End