Heap Shape Scalability Scalable Garbage Collection on Highly

Heap Shape Scalability Scalable Garbage Collection on Highly Parallel Platforms Kathy Barabash, Erez Petrank Computer Science Department Technion, Israel

Outline n Is tracing GC ready for the many-core? q n Evaluating the heap shape scalability q n Idealized Trace Utilization Improving the heap shape scalability q q n How the heap shape is related? Solution 1: Reshaping with Shortcut References Solution 2: Tracing with Speculative Roots Related work & conclusion ISMM 2010 2

Is Tracing GC Ready for Manycore ? n GC tracing q n Roots a Sequential trace q n Traverse lots of objects Each live object is touched (BFS, DFS) b e Parallel trace q q ISMM 2010 Load balancing 1 K cores really soon Heap c d f g j k h i m l 3

Can Heaps Spoil the Scalability? Roots n 1 2 4 M live objects q 4 M n 3 Sequential trace q n Single linked list 4 M steps Parallel trace q Not any faster 4 K Heap ISMM 2010 4

Deep Object Graphs Can be Evil Definition: Object Depth Length of the minimal path from some root object Object-Graph Depth Maximal live object depth Example: Object Depths 0 How deep are object graphs of Java programs? 1 2 Heap ISMM 2010 q Spec. JVM, Dacapo, Spec. JBB q Instrumented BFS trace 3 5

Object-Graph Depths of Java Benchmarks Name Heap Size (MB) GC Cycles Max Depth Java compiler run 3 times 32 15 1, 234 3 D raytracer 32 8 1, 416 Java byte code analyzer 48 344 1, 195 Java code analyzer 48 59 18, 482 Transforms XML into HTML 128 129 8, 476 Description Spec. JVM javac mtrt Dacapo bloat pmd xalan Other 15 benchmarks ISMM 2010 128 6

Object-Graph Depths of Java Benchmarks Name Heap Size (MB) GC Cycles Max Depth Java compiler run 3 times 32 15 1, 234 3 D raytracer 32 8 1, 416 Java byte code analyzer 48 344 1, 195 Java code analyzer 48 59 18, 482 Transforms XML into HTML 128 129 8, 476 Description Spec. JVM javac mtrt Dacapo bloat pmd xalan Other 15 benchmarks ISMM 2010 128 7

Object-Graph Depths of Java Benchmarks Name Heap Size (MB) GC Cycles Max Depth Java compiler run 3 times 32 15 1, 234 3 D raytracer 32 8 1, 416 Java byte code analyzer 48 344 1, 195 Java code analyzer 48 59 18, 482 Transforms XML into HTML 128 129 8, 476 Description Spec. JVM javac mtrt Dacapo bloat pmd xalan Other 15 benchmarks ISMM 2010 128 8

Not all Deep Object Graphs are Evil n Roots Object-graph q 1 2 n 3 Sequential trace q … n 4 K 4 K 4 M steps Parallel trace q 4 K 1 K same sized linked lists of 4 K objects Scales well for up to 1 K processors Heap ISMM 2010 9

Deep and Narrow Object Graphs are Evil Definition: Object Depths Distribution Amount of objects at different depths Example: Graphical Representation (Object-graph shape): #objects 2 4 # objects 1 3 Heap ISMM 2010 1 depth 10

Object-Graph Shapes of Java Benchmarks # objects jython depth # objects xalan depth ISMM 2010 11

# objects (log 10) Object-Graph Shapes of Java Benchmarks db jython jess bloat jack javac lusearch mtrt hsqldb xalan antlr pmd depth (log 10) ISMM 2010 depth (log 10) 12

The Idealized Trace Utilization Simulate the idealized traversal by N threads q q Perfect load balancing Perfect cache behavior q q BFS traversal Single time tick object scan During the traversal, count q q Objects available to be scanned at every time tick Processor slots: some are busy and some are wasted At the end, report the utilization (ITU) Total Scanned Objects * 100% Total Processor Slots ISMM 2010 13

Idealized Trace Utilization Example Core 1 4 Tracers Core 2 Core 3 Core 4 Heap objects Time ticks 1 Scanned objects 2 2 3 4 5 6 7 8 5 9 11 12 13 14 15 15 * 100% = 47 % ITU = * 100% = 8*4 Total Processor Slots Total Scanned Objects ISMM 2010 14

Graphical Representation 2. Draw the graph # objects 1. Simulate and compute depth ISMM 2010 15

Worst Case ITU for Java Benchmarks ISMM 2010 16

Average ITU for Java Benchmarks ISMM 2010 17

What’s Next? n Problematic heaps exist q n javac, mtrt, pmd, bloat, xalan Can we improve the trace scalability without modifying the benchmarks? q Reshape with Shortcut References q Trace with Speculative Roots ISMM 2010 18

Reshape with Shortcut References n Roots Sequential trace q 1 16 K n 2 New references are added q 3 q 4 n Invisible to the program Useful for the tracers Parallel trace q 4 K 16 K steps Scales for 4 processors Heap ISMM 2010 19

Evaluation Prototype n Devise a shortcut strategy q n When the program is stopped for GC q q q n Where shortcuts are needed Compute the Idealized Trace Utilization Run the shortcuts adding algorithm Compute the ITU for the modified heap Report q q ISMM 2010 ITU improvement Amount of shortcuts added 20

Shortcut Strategy and Parameters n Identify candidate subgraphs q With at least size objects q n Size=5 Depth=4 With depth-to-size ratio no less than ratio Ratio=0. 8 Add shortcut to the root of the subgraph q Leading to the objects length pointers away q Next shortcut introduced not closer than distance pointers away Distance (2) 1 ISMM 2010 2 3 4 Length (4) 5 6 7 8 9 21

Results for Spec. JVM mtrt Size=50 Ratio=0. 2 ~ 500 K of live objects Length=50 Max shortcuts – 110 Distance=25 Avg shortcuts – 94 ISMM 2010 22

Results for Da. Capo xalan Size=50 Ratio=0. 2 ~ 400 K of live objects Length=50 Max shortcuts – 888 Distance=25 Avg shortcuts – 536 ISMM 2010 23

Results for Da. Capo bloat Size=50 Ratio=0. 2 ~ 400 K of live objects Length=50 Max shortcuts – 940 Distance=25 Avg shortcuts – 378 ISMM 2010 24

Results for Da. Capo pmd Size=600 Ratio=0. 1 ~ 434 K of live objects Length=120 Max shortcuts – 5, 874 Distance=40 Avg shortcuts – 432 ISMM 2010 25

Results for Spec. JVM javac Size=500 Ratio=0. 1 ~ 383 K of live objects Length=100 Max shortcuts – 292 Distance=50 Avg shortcuts – 16 ISMM 2010 26

Trace with Speculative Roots n Roots Sequential trace q 4 M n Helper tracers q q n 4 K Heap ISMM 2010 16 M steps Pick random roots Trace using custom colors Parallel trace q Scales for 4 processors 27

Speculative Trace n Helper tracer q q q n Regular trace q n Pick up the root Pick up the color, e. g. red Trace; if blue object is discovered, mark blue as reachable from red Trace from root; if blue object is discovered, mark blue as live Complete trace q q ISMM 2010 All colors reachable from live colors marked live All objects marked by live colors survive the collection 28

Evaluation Prototype n n n 4 regular tracers, 4 helper tracers Speculative roots – random unmarked objects ITU before and after the colored trace a b e Useful helpers work c q d f g j k h i Wasted helpers work q m ISMM 2010 Dead objects colored by dead colors Floating garbage q Heap Live objects colored by live colors Dead objects colored by live colors l 29

Limit the floating garbage n Maximal amount of objects colored by a single color q q n Make the random roots choices smarter q q n Helpers must save discovered but not traced objects Trace completion phase takes care of the saved fronts To avoid choosing dead objects To reach deeper parts of the live object graph Filter for the recursive objects q ISMM 2010 Objects with referents of their own type 30

Results n Lots of floating garbage q n Hard to find good roots q n Progressively harder as the live objects are getting marked Trace completion phase is complex q n Even with the filter Can defeat the purpose Modest improvement in the Idealized Trace Utilization scores ISMM 2010 31

Results for Da. Capo xalan Worst case ITU improvement, with the random choices filter ISMM 2010 32

Results for Da. Capo bloat Worst case ITU improvement, with the random choices filter ISMM 2010 33

Related Work n Parallel Garbage Collection Folklore q n Siebert (ISMM’ 08) q q n There are heap structures that can foil any clever load balancing scheme Reported object graph depths for Spec. JVM benchmarks Proposed upper bound on the worst case scalability as a way to compute RT guarantees for the GC tracing Random tracing originally proposed by Click ISMM 2010 34

Summary n Studied the heap shape properties of Java benchmarks q n Devised a measure to quantify the heap shape scalability q n Out of twenty considered benchmarks, five had not scalable heap shapes during the run Idealized Trace Utilization Proposed, prototyped and evaluated two approaches to improve the tracing scalability q ISMM 2010 Reshaping with Shortcuts appears to be more promising than Tracing from Speculative Roots 35

Thank You! ISMM 2010 36