NUMA Tuning for Java Server Applications Mustafa M

NUMA Tuning for Java Server Applications Mustafa M. Tikir

Introduction l Cache-coherent SMPs are widely used – High performance computing – Large-scale applications – Client-server computing l cc-NUMA is the dominant architecture – Allows construction of large servers – Data locality is an important consideration • Faster access to local memory units 2 Platform Local Access Remote Access Ratio Sun Fire 6800 225 ns 300 ns 1: 1. 33 Sun Fire 15 K 225 ns 400 ns 1: 1. 78 SGI Altix 3000 145 ns 605 ns 1: 4. 17 University of Maryland

Dynamic Page Migration l Effective for scientific applications – Regular memory access patterns l Large static arrays with many pages – Divided into segments – Distributed to multiple computation nodes • A few nodes access each data segment most l Our earlier work – Moved pages at fixed time intervals • Profiles gathered from hardware counters – Resulted in up to • 90% reduction in non-local accesses • 16% improvement in execution times 3 University of Maryland

Java Server Applications l Java programs – Make extensive use of heap-allocated memory – Typically have significant pointer chasing l Dynamic page migration may not be as beneficial – A page may have objects with different access patterns • Page placement is transparent to the standard allocation routines • Larger page size increases the likelihood – cc-NUMA servers tend to use super pages l Heap objects should be allocated or moved – Local to the processor accessing them most • Migration at the object granularity 4 University of Maryland

Page Migration for SPECjbb 2000 % Reduction 27. 4 24. 5 20. 9 % Improvement -2. 8 -3. 4 -3. 1 l Around 25% reduction in non-local accesses – Unlike scientific applications where it is up to 90% l Around 3% reduction in throughput – Overhead due to migrations of many pages 5 University of Maryland

Memory Behavior at Object Granularity l Source code instrumentation of Hot. Spot VM – Object allocations by the Java application – Internal heap allocations by the VM – Changes in object addresses due to garbage collection l Instrumentation using dyninst – Additional helper thread • For address transaction sampling • Via Sun Fire Link hardware counters l Execution is divided into distinct intervals – Execution interval • Gathers information on object allocations and accesses – Garbage collection interval • Dumps allocation and transaction buffers 6 University of Maryland

Experiments using SPECjbb 2000 Java Heap Region Young Generation % Non-Local 83. 7 Eden Space 11, 389, 586 34. 2 83. 8 Survivor Space 536, 645 1. 6 82. 7 16, 477, 990 236, 189 3, 755, 937 49. 5 0. 7 11. 3 84. 0 81. 0 80. 4 Old Generation Permanent Generation Internal Structures l Memory Accesses Count % 11, 926, 231 35. 8 Young Generation – Objects are initially allocated – Objects stay in until old enough to be tenured • Survivor spaces l Tenured (old) Generation – Objects reaching a certain age are promoted l Permanent Generation – The reflective data of the VM are allocated – Such as class and method objects 7 University of Maryland

Potential Optimizations l Estimation study using finer grained techniques – Based on information gathered during measurement • Heap allocations and accesses l Potential object centric optimizations – Static-optimal placement • Has information on all object accesses • Places objects at allocation time – Prior-knowledge placement • Has information on object accesses during the next execution interval • Moves objects at garbage collection time – Object-migration placement • Gathers information since the start of execution • Moves objects at garbage collection time 8 University of Maryland

Estimation Study Results l Migration is effective in old generation – Many objects in young generation die fast l One or a few processors access objects in young generation – Majority of accesses are from the allocator processor l 9 SPECjbb 2000 has some dynamically changing memory behavior in the old generation University of Maryland

NUMA-Aware Java Heaps l NUMA-Aware heap layouts – NUMA-Eden • NUMA-Aware young generation • Original old generation • Focus on the objects in the young generation – NUMA-Eden+Old • NUMA-Aware young generation • NUMA-Aware old generation – Combined with dynamic object migration • Focus on the access locality to all objects 10 University of Maryland

NUMA-Aware Young Generation l We divide eden space into segments – Each locality group is assigned a segment • Pages in each segment are placed local to the group l Object allocation – The requestor processor is identified – Object is placed in the segment of the processor’s group l Garbage collection – When a segment does not have enough space – Other segments are also collected even if not full • Potentially eliminates future synchronization 11 University of Maryland

NUMA-Aware Old Generation l We divide tenured space into segments – Each locality group is assigned a segment • Pages in each segment are placed local to the group l When an object is promoted to old generation – Preferred location of the object is identified • Processor that accesses the object most – Object is moved to the segment of the processor's group l Object migrations during full garbage collection – Preferred locations of all objects are re-computed l Additional object migrations – At every fixed number of minor collections • To match the dynamically changing behavior 12 University of Maryland

Experimental Setup l Representative Java workloads for simulation – Generated from actual runs – Sequence of requests • To allocate or access objects by processors • Same order as the actual run l Workload Execution Machine – A hybrid execution simulator • Consumes the generated parallel workload • Issues memory allocations and accesses to the machine – Implements the underlying memory management algorithms • Original algorithms in the Hot. Spot VM • Algorithms for NUMA-Aware heap layouts 13 University of Maryland

NUMA-Aware Heap Experiments l Application – SPECjbb 2000 benchmark on Hot. Spot VM • Run with 12 warehouses l Platform – 24 processor Sun Fire 6800 – 24 GB main memory l l Sampling at every 1 K transactions Partial workload from the actual run – 10 M allocation records – 28 M memory accesses l Generated workloads with higher pressure – Scaled 16 and 32 times 14 University of Maryland

Reduction in Non-Local Accesses Scale Factor 16 32 15 Heap Configuration Young Gen. Old Gen. Young + Old Gen. NUMA-Eden 57. 6 % 0. 3 % 28. 1 % NUMA-Eden+Old 55. 3 % 27. 5 % 41. 0 % NUMA-Eden 50. 9 % 1. 2 % 27. 3 % NUMA-Eden+Old 48. 0 % 30. 2 % 39. 5 % University of Maryland

Execution Times l NUMA-aware heaps are effective – 27% improvement for NUMA-Eden configuration – 40% improvement for NUMA-Eden+Old configuration l 16 More effective for higher memory pressure University of Maryland

Conclusions l NUMA-Aware heap layouts – Up to 41% reduction in non-local accesses – Up to 40% improvement in workload execution l Dynamic object migration is beneficial – Compared to using only NUMA-aware young generation l NUMA-aware heaps are more effective – As the memory pressure increases l More effective on larger servers – Sun Fire 15 K (latency ratio => 1: 1. 78) – SGI Altix 3000 (latency ratio => 1: 4. 17) 17 University of Maryland