Locality Optimizations in ccNUMA Architectures Using Hardware Counters

Locality Optimizations in cc-NUMA Architectures Using Hardware Counters and Dyninst Mustafa M. Tikir Jeffrey K. Hollingsworth University of Maryland

Introduction l Cache-coherent SMPs are widely used – High performance computing – Large-scale applications – Client-server computing l cc-NUMA is the dominant architecture – Allows construction of large servers – Data locality is an important consideration • Faster access to local memory units Platform 2 Local Remote Access Time Ratio Sun Fire 6800 225 ns 300 ns 1: 1. 33 Sun Fire 15 K 225 ns 400 ns 1: 1. 78 University of Maryland

Data Placement l Memory intensive applications on cc-NUMA servers – May have significant non-local memory accesses l Possible optimization to increase locality – First-touch placement of memory pages • Commonly used in modern systems • May not place pages local to the processors accessing them most – Dynamic page placement/migration • Page access frequencies at runtime 3 University of Maryland

Our Page Migration Approach l User-level dynamic page migration – Profiling and page migration during the same run l Application Profiling – Gathers data from hardware counters • Sample the interconnect transactions – Transaction Type + Physical Address + Processor ID – Identifies preferred locations of memory pages • Memory unit local to the processor that accesses most l Page Placement – Kernel moves memory pages to their preferred locations – At fixed time intervals – Pages are frozen for a while if recently migrated • Eliminates ping-ponging of memory pages 4 University of Maryland

Hardware/Software Components Address Bus Sun Fire Link Hardware Counters Processor 1 Memory Unit (processor_bind) Processor 3 Explicit binding Transaction Sampling Instrumentation Software Processor 2 Application Physical Page Processor 4 Processor 1 Processor 2 Processor 3 Processor 4 Memory Unit Physical Page Sun Fire 6800 Thread 1 5 System Board 2 System Board 1 Virtual to Physical Mapping (meminfo) Threadj Virtual Page University of Maryland Page Migration using move-on-next-touch feature (madvise)

Instrumentation Code Insertion l Instrumentation using Dyninst – Entry point of main • Loads a shared library • Creates two helper threads – One for address transaction sampling – Other for actual migrations of the pages – Exit point(s) of thr_create • Calls processor_bind – Binds new threads to available processors – Helper threads are bound to dedicated processors – Entry point of exithandle • Termination detection • Clean-up hardware counters 6 University of Maryland

Preliminary Experiment l Impractical to record all transactions – Interval sampling • Sampling at every Nth transaction – Continuous sampling • Sampling at the maximum speed of the instrumentation software l 7 Are samples representative of transactions? University of Maryland

Representative Sampling Technique l Potential sampling error – How much do sampled transactions deviate from all transactions? l Distance between two sets – SALL and SSAMPLE – Ratio of transactions requested by a processor, P SAll PA PS 8 University of Maryland SSample

Sampling Error for CG Continuous Sampling Proc 0 Proc 1 Proc 2 Proc 3 Proc 4 Proc 5 Average Error % Rate Sampled l 0. 51 0. 61 0. 47 0. 58 0. 65 0. 57 Interval Sampling at Every 4 K 0. 03 0. 04 0. 01 0. 00 0. 02 0. 03 64 0. 09 0. 23 0. 02 0. 15 17. 56 0. 19 0. 78 3. 07 9. 75 Interval sampling is more representative Continuous sampling is less representative due to difference between the rates – Transaction samples are taken – Processor requests transactions 9 256 0. 03 0. 04 0. 02 0. 01 0. 02 0. 03 0. 56 0. 02 0. 11 – Interval used also has an impact l 1 K 0. 03 0. 04 0. 02 0. 01 0. 02 University of Maryland

Page Migration Experiments l Applications – Open. MP C implementation of NAS Parallel Benchmark suite • BT(B), CG(C), EP(C), FT(B), LU(C), MG(B), SP(C) • Optimized to support parallelized code l Platform – 24 processor Sun Fire 6800 – 24 GB main memory l Execution – 12 threads • 2 threads on each system board – Page migration at every 5 seconds – Interval sampling at every 1 K transactions 10 University of Maryland

Reduction in Non-Local Memory Accesses Reduction 38. 0% 81. 0% 67. 0% 54. 0% 19. 7% 89. 6% 58. 8% Trans. (M) 38, 507 15, 721 42 2, 297 48, 682 841 116, 116 11 University of Maryland

Performance Improvement # Migrations 112, 310 47, 213 2, 071 177, 602 132, 696 49, 884 138, 943 Original Time 996 625 293 113 1, 981 31 3, 901 % Overhead 1. 2% 0. 8% 0. 1% 12. 8% 0. 7% 10. 2% 0. 5% 12 University of Maryland

SPECjbb 2001 Results % Reduction 25. 3% 26. 1% 24. 4% Trans. (M) 2, 017 2, 626 2, 621 l Potential improvement? – Migration working at object granularity 13 University of Maryland

MG. B Address Space [0 -512 MB) 14 University of Maryland

MG. B with Page Migration 15 University of Maryland

Conclusions l Our dynamic page migration approach – Reduced non-local memory accesses by upto 90% – Improved the execution times by upto 16% l Potentially more effective on larger cc. NUMA servers – Sun Fire 15 K (latency ratio => 1: 1. 78) l User level page migration approach – Relies on the OS kernel to provide the actual migration mechanism. 16 University of Maryland

Questions? ? ? 17 University of Maryland