Analysis of the Impact of CPU Virtualization on

Analysis of the Impact of CPU Virtualization on Parallel Applications in Xen Kenichi Kourai Riku Nakata Kyushu Institute of Technology

Parallel Applications in VMs ¡Cloud computing is being used for running parallel applications ¡Users can use necessary resources on demand ¡Users can pay only for used resources ¡Iaa. S provides virtual machines (VMs) ¡Users can use any OS and libraries required by applications application OS CPU 0. . . CPUn VM cloud

CPU Virtualization ¡A VM owns virtual CPUs (v. CPUs) ¡Thread scheduling: Application threads are scheduled to v. CPUs by OS in a VM ¡v. CPU scheduling: v. CPUs are scheduled to physical CPUs (p. CPUs) by the hypervisor VM 1 v. CPU 0 thread OS OS v. CPU 1 v. CPU 0 VM 2 v. CPU 1 hypervisor processor 1 p. CPU 0 p. CPU 1 p. CPU 2 p. CPU 3 processor 2

v. CPU Scheduling ¡p. CPUs are assigned to arbitrary v. CPUs ¡If there are no constraints ¡v. CPU affinity ¡Enable the static binding of v. CPUs to specific p. CPUs ¡NUMA-node affinity ¡Attempt to assign p. CPUs in the same node to a VM v. CPU 0 processor 1 p. CPU 0 v. CPU 1 p. CPU 1 memory NUMA node 1 v. CPU 2 v. CPU 3 p. CPU 2 p. CPU 3 processor 2 memory NUMA node 2

Impact of CPU Virtualization ¡CPU virtualization can affect the performance of parallel applications ¡Extra v. CPU scheduling is performed ¡The thread scheduler cannot obtain physical information precisely VM 1 v. CPU 0 thread OS OS v. CPU 1 v. CPU 0 ? VM 2 v. CPU 1 hypervisor processor 1 p. CPU 0 p. CPU 1 p. CPU 2 p. CPU 3 processor 2

Experimental Setup ¡Target application ¡Fibonacci parallelized by Tascell [Hiraishi et al. '09] ¡A representative of searching applications ¡Tascell: backtrack-based dynamic load balancing ¡E. g. , fib(48) = fib(47) + fib(46) 48 ¡Worker A is executing fib(48) ¡Worker B steals the task of fib(46) from A PC Processor Opteron 6376 (16 cores) x 2 worker A VM v. CPU 47 16 Memory 320 GB Memory 4 GB Hypervisor Xen 4. 4. 0 OS Linux 3. 13. 0 46 worker B

Virtualization Overhead ¡We measured the execution time of tascell-fib in a VM and a physical machine (PM) ¡Change the number of threads from 1 to 16 ¡Assign 16 p. CPUs only from one processor to a VM VM PM . . . tascell-fib OS. . . v. CPUs hypervisor. . . p. CPUs . . . tascell-fib OS. . . p. CPUs

Comparison between VM and PM ¡Scalability in a VM was strange ¡Steeply slow down in 4 threads ¡Scale well again in more than 8 threads time (sec) 64 72% longer in 8 threads 32 16 VM 8 PM 4 1 2 4 # of threads 8 16

Identify Problematic Scheduler(s) ¡We examined which scheduler causes the strange scalability ¡To use only the v. CPU scheduler ¡We assigned each thread to a different v. CPU ¡To use only the thread scheduler ¡We assigned p. CPUs to v. CPUs one by one. . . thread VM thread scheduler. . . v. CPUs v. CPU scheduler. . . p. CPUs

Problematic Scheduler ¡Only the v. CPU scheduler is problematic ¡Scalablity was similar to when using both schedulers v. CPU scheduler only thread scheduler only both schedulers time (sec) 64 32 16 8 1 2 4 # of threads 8 16

Change p. CPU Distribution ¡Our assumption ¡The strange scalability is caused by p. CPU assignment from only one processor ¡We changed p. CPU distribution m-n between two processors ¡m and n p. CPUs are assigned from processor 1 and 2 ¡Assign 16 p. CPUs in total from two processors to a VM 6 -10 6 processor 1 VM 10 processor 2

Impact of p. CPU Distribution ¡p. CPU distribution largely affected scalability ¡ 0 -16: one of the worst (used in previous experiment) ¡ 8 -8: one of the best (still slowed down in 8 threads) 0 -16 3 -13 6 -10 time (sec) 64 1 -15 4 -12 7 -9 2 -14 5 -11 8 -8 32 16 38% faster 8 1 2 4 # of threads 8 16

p. CPU Usage ¡We measured the p. CPU utilization in 8 threads CPU utilization (%) ¡ 8 -8: p. CPUs in processor 1 and 2 were sparsely used ¡ 0 -16: Only the first half of the p. CPUs in processor 2 was densely used 0 -16 140 120 100 80 60 40 20 0 8 -8 processor 1 8 -8 processor 2 0 -16 8 -8 0 2 4 6 8 10 12 14 16 18 20 p. CPU number 22 24 26 28 30

p. CPU Conflicts ¡Clustered Multi-Thread microarchitecture ¡ 2 cores are packaged as a module ¡Share an instruction decoder, L 2 cache, and an FPU ¡Performance can be degraded when 2 cores in a module are competing ¡We measured the p. CPU utilization ¡Calculate the number of modules whose 2 cores were used simultaneously p. CPU 0 decoder p. CPU 1 L 2 module 0 FPU p. CPU 2 decoder p. CPU 3 L 2 module 1 FPU . . .

Competing Modules ¡Strong correlation with performance ¡ 8 -8: contention gradually increased ¡ 0 -16: contention increased in 4 -8 and 12 -16 threads ¡The cause of strange scalability 4 0 -16 3 8 -8 64 contention 2 1 0 time (sec) # of modules 5 performance 32 0 -16 16 8 -8 8 0 2 4 6 8 10 12 14 16 # of threads 1 2 4 8 # of threads 16

Changes of p. CPU Usage ¡We inspected how CPU cores were used ¡ 8 -8: only one core per module was used until 8 threads ¡The other core was used in more than 8 threads ¡ 0 -16: only one core per module was used in 1 -4 and 9 -12 threads ¡The other core was used in 5 -8 and 13 -16 threads processor 1 processor 2

Changes of p. CPU Usage ¡We inspected how CPU cores were used ¡ 8 -8: only one core per module was used until 8 threads ¡The other core was used in more than 8 threads ¡ 0 -16: only one core per module was used in 1 -4 and 9 -12 threads ¡The other core was used in 5 -8 and 13 -16 threads processor 1 processor 2

Root Cause ¡Conflict between v. CPU and NUMA-node affinities ¡Xen automatically assigned the node affinity ¡We manually set the v. CPU affinity ¡In our case ¡ 8 -8: nodes 0 and 2 were equally used (no conflict) ¡ 0 -16: node 2 was used first (due to conflict) 8 -8 processor 0 1 node p. CPU 0 0 -7 1 8 -15 2 16 -23 3 24 -31 0 -16 processor 0 1 node p. CPU 0 0 -7 1 8 -15 2 16 -23 3 24 -31

Disable v. CPU/Node Affinities ¡Using no affinities achieved the best scalability ¡Using either v. CPU or node affinity achieved almost the same performance as 8 -8 time (sec) 64 32 16 0 -16 only v. CPU affinity only node affinity no affinities 8 4 1 2 4 # of threads 8 16

Optimal v. CPU Affinity ¡Manually assign only one core per module ¡p. CPUs 0, 2, 4, . . . , 30 ¡Similar to no affinities ¡ 4. 6% higher in 16 threads time (sec) 64 32 16 no affinities opt 8 4 1 2 4 # of threads 8 16

Competing Modules Reduced ¡Always zero in the optimal v. CPU affinity ¡Very small but not zero in no affinities ¡Similar to 8 -8 in either v. CPU or node affinity # of modules 5 0 -16 only v. CPU affinity only node affinity no affinities opt 4 3 2 1 0 0 2 4 6 8 10 # of threads 12 14 16

Other Parallel Applications ¡We examined the NAS Parallel Benchmarks ¡The performance in the optimal v. CPU affinity was always the highest ¡The performance in 0 -16 was lower than that in 8 -8 ¡Except EP (scaled very well in any cases) 64 MG time (sec) 16 8 0 -16 8 -8 opt 4 2 LU 32 16 0 -16 8 -8 opt 8 4 1 2 4 8 # of threads 16

Causes of Degradation in 0 -16 ¡We measured resource conflicts using hardware performance counters instruction fetch stall ¡Cause in tascell-fib ¡FPU dispatch stalls ¡LLC misses increase ¡Instruction fetch stalls 0. 3 0. 2 0. 1 0. 0 -0. 1 -0. 2 BT CG DC EP FT IS LU MG SP UA increase LLC miss 0. 8 0. 6 0. 4 0. 2 0. 0 BT CG DC EP FT IS LU MG SP UA

Related Work ¡Intel Hyper-Threading ¡Two threads in one CPU core share an integer unit and L 1/L 2 cache ¡Performance bottlenecks are different ¡MASA [Nakajima et al. '02] ¡Assist thread scheduling for Hyper-Threading ¡Applicable to v. CPU scheduling ¡Lock-holder preemption [Uhlig et al. '04], v. CPU stacking [Sukwong et al. '11] ¡Well-known problems on v. CPU scheduling

Conclusion ¡We analyzed the root cause of strange scalability in a VM ¡Conflict between 2 CPU cores in each module ¡Conflict between the v. CPU and node affinities ¡We presented methods to avoid these conflicts ¡Disabling v. CPU/node affinities ¡Optimal v. CPU affinity ¡Future work ¡Develop a v. CPU scheduler that automatically adjusts affinities