Graph processing with PUMA Stijn Eyerman Wim Heirman

Graph processing with PUMA Stijn Eyerman, Wim Heirman, Ibrahim Hur, Joshua B. Fryman

DARPA HIVE project “To build a graph analytics processor that can process streaming graphs 1000 X faster and at much lower power than current processing technology” Intel’s response: Programmable Unified Memory Architecture (PUMA) § Why is graph processing challenging? § What makes PUMA more fit to graph processing? § How does PUMA’s architecture look like? § How well does PUMA perform? § Can we buy PUMA? – Not yet Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 2

Graph applications are no good match for current processors Regular applications: • Branch predictor: branches have regular pattern • Caches: same or neighboring data is likely reused • Vector: same operation on neighboring data Graph analysis applications: • Branch outcome data-dependent • Randomly scattered single element accesses • Scattered data, indirect accesses Very low performance GPU has similar problems (branch divergence, low memory coalescing) and memory capacity and scale-out limitations Solutions: • Graph accelerator: fixed functionality (sparse linear algebra, vertex or edge centric), needs host and data transfer • Instruction set processor optimized for graph applications: flexible, self-contained PUMA Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 3

Graph processing challenges Xeon core Cache MPI TCP IP NIC PUMA Memory MPI TCP IP NIC Hardware distributed shared memory Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 4

PUMA offload engines boost performance and efficiency PUMA wait if contended atomics gather lock data load data update data write data unlock data load A[0] r 1 load B[r 1] r 2 store r 2 C[0] load A[1] r 1 load B[r 1] r 2 store r 2 C[1] invalidate copies in all caches write back to memory atomic update data instr x instr y instr z offload engine dma_gather A, B, C, n instr x instr y instr z dma_wait (strided) copy, barrier, queue, etc. A Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 B C 5

PUMA core • 1 PUMA core = multiple processor pipelines with multiple threads per pipeline • Novel 64 -bit RISC instruction set, graph optimized instructions (e. g. , single instruction indirect load) • Local scratchpad with unique address range in global address space (GAS), accessible for all other cores • Offload engine performs operations in the background, initiated by non-blocking instruction in the pipelines • Memory controller with 8 B granularity, addressing partition of GAS • Network interface with 8 B packets Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 6

PUMA hierarchical system § Multiple cores tile, multiple tiles node, multiple nodes system; full system = millions of threads § Distributed global address space across the full system § Hyper. X network topology: hierarchical topology with all-to-all connections on each level (tile, node, system) low diameter § Optical connections between tiles and between nodes for high bandwidth and low latency Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 7

Programming PUMA Work in progress! § Initial low-level programming – SPMD or task-based parallelism – C with PUMA specific instructions (offload) using intrinsics – LLVM compiler § Currently support for C++, pthreads, Open. MP, tasking § Under development: graph libraries (Graph. BLAS), other programming languages (Open. CL), Python frontend Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 8

PUMA evaluation A PUMA configuration Simulation PUMA binary Functional simulation (up to a few tiles) performance Timing simulation profile data Analytical model (full system extrapolation) inner loop instrs max execution time Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 9

PUMA performance comparison single node Graph kernels/apps graph benchmarks Random walks, Graph search, Pagerank, Sp. MV, Louvain, BFS, etc. 4 sockets, 72 cores optimize multi-node Xeon cluster Infiniband MPI shared memory no speedup or slowdown ~ equal power port & optimize PUMA 1 node 16 node projections Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 10

Speedup of PUMA versus 1 Xeon node Application PUMA 1 node PUMA 16 nodes Application Classification 6. 9 x 111 x Graph Wave 8. 0 x 125 x Random Walks 279 x 2, 606 x Parallel Decoding FST 6. 8 x 109 x Graph Search 34 x 544 x Geolocation 15 x 243 x Louvain Community 41 x 555 x Sp. MV 29 x 467 x TIES Sampler 93 x 419 x Sp. MSp. V 111 x 1, 387 x Graph 2 Vec 42 x 178 x Breadth-First Search 7. 5 x 117 x Graph. SAGE 3. 1 x 46 x Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 11

Conclusions PUMA is a programmable instruction set processor optimized for sparse graph applications Main features: high-bandwidth system-wide shared memory, small granularity memory accesses, massive multithreading, configurable caching, offload engines One PUMA node is 1 to 2 orders of magnitude faster than one Xeon node, and scales well to multinode Hardware and software environment still under development Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 12

Impact of PUMA features on Sp. MV performance sparse matrix row 1 nz_idx 35 Speedup vs. Xeon row 0 X dense vector nz_val nz_idx nz_val . . . r = row_id(thread_id); for(i=0; i<nnz[r]; i++) C[r] += nz_val[r][i] * V[nz_idx[r][i]]; 30 25 20 15 10 5 Plain • Nothing cached • Many threads hide memory latency • 8 B accesses do not waste bandwidth Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 14

Impact of PUMA features on Sp. MV performance sparse matrix row 1 nz_idx 35 Speedup vs. Xeon row 0 X dense vector nz_val nz_idx nz_val . . . r = row_id(thread_id); nz_val_c = CACHE_ADDR(nz_val); 30 25 20 15 10 5 0 for(i=0; i<nnz[r]; i++) C[r] += nz_val_c[r][i] * V[nz_idx[r][i]]; Plain Selective caching • nz_val accessed sequentially cache exploits spatial locality lower latency Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 15

Impact of PUMA features on Sp. MV performance sparse matrix row 1 nz_idx 35 Speedup vs. Xeon row 0 X dense vector nz_val nz_idx nz_val . . . 30 25 20 15 10 5 r = row_id(thread_id); 0 nz_val_c = CACHE_ADDR(nz_val); scratchpad double buf[N]; puma_dma_gather(V, nz_idx[r], buf, nnz[r]); puma_dma_wait(); for(i=0; i<nnz[r]; i++) C[r] += nz_val_c[r][i] * buf[i]; Plain Selective caching DMA gather • dma_gather performs accesses in parallel • does not move nz_idx to core • scratchpad for low latency Graph processing with PUMA -- Stijn Eyerman, Wim Heirman -- FOSDEM 2020 16