The 24 th Geant 4 Collaboration Meeting Jefferson

The 24 th Geant 4 Collaboration Meeting Jefferson Lab, VA, USA September 23~28, 2019 Geant 4 profiling for using the KISTI-5 supercomputer Kihyeon Cho, Insung Yeo and Myeong Hwan Mun (KISTI) 1

2

Contents • Introduction • Profiling using brachytherapy 1. KISTI-4 (tachyon 2) 2. KISTI-5 (KNL) 3. KISTI-5 (Skylake) • Results • Summary 3

Introduction 4

The changing nature of scientific research e-Science 8 Prof. D. Reed

• Supercomputing meets machine learning. ⇒ Evolving computing architecture 6 Prof. J. Lee

Evolving Computing Architecture • KISTI Supercomputing Center – KISTI-4 ⇒ KISTI-5 KISTI-4 KISTI-5 7

8 T. Hong

9 T. Hong

KISTI-5 supercomputer 10

KISTI-5 supercomputer building 11

Architecture of KISTI-5 supercomputer (SSD) @ Available KNL CORE Time = Available SRU Time x 4, 352 @ Available SKL CORE Time = Available SRU Time x 1, 280 1 accout: 20, 000 CPU*hour M. S. Joh 12

KISTI-4 vs. KISTI-5 Specification KISTI-4 Name Tachyon 2 Model SUN Blade 6275 Process Intel Xeon X 5570 (Nehalem) 2. 93 GHz Intel Xeon Phi 7250 (KNL) 3. 0464 TFlops/CPU Intel Xeon 6148 (Skylake) 1. 536 TFLops/CPU Node 8 core/node 3, 200 node 68 core/CPU 1 CPU/node 8, 305 node 20 core/CPU 2 CPU/node 132 node Core 25, 408 564, 740 5, 280 Rpeak 0. 3 Pflops 25. 3 Pflops 0. 4 Pflops Memory DDR 3/1333 MHz 16 GBx 6, 6 Ch/CPU 96 GB/node 778. 6 TB 16 GBx 12, 6 Ch/CPU 192 GB/node 24. 8 TB 76. 8 TB Storage KISTI-5 Nurion KNL Nurion Skylake Cray C 5500 234 TB disk 2. 3 PB disk 21 PB disk 0. 8 PB SSD 2. 1 PB Tape 10 PB Tape Interconnect Infiniband 40 G 4 XQDR OPA@12. 3 GB/s Fat-Tree, 50% Blocking Service date 2010. 8~2018. 11 2018. 10~

KISTI-4 vs. KISTI-5 Gaussian Quantum Espresso (QE) • No significant performance improvement for Gaussian • ~8 x performance improvement for QE on single node. 14 ⇒ How about Geant 4?

Profiling using Brachytherapy 15

Profiling system Content Geant 4 Version Brachytherapy 10. 02 Brachy version • 10. 03 p 02 10. 03 p 03 10. 04 10. 02 Macro file – Iodium. Source. Macro. mac (endocavitary brachytherapy) – Iridium. Source. Macro. mac (interstitial brachytherapy) – Leipzig. Source. Macro. mac (superficial brachytherapy) • Physics list – QGSP_BIC_LIV – QGSP_BIC_EMZ – QGSP_BIC_EMY • Supercomputer – KISTI-4 (Tachyon 2) ⇒ Last year’s collaboration meeting – KISTI-5 (Nurion KNL) – KISTI-5 (Nurion Skylake) • 1 sequential core to compare with KISTI-4 16

1. KISTI-4 (Tachyon 2) ※ We had shown this at the last year’s collaboration meeting. 17

QGSP BIC EMY A significant suppression of sampling atomic de-excitation

QGSP BIC EMZ

QGSP BIC LIV

2. KISTI-5 (Nurion KNL) 21

QGSP BIC EMY

QGSP BIC EMZ

QGSP BIC LIV

3. KISTI-5 (Nurion Skylake) 25

QGSP BIC EMY

QGSP BIC EMZ

QGSP BIC LIV

Results Macro file: Iodium. Source. Macro. mac Physics list : QGSP_BIC_EMZ 29

CPU Time in sec/event 30

Total memory in count(First event) 160 140 120 100 MB 80 60 40 20 0 1 10. 02 2 10. 03 3 10. 03 P 02 4 10. 03 P 03 5 10. 04 Geant 4 Version 31

Total Memory in count (Last event) 400 350 300 MB 250 200 150 100 50 0 10. 021 10. 032 3 10. 03 P 02 4 10. 03 P 03 5 10. 04 Geant 4 Version 32

Summary • Done – Low energy physics profiling code – Version dependency – KISTI-4 vs. KISTI-5 supercomputer • Future – To use parallel processing with more cores – Open. Speed. Shop – To prepare for exascale computing 33

Acknowledgement • Soon Yung Jun 34

35

Thank you for your attention! 36

Back up

KISTI Supercomputing center • KISTI-5 Supercomputer – Processing: 25. 7 PF • Heterogeneous: 25. 3 PF CS 400 w/KNL • CPU-only: 0. 4 PF CS 500 w/SKL KISTI-5 – Storage • 21 PB SPS • 10 PB Archive – Launched in November 2018 38

39 T. Hong

40 T. Hong

41 T. Hong

42 T. Hong

QGSP BIC EMY Iodine Leipzig Iridium

Thank you! 44