SCAI Super Computing Application Innovation CINECA Sanzio Bassini

SCAI Super. Computing Application & Innovation CINECA Sanzio Bassini – May 2018

HPC infrastructure design point 15 Peta bytes 6600 Server nodes OPA GSS Marconi BDW - OPA Marconi KNL - OPA Login GSS D. A. V. I. D. E. (PRACE PCP) AI + NN Marconi SKL - OPA Gateway Key to deliver value ETH-core + Mellanox Gateway Vi. Z Internet HBP PRACE / EUDAT / EOSC INFN-CNAF Academia and national Institute ETH-25 Gbit tape Fibre SW servers 50 PByte Max capacity May, 2018 Mellanox FDR NFS Servers FEC Servers 5100 TMS Cloud (Big. Data & AI) 800 Server nodes Cineca - Super. Computing Applications & Innovations 2

12 Top 500 (November 2017) giu-12 nov-12 giu-13 nov-13 giu-14 nov-14 giu-15 nov-15 giu-16 14 giu-17 nov-17 95 82 70 54 45 36 31 nov-16 14 nov-11 23 giu-11 17 nov-10 15 giu-10 12 9 7 IBM SP 6 IBM PLX IBM BG / IBM BG / IBM BG / Marconi 5376 GPU Q Q Q Q Q A 2 A 2+A 3 May, 2018 Cineca - Super. Computing Applications & Innovations 3

Marconi configuration Cloud / Data Proc. Scale Out 2300 Lenovo Stark servers > 7 PFlops Intel Sky. Lake 24 cores @ 2. 1 GHz. 196 GByte x node 3600 Intel/ lenovo servers > 11 PFlops Intel PHI code name Knight Landing 68 cores @ 1. 4 GHz. single socket node: 96 GByte DDR 4 + 16 GByte MCDRAM 792 Lenovo Ne. Xt. Scale servers Intel E 5 -2697 v 4 Broadwell - 216 nodi eth x cloud HT INFN - 216 nodi eth x cloud HPC/DP - 360 nodi QDR x Tier 1 – HPC 100 Lenovo Ne. Xtscale servers Intel E 5 -2630 v 3 Haswell QDR + Nvidia K 80 720 Lenovo Ne. Xt. Scale servers Intel E 5 -2697 v 4 Broadwell 18 cores @ 2. 3 GHz. 128 GByte x node Lenovo GSS + SFA 12 K + IBM Flash >30 PByte May, 2018 Cineca - Super. Computing Applications & Innovations 4

D. A. V. I. D. E. Logical Name D. A. V. I. D. E. (August 2017) Model E 4 Cluster Open Rack Architecture Open. Power NVIDIA NVLink Processor Open. Power 8 NVIDIA Tesla P 100 SXM 2 # of core - # of node 45 x (2 Power 8 + 4 Tesla P 100) # of rack - RAM per node - Interconnection Mellanox EDR Operating System GNU/Linux Total Power - Peak Performance ~ 1 Pflops May, 2018 Cineca - Super. Computing Applications & Innovations 5

Italian & European HUB The new data center of intergovernative organization for medium range weather forecast ECMWF will be located in Bologna big data technopole. Protezione Civile / SMR Regione Emilia Romagna Operation numerical weather forecast FENIX: Digital infrastructure federation between CINECA (Italy), Barcelona Super. Computing Centre (BSC) Spain, Commissariat à l'énergie atomique (CEA) France, ETH Zurich's Swiss National Supercomputing Centre (CSCS) Switzerland, Jülich Supercomputing Centre (JSC), Germany. Ma. X European Center of Excellence for supports developers and end users of advanced applications in the field of materials, and works at the frontiers of the current and future High Performance Computing (HPC) and big data CINECA & Eni partnership for development of new applications for seismic analysis and reservoirs simulation and technology watch for emerging domain as AI and Quantum computing. Eni is the second largest Oil & Gas Company in Europe. CINECA & Unipol. SAI Group partnership for R&D based on big data analytics and AI in the domains of risk management, asset liability, fraud. Unipo. SAI Group is the third largest insurance company in Europe. Drug design and open lab between CINECA, Alfawasserman, Dompè Pharmaceutical, Chiesi pharmaceutical. May, 2018 Cineca - Super. Computing Applications & Innovations 6

European funded R&D Digital infrastructure HPC infrastructure PPIHBP Center of Excellence Material Science Environment Fabric of the future Industry 4. 0 Life Science Multimedia / Cultural Heritage Energy efficiency Low Power Microprocessor New AIDA EPEEC May, 2018 Cineca - Super. Computing Applications & Innovations 7

Assetto gerarchico D’Antonio D’Onorio Guida Morelli Mucci Muscianisci Rorro Testi Bordogna May, 2018 Caroli Montanari Trotta Pedrazzi Scarnò Sforzini Arlandini Zannoni Sviluppo Marruzzo Orlandini Manzitto Campagna Latini Ficarelli Affinito Chillemi Amati Castrignanò Cestari Cermonesi Ippolito Padrin Spallanzani Affinito Bna Calori Emerson Guarrasi Ruggiero Salvadore Sanna Bonfà Arlandini Pascolo Ponzini Spisso (Zannoni) Bueno Giuliani Grottesi Lanucara Falciano Marani Marras Rosati Calonaci Cavazzoni Cacciari Carpenè Turra (Fiameni) Servizi Fiameni Scipione Baccarelli Besker Caraceni Liguori Innovazione Direzione Gori Delli Ponti Monfardini Truini Guidazzoli Galetti Produzione (Rossi) Alberigo Rossi Ricerca Perticoni Piacentini Imboden Negri Giovannini Comparato Fatigati Tagliaventi Ialongo Tacconi Ramieri Ferraro Invernizzi Quadrani Dagna Zanella Ghetti Cineca - Super. Computing Applications & Innovations 8

Assetto funzionale Direzione Calcolo Istituzionale / supporto alla ricerca (Ambito prevalente) Eni / Innovazione Progetti Finanziati (Ambito prevalente) Eni project Zannoni / management Sviluppo SW Gori e Interfacce Sviluppo Cavazzoni Servizi Fiameni May, 2018 Chillemi User support di secondo livello Turra Big data Analitycs MW e Data Management, Architettura Data storage, Open Science Cloud Affinito Progetti EU Arlandini di HPC Calcolo tecnico, SME Scipione Metadati / Guidazzoli Annotation Visit Rossi Galetti Definizione ambiente di calcolo e monitoraggio / User supprt di primo livello / Gestione sistemi / Alberigo gestione / gestione cloud SW di produzione Eni Calonaci Produzione Attività Ambito Gestione call ISCRA e PRACE e cost statement progetti UE / Gestione Contratti Cineca - Super. Computing Applications & Innovations 9

Attività trasversali • Presidio board internazionali: • PRACE. Bo. D: Debora Testi, Council: Gabriella Scipione. • ETP 4 HPC: Carlo Cavazzoni. BDVA: Roberta Turra, EUDAT: Giuseppe Fiameni, CECAM: Fabio Affinito • Partecipazione al board di SIMAI: Alessandro Marani • Open LABS/HPC thematic hubs: • Energy Efficiency, Unibo (Luca Benini): Carlo Cavazzoni • Big Data / AI, Unimore (Sonia Bergamaschi, Rita Cucchiara): Giuseppe Fiameni • Material Science, Ma. X/Sissa/CNR/QEF (Elisa Molinari, Stefano Baroni): Pietro Bonfà • Bioinformatica, Elixir/CNR (Graziano Pesole): Giovanni Chillemi, Tiziana Castrignanò • CECAM UNIBO – Cineca (Andrea Cavalli): Fabio Affinito • FENIX: CINECA, CSCS, JUELICH: Giuseppe Fiameni • LENS e HBP Italia, (Francesco Pavone): Roberto Mucci • Collaborazioni • Intel IPCC: Fabio Affinito • Open Power Fundation: Carlo Cavazzoni • • • Benchmarking / Validation: Giorgio Amati Review tecnica progetti PRACE, ISCRA, LISA: Massimiliano Guarrasi Scuole / Corsi: Elda Rossi May, 2018 Cineca - Super. Computing Applications & Innovations 10

Access and development model • Open access peer review: PRACE, ISCRA, LISA • European flagship and RI: Hpc-EUROPA, Human Brain Project, EPOS, Elixir • Partnership: INFN, OGS, IIT, SISSA, INAF, ICTP, Polimi, Bicocca, MI Statale, Unibo, … • Quality of service: EUROfusion, ARPA-SMR / Civic protection • Innovation for Industries and SME – Open innovation: Fortissimo, PRACE SHAPE, Lisa for innovation – Framework agreement: Eni, Unipol • Special Project: Quantum Computing – Access to D-Wave Qbits system May, 2018 Cineca - Super. Computing Applications & Innovations 11

Access Panel 7 Membri Per discipline Assegnazione delle risorse di supercalcolo ISCRA • Scienze della materia 1) Mauro Ferrario Università di Modena 2) Claudio Zannoni Università di Bologna • Scienze della terra 3) Antonello Provenzale CNR Istituto di Geoscienze e Georisorse - Torino • Scienze della vita 4) Paolo Ruggerone Università di Cagliari • Fisica delle alte energie 5) Raffaele Tripiccione Università di Ferrara • Scienze dell’Universo 6) Giovanni Peres Università di Palermo • Ingegneria e fluidodinamica 7) Alfredo Soldati Università di Udine May, 2018 Cineca - Super. Computing Applications & Innovations 12

Training (2017) May, 2018 Cineca - Super. Computing Applications & Innovations 13

Production May, 2018 Cineca - Super. Computing Applications & Innovations 14

Usage by domain May, 2018 Cineca - Super. Computing Applications & Innovations 15

Users distribution May, 2018 Cineca - Super. Computing Applications & Innovations 16

Help desk May, 2018 Cineca - Super. Computing Applications & Innovations 17

INFN Access Accordo Quadro 2012 • 100 Milioni di core h di Fermi • Accordo Attuativo: Cofinanziamento di Galileo 500 K Euro corrispondenti a circa 10 milioni di core-h / anno Accordo Quadro 2015 • 1° Accordo attuativo: 6% delle risorse Tier-0, ha validità da inizio 2015 a fine 2018; 18 M core-h su A 1; 120 M core-h su A 2 • 2° Accordo attuativo: Cofinanziamento A 3+A 1, ha validità da inizio 2018 a fine 2020; Contributo: 1. 5 M euro - Nodi skl: 420; Contributo: 500 K Euro - Nodi bdw: 216 MARCONI 1 2014 2015 2016 2017 2018 May, 2018 MARCONI 2 MARCONI 3 GALILEO FERMI 132. 500. 000 13. 000 225. 000 6. 500. 000 15. 000 50. 000 18. 000 120. 000 15. 000 9. 000 120. 000 164. 000 Cineca - Super. Computing Applications & Innovations 18

Euro. HPC declaration Declaration signed in Rome 23/03/2017 by: France Netherlands Germany Italy Portugal Luxembourg Spain 8 more countries signed the Declaration: May, 2018 Belgium Slovenia Greece Croatia Cineca - Super. Computing Applications & Innovations Bulgaria Czech Republic Switzerland Cyprus 19

Euro. HPC Mission and objectives ■ to provide scientists, industry and the public sector from the Union with latest HPC and Data Infrastructure and support the development of its technologies and its applications across a wide range of fields. ■ to provide a framework for acquisition of an integrated world-class pre- exascale supercomputing and data infrastructure in the Union; ■ to provide Union level coordination and adequate financial resources to support the development and acquisition of such infrastructure, which will be accessible to users from the public and private sector primarily for research and innovation purposes; Present EU Financial Framework 2 Exascale machines 2 Pre-exascale machines 2018 May, 2018 2019 2020 2021 Next EU Financial Framework 2022 2023 2024 Cineca - Super. Computing Applications & Innovations 2025 2026 20

Euro. HPC Mission and objectives 2 Pre-exascale 2018 2019 2020 2021 Infrastructure Acquisition Operating machines 2022 2023 2024 2025 2026 ■ Pillar 1: ■ Acquisition of infrastructure: HPC ecosystem è 2 pre-exascale machines è >= 2 peta-scale machines ■ Research & Innovation Applications & Skills May, 2018 Joint Undertakings Installation, deployment and operation via hosting entities + access to users ■ Pillar 2: ■ European exascale technologies and systems (incl. low power processor) ■ Excellence in HPC applications; Co. E; competence centres for industry (incl. SME); Training and Outreach Cineca - Super. Computing Applications & Innovations 21

Activites and Funding JU Admin/Running costs May, 2018 Total (M€) 270 290 560 Pillar 2 Research & Innovation Applications & Skills PS Pillar 1 Infrastructure Acquisition Operating machines EC 206 186 392 10 10 20 Cineca - Super. Computing Applications & Innovations Private Mem. 422 22

Bologna Big Data Science Park Protezione civile and regional agency for development and innovation CINECA & INFN Exascale Supercomputing center ECMWF Data Center BIG DATA FOUNDATION Agenzia Nazionale Meteo «Ballette innovation and creativity center» University centers IOR biobank May, 2018 Conference and Education Center Enea center Competence center Industry 4. 0 Cineca - Super. Computing Applications & Innovations 23

HPC and Verticals Value delivered to users codesign SERVICES APPLICATIONS DATA PEOPLE VALUE COMPUTE HW infrastructure (clusters, storage, network, devices) Application Domains May, 2018 procurement HW infrastructure (clusters, storage, network, devices) procurement codesign Solutions Resources (HPC, HTC, Cloud, Viz, etc. ) Support, Consultancy, Training Connectors to other infrastructures High Throughput 3 D Viz AI Accelerated computing Big DATA VALUE Applications integration (Meteo, Astro, Materials, Visit, Repo, Eng. , Analytic, etc…) Cineca - Super. Computing Applications & Innovations Solutions 24

Towards Exascale Cavazzoni May, 2018 Cineca - Super. Computing Applications & Innovations 25

Energy constrains Cavazzoni May, 2018 Cineca - Super. Computing Applications & Innovations 26

Cineca Roadmap 5 x 5 x 5 x 20 x 100 TF 1 MW 2 PF 1 MW 11 PF+ 9 PF 3. 5 MW 10 x (in total) 50 PF+ 10 PF ~4 MW >250 PF+ >20 PF ~8 MW 2 x (latency cores) 1 x (latency cores) Paradigm change Procurement in progress Pre-exascale - Project - 2009 2012 2016 2019 2021/2022 IBM SP 6 Power 6 Fermi IBM BGQ Power. A 2 Marconi Lenovo Xeon+KNL Marconi PPI 4 HPC ICEI - PPIHBP Euro. HPC May, 2018 Cineca - Super. Computing Applications & Innovations 27