NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER Visapult and

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER Visapult and the SCXX Bandwidth Challenge Hat Trick Wes Bethel LBNL

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER The Punch Line • • SC 00 – 1. 5 Gbps peak, 660 Mbps sustained. SC 01 – 3. 3 Gbps sustained. SC 02 – 16. 8 Gbps sustained. SC 03 – we’re taking the year off. How did we do this? 2

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER Visapult Architecture 3

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER SC 00 Bandwidth Challenge – the year of TCP 4

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER SC 01 Bandwidth Challenge – the year of UDP • 5

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER SC 01 BWC – What’s this UDP stuff? • • Do away with “frame boundaries. ” Each packet is independent. Flow-rate regulation. Decouple simulation PE from Visapult PE (domain decomposition independence). 6

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER SC 02 Bandwidth Challenge Resource Map 7

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER SC 02 SCInet Weathermap 8

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER SC 02 SCInet 10 Gb Interface Traffic 9

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER 10

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER SC 02 BWC Visapult Improvements • Omniview capabilities • Overhauled custom UDP encoding & connection handshake – more flexibility and better performance. 11

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER 12

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER Lessons Learned • Rate regulated flows seriously kick booty. Want general purpose implementation. (Tsunami) • Corollary: TCP sucks. • Visapult is completely custom. Want better support to more generally deploy component-based RDV tools (Diva). • Application drivers particularly useful to push technology in many areas, not just networking. 13

NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER 14