Super Jumbo Frames 16 64 k path MTU

Super Jumbo Frames 16 -64 k path MTU performance analysis BCNET-Netera Advanced Networks Conference 2006

What are Super Jumbo Frames ? • SJF Payload Data > 9000 bytes • IPv 4 Header 16 bit limit • 65, 535 minus header • IPv 6 Extension 32 bit limit • 4, 294, 967, 295 minus header RRX Network

Why do SJF? • Moores Law starting to plateau • HPC community going to larger interconnect solution frames • Main backbones scaling up to 100 G RRX Network

SJF Hardware? • Enterasys ER 16 Series Routers • MTU = 65442 bytes • Spirent Communications –AX/4000 • MTU = 64724 bytes RRX Network

Inter Frame Gap (IFG) • IFG - time between end of one packet and beginning of next • IFG Inflation - when bigger payload causes IFG increase • Minimum IFG implies max network performance RRX Network

Setup RRX Network

Results CA*net SC’ 05 - 25 ms IFG Abilene throughput RRX Network

IFG Inflation CA*net 0. 095 Abilene 0. 364 RRX Network

Simulation 640000 2. 56 M 6. 4 M RRX Network

SJF Memory Issues • Enterasys ER 16 uses direct memory access (DMA) to a common main memory system • Ethernet integrated circuits generally have internal 64 or 128 kilobyte buffers RRX Network

SJF Contributors • E. Paul Love (Xnet co-chair) • Paul Daspit (Xnet co-chair) • Greg Goddard (Spirent) • Bob Anderson (Spirent) • Randy Ottman (Enterasys ) • Arnold Sodder (Enterasys ) • Issa Eid (Enterasys ) • Gary Finley (Netera) • Loki Jorgenson (Apparent Networks) RRX Network • Martin Siegert (SFU-West. GRID) • Lixin Liu (SFU) • Peter Van Epp (SFU) • Bryan Caron (Uof. Alberta) • Brian Corrie (SFU-IRMACS) • John Moore (NCSU) • Kevin Walsh (SDSC) • Paul Schopis (ITEC-Ohio)

End of Presentation RRX Network