Deploying 400 Gbits on the Janet network Rob

Deploying 400 Gbit/s on the Janet network Rob Evans, Chief Network Architect Neil Shewry, Head of Network Delivery

The Janet network – before. 2 Deploying 400 Gbit/s on the Janet network

The Janet network – the plan. 3 Deploying 400 Gbit/s on the Janet network

Why did we want/need 400 G? • Ever-increasing capacity demands • Incoming IP traffic to Janet from GEANT, peering, and transit is > 600 Gbit/s • More efficient spectrum usage • Save on chassis space! • 4 x 100 G client plus 400 G line in single slot, saves 3 slots compared to OTR, saves 7 slots (half a chassis) compared to OCI+OCLD. • Flexgrid 9: 1 WSS are two slots, older fixed grid 9: 1 WSS occupied 3 4 Deploying 400 Gbit/s on the Janet network

Where should we deploy 400 G? • You might have thought where we wanted 600 G • Busiest parts of the network • Lots of disruption to migrate from fixed grid to flexgrid. • Some routes too long to run Wavelogic Ai cards at 400 G in 75 GHz. • Other Wavelogic cards are available… • Birmingham – Manchester – Leeds – Nottingham • Require 400 G capacity • Replace existing 100 G cards and re-deploy for South upgrade from 400 G to 600 G. 5

Will it work? • 100 G transmission uses DP-QPSK encoding • Two polarisations, quadrature phase shifting • Wavelogic Ai chipset adjusts encoding depending on OSNR • The better the OSNR, the more capacity • Proprietary modulation formats based on QPSK/8 QAM/16 QAM/32 QAM/64 QAM • Step 1: Know what we’ve got • Original OTDR results now six years old • Take backbone fibres out of service to re-measure 6 Deploying 400 Gbit/s on the Janet network

OSNR: Why and how? • Change from EDFA-based amplifiers to Raman • Strict loss requirements in the first 20 km of fibre • Raman amplifiers include OTDR to check fibre is in spec before enabling • Modelling on the planned spans • Good to go on Birmingham to Manchester and Nottingham to Leeds (& vice versa) • Manchester to Leeds: -0. 42 d. B • Leeds to Manchester: -0. 09 d. B • Anything below -0. 3 d. B is out of spec • Conservative stance is that we can only do 300 G between Leeds and Manchester 7 Deploying 400 Gbit/s on the Janet network

Flexgrid and colourless add/drop • Existing WSSs all 50 GHz fixed grid with wavelength-specific add/drop mux/demux (CMD 44) • 400 Gbit/s uses 56 Gbaud in 75 GHz • ‘Simple’ matter of replacing fixed-grid WSSs with flex-grid and introducing colourless add/drop. 8 Deploying 400 Gbit/s on the Janet network

Flexgrid conversion 9 Deploying 400 Gbit/s on the Janet network

Not quite colourless 10 Deploying 400 Gbit/s on the Janet network

If only it was that easy… • Network-wide software upgrade beforehand • Not just one, but two: 10. 2 -> 11. 2 -> 12. 1 • One. Control • Change in licensing approach for 12. 1 • License alarms after upgrade • Deploy external license server • Different power calculations between releases of software • Existing services entered a “waiting for power” state after upgrade 11 Deploying 400 Gbit/s on the Janet network

It was all worth it in the end… • Initially provisioned Manchester – Leeds with 3 x 100 G clients • Wavelength came up at 400 G • Able to add fourth service and run it at 400 G 12 Deploying 400 Gbit/s on the Janet network

How we delivered it

Conception 14

Conception 15

Conception § Initial modelling (based on ‘old’ fibre characteristics) 16

Conception § Initial modelling (based on ‘old’ fibre characteristics) 17

Conception § Initial modelling (based on ‘old’ fibre characteristics) 18

Business case § Initial designs § Indicative costs § Options appraisal 200 G -> 400 G -> 600 G 19

Business case § Initial designs § Indicative costs § Options appraisal 200 G -> 400 G -> 600 G § Approval to proceed in July 2017 20

Detailed planning § Fibre characterisation § Equipment upgrades § Optical modelling § 300 Gbit/s 21

Detailed planning § Fibre characterisation § Equipment upgrades § Optical modelling § 300 Gbit/s 22

Detailed planning § Fibre characterisation § Equipment upgrades § Optical modelling § 300 Gbit/s § Service impact 23

Detailed planning § Fibre characterisation § Equipment upgrades § Optical modelling § 300 Gbit/s § Service impact § Software upgrades 24

Orders placed & deployment begins 25

Orders placed & deployment begins § Hardware installation all over the place § Network management system upgrade § 2 steps § Network element software upgrades § Upgrade § Reboot 26

Problems! 27

Problems! § Card failures 28 https: //imgflip. com/memegenerator/127263078/Broken-Computer

Problems! § Scheduling 29

Problems! § Po. P access 30 https: //www. vecteezy. com/vector-art/160162

Problems! § Manchester 31

Problems! § Manchester 32

Problems! § Manchester § Lots of visits § Lots of cleaning 33 § Lots of splicing

Job nearly done 34

Job done § 400 Gbit/s wavelengths ready for service 35

Job really done May 2019! § Migration of traffic § 100 Gbit/s card redeployment § Return of surplus equipment § Inventory 36

Job really done May 2019! § Migration of traffic § 100 Gbit/s card redeployment § Return of surplus equipment § Inventory 37

Job really done May 2019! § Migration of traffic § 100 Gbit/s card redeployment § Return of surplus equipment § Inventory 38

Where next? • Migrating the rest of the network to flexgrid • In process at the moment • Deploying true colourless add/drop • Next up • 400 GE clients • A whole other talk… on QSFP 56 -DD, etc 39 Deploying 400 Gbit/s on the Janet network

Questions? 40 Deploying 400 Gbit/s on the Janet network