CANARIE CAnet 4 Planning http www canarie ca

CANARIE CA*net 4 Planning http: //www. canarie. ca http: //www. canet 3. net Bill. St. Arnaud@canarie. ca Tel: +1. 613. 785. 0426

Customer Empowered Network City C City A Carrier Neutral IX Condo Wavelengths Carrier Neutral IX City B Condo Dark Fiber Condo Wavelengths

What is e. Science? · The ultimate goal of e-science is to allow students and eventually members of the general public to be full participants in basic research. · Using advanced high speed networks like CA*net 4 and novel new concepts in distributed peer to peer computing, called “Grids” many research experiments that used to require high end super computers can now use the computer capabilities of thousands of PCs located at our schools and in our homes. · High performance computers that are part of C 3. ca can be seamlessly integrated with e. Science distributed computers using CANARIE Wavelength Disk Drive over CA*net 4 · Allows researcher access to the significant computational capabilities of all these distributed computers at our schools and homes · With e-science it might be possible that the next big scientific discovery could be by a student at your local school.

ALTA Cosmic Ray e. Science · The earth is constantly bombarded by subatomic particles from space, with an energy spectrum that reaches far higher than any terrestrial accelerator could hope to probe. · At the highest energies such showers can be detected at the Earth’s surface over areas on the order of 100 square kilometers. · It is believe some of these cosmic rays were created at the creation of the universe · Will allow researchers to gainer a deeper understanding of deepest reaches of space and time

ALTA Cosmic Ray e. Science · The ALTA project is a collaborative scientific research project involving the University of Alberta Center for Subatomic Research and over 50 high schools across Canada in the area of cosmic ray detection. · Teachers and students actively contribute to the physics research while learning about an exciting area of modern science. · Distributed computing at schools will be required to analysize data from sensors in near real time · Program has now expanded into USA and soon countries around the world · · CHICOS (California HIgh school Cosmic ray Ob. Servatory), Caltech, UC/Irvine and Cal State/Northridge, California, USA. CROP (the Cosmic Ray Observatory Project), University of Nebraska, Lincoln, NE, USA. WALTA (WAshington Large area Time coincidence Array), University of Washington, Seattle, WA, USA. SALTA Roaring Fork Valley area of Colorado

Neptune – Undersea Grid

Wavelength Disk Drives · CA*net 4 will be “nation wide” virtual disk drive for grid applications · Big challenges with grids or distributed computers is performance of sending data over the Internet · TCP performance problems · Congestion · Rather than networks being used for “communications” they will be a temporary storage device · Ideal for “processor stealing” transaction intensive applications where you don’t know where the next available processor is located · CFD · Visualization

Wavelength Disk Drives St. John’s Regina Calgary CA*net 3/4 Winnipeg Charlottetown Montreal Fredericton Vancouver WDD Node Halifax Toronto Ottawa Computer data continuously circulates around the WDD

WDD Architecture WDD Partners: CANARIE, Can-Sol, Viagenie CRC, Carleton U, MACI C 3. Ca, Memorial, Dalhousie Ude. Montreal, Uo. Toronto, SFU, Uo. Alberta, BCnet 3. The SGI writes back the task onto the ring where it is received by Forest Fire Raster Engine and results displayed on X-Window terminal at CRC Uo. Alberta Ude. Montreal Dalhousie Uo. Toronto Memorial SFU CRC WDD Node Vancouver WDD Node Calgary Halifax WDD Ring on CA*net 3 Forest Fire Modeling Raster Engine 1. Forest Fire Modeling Raster Engine injects 64 K x 64 K raster computational tasks into WDD ring 2. Tasks circulate in WDD ring and first available SGI processor removes next task out of the ring and completes computation

Forest Fire Modeling e. Science · Emergency officials and civic defense officials need to model forest fires in real time · But each forest fire model may take hours to compute · By utilizing thousands of distributed computers at our schools and Wavelength Disk Drive on CA*net 4 network forest fire models in near real time · First prototype to be demonstrated on CA*net 3 in May using 256 SGI processors across the country on WDD

CA*net 4 Possible Architecture Layer 3 aggregation service Optional Service Available to any Giga. POP St. John’s Calgary Regina Winnipeg Large channel WDM system Charlottetown Vancouver Europe Montreal Customer controlled Seattle optical switches Fredericton Ottawa Chicago Toronto Halifax New York

STAR LIGHT Interconnection? · We see STAR LIGHT, CA*net 4, DTF and Vancouver Transit exchange facing same design issues · How do we signal interconnect wavelengths (SDH/SONET subchannels) between STAR LIGHT participants? · Like STAR TAP we will probably need a mix of Layer 1 -3 solutions · Layer 1 cross connect ATM plus · Layer 3 router and/or route server · Current ATM approaches · Full mesh ATM like current STAR TAP · Not possible with wavelengths or SDH/SONET channels · PVC created on demand · E. g Peer maker at MAEs

STAR LIGHT Options · Layer 0 - Patch panel or optical switch · Needs common wavelength and protocol · Not easily subject to change and will not allow multiple peers · Layer 1 - SDH/SONET cross connect switch · Issues related to how identify and address SDH/SONET channels · Layer 2 - GMPLS using IP and SONET/optical switch · Main thrust of industry –see Juniper/Nortel, Accelight, Cisco, NTT · Requires significant centralized management · Layer 2 -Map SDH/SONET channels to Gb. E channels & use Gb. E switch · Layer 3 - Each network terminates on its own router & routers meshed together · N squared meshing · Layer 3 - BIG ROUTER · Will it scale and needs central management and AS · Layer 4 – OBGP with CWDM with optical switch · Each CWDM wavelength mapped to SDH/SONET channel · Control of switch is by research networks

OBGP Status Report · OBGP first draft submitted to IETF · Prototype working at Carleton U · We want input on next steps for OBGP and see if it will fit within STAR LIGHT plans · Key features: · SDH/SONET & Optical cross connects controlled by attached networks · SDH/SONET & Optical cross connects identified by IP addresses & AS · RPSL with OON extensions is database used to query who is connected at switch and at what port · BGP OPEN message is used like Peer maker to request optical peering across the switch · BGP UPDATE message and community Tags ( and maybe GMPLS) will be used to setup multihop wavelengths

OBGP · Proposed new protocol to support control and management of wavelengths and optical switch ports · Control of optical routing and switches across an optical cloud is by the customer – not the carrier – true peer to peer optical networking · Use establishment of BGP neighbors or peers at network configuration stage for process to establish light path cross connects · Customers control of portions of OXC which becomes part of their AS · Optical cross connects look like BGP speaking peers – serves as a proxy for link connection, loopback address, etc · Traditional BGP gives no indication of route congestion or Qo. S, but with DWDM wave lengths edge router will have a simple Qo. S path of guaranteed bandwidth · Wavelengths will become new instrument for settlement and exchange eventually leading to futures market in wavelengths · May allow smaller ISPs and R&E networks to route around large ISPs that dominate the Internet by massive direct peerings with like minded networks

Wavelength Scenarios Workstation to Workstation Wavelength University to University Wavelength CWDM Giga. POP to Giga. POP Wavelength Regina BCnet Vancouver Campus OBGP switch Winnipeg RISQ Halifax Calgary Montreal Seattle St. John’s Toronto

Wavelength Setup AS 2 - AS 5 Peer AS 3 10 University 12 Regional Network 3 13 AS 1 2 15 4 AS 1 - AS 6 Peer AS 2 7 Dark Fiber 5 9 1 Regional Network AS 5 14 AS 4 8 6 ISP router Wavelength Object owned by primary customer Wavelength Subcontracted by primary customer to a third party AS 6 University

Wavelength Logical Mapping AS 2 - AS 5 Peer AS 3 10 University 12 Regional Network 3 13 AS 1 2 15 4 AS 1 - AS 6 Peer AS 2 7 Primary Route Backup Route 5 9 1 Regional Network AS 5 14 AS 4 8 AS 6 6 University ISP router

Resultant Network Topologies University BGP Peering on switches at the edge Packet Forwarding in the core 9 AS 1 15 Regional Network 8 AS 5 1 2 10 P 7 G OB 2 13 12 3 10 1 7 14 9 AS 2 8 AS 6 5 Regional Network Potential OBGP Peering ISP router University

OBGP Variations 1. OBGP Cut Thru · 2. OBGP Optical Peering · 3. External router controls one or more switch ports so that it can establish direct light path connections with other devices in support peering etc OBGP Optical Transit or Qo. S · 4. OBGP router controls the switch ports in order to establishes an optical cut through path in response to an external request from another router or to carry out local optimization in order to move high traffic flows to the OXC To support end to end setup and tear down of optical wavelengths in support of Qo. S applications or peer to peer network applications OBGP Large Scale · To prototype the technology and management issues of scaling large Internet networks where the network cloud is broken into customer empowered BGP regions and treated as independent customers

OBGP Optical Peering · Primary intent is to automate BGP peering process and patch panel process · Operator initiates process by click and point to potential peer · Original St. Arnaud concept · Uses only option field in OPEN messages · Requires initial BGP OPEN message for discovery of OBGP neighbors · Virtual BGP routers are established for every OXC and new peering relationships are established with new BGP OPEN message · Full routing tables are not required for each virtual router · No changes to UPDATE messages · No optical transit as all wavelengths are owned by peer · Uses ARP proxy for routers on different subnets · Wade Hong Objects concept · · · · Uses an external box (or process) to setup optical cross connects SSH is used to query source router of AS path to destination router Each optical cross connect is treated as an object with names given by AS path Recursive queries are made to objects to discover optical path, reserve and setup NEXT_HOP at source router is modified through SSH End result is a direct peer and intermediate ASs disappear Requires all devices to be on same subnet