AARNet 3 The Next Generation of AARNet 2004

AARNet 3 The Next Generation of AARNet © 2004 AARNet Pty Ltd

Background to AARNet • AARNet Pty Ltd (APL) is a not for profit company owned by 37 Australian Universities and the Commonwealth Scientific & Industrial Research Organisation (CSIRO) • Operates a national network providing commodity and research Internet access to members and clients • Clients include Defence Science and Technology Organisation (DSTO), National Library of Australia, Australian Institute of Marine Science • Current network deployed in 1997, based on ATM mesh between state and territory networks (RNO) • Also operates a STM-1 ring to the USA (Hawai‘i and Seattle) on Southern Cross, primarily for research but some commodity via Pacific Wave • Currently buys commodity access at each RNO from Optus or Telstra © 2004 AARNet Pty Ltd 2

Design Issues • Redundancy & Resilience • Support for IPv 4 and IPv 6 – unicast and multicast • Traffic Accounting and Monitoring • End to end performance measures • Support Qo. S (diffserv) • Support for large traffic flows, jumbo frames © 2004 AARNet Pty Ltd 3

Redundancy & Resilience • Dual points of presence (POP) in major capital cities • Diverse, dual unprotected national links – Will use MPLS Fast Reroute for protection – Provides ability to burst above capacity • Use single metro dark fibre pair to connect intra city POP sites • Creates rings between cities • Provides opportunity for members and customers to build diverse, redundant connections to AARNet © 2004 AARNet Pty Ltd 4

Australian Network © 2004 AARNet Pty Ltd 5

10 Gbps Backbone • Provided on the “Nextgen Networks” network • Two fibre pairs on each path – STM-64 service provided on first pair for inter capital trunks – Second pair may be lit with CWDM to allow Gigabit Ethernet drop off to regional members, other solutions to be considered – Member must provide tail to the regional network © 2004 AARNet Pty Ltd 6

Member Connections • Diverse connection to each POP – Two diverse, independent links, one to each POP • Dual connection connecting each POP – Two links over same infrastructure to single POP – AARNet trunks one link to the second POP though switches • AARNet provided diversity – Single link to one POP, AARNet provides LAN linking both AARNet POP sites and the member © 2004 AARNet Pty Ltd 7

Connections through the Giga. POPs © 2004 AARNet Pty Ltd 8

Trans Pacific Transmission • “SX Trans. PORT” - Dual STM-64 (10 Gbps) – Hawai‘i - Manoa and Seattle (Abilene, CA*net 4) – Los Angeles (Abilene, CENIC, CUDI) – Look to add Mauna Kea to Los Angeles path later • Dual STM-4 (622 Mbps) for commodity Internet – PAIX Palo Alto (Silicon Valley) – Los Angeles • Add drop offs to existing STM-1’s (155 Mbps) – University of South Pacific, Fiji – Possibly Auckland, New Zealand – Connects to 155 Mbps path to Tokyo from Hawai‘i © 2004 AARNet Pty Ltd 9

AARNet’s Pacific Ocean links © 2004 AARNet Pty Ltd 10

Equipment • Core Router – 40 Gbps capable – Redundant power but not CPU – Packet over SDH to STM-64 (roadmap to STM-256) – Gigabit and 10 Gigabit Ethernet • Core Switch – Pure L 2 switching – Fast, Gigabit and 10 Gigabit Ethernet only • Member Edge and POP based “Legacy” routers – 3 x Gigabit Ethernet (Member, POP “A”, POP “B”) – 1 x Fast Ethernet dedicated to flow accounting – Capability to handle legacy (slow) interfaces © 2004 AARNet Pty Ltd 11

Backbone Routers - Procket 8812 • • 22 RU (95. 3 x 44. 2 x 64. 8 cm) 12 Line Cards 48 Media Adapters (MA) Route Processor – Procket developed System Control Chip – 500 MHz IBM Power PC – 2 GB main memory – 512 MB Compact Flash (system program storage with redundant images) – 20 GB Hard Disk Drive (system log files) – 960 Gbps 1. 2 Bpps • • © 2004 AARNet Pty Ltd 12 1 Port STM-64 MA 1 Port 10 Gigabit Ethernet MA 10 Port Gigabit Ethernet MA 8 Port STM-1/STM-4 MA

Pro/8812 under test • Sitting in the Nextgen node room in Adelaide • Testing the STM-64 circuit between Adelaide and Perth • 32 Kbyte Packets ‘back-toback’ at 10 Gbps © 2004 AARNet Pty Ltd 13

Core Backbone Switches - Cisco 6509 • 20 RU (84. 4 X 43. 7 x 46. 0 cm) • 9 Slot Chassis • Supervisor 720 – 720 Gbps – 30 Mpps Centralized, up to 400 Mpps for CEF 720 interface modules equipped with d. CEF (DFC 3) or a. CEF daughter cards • 4 port 10 Gigabit Ethernet • 48 port 10/1000 UTP based Ethernet • 24 port SFP Gigabit Ethernet • Potential for service modules later © 2004 AARNet Pty Ltd 14

Edge Routers - Cisco 7304 • 4 -RU (10 cm) compact chassis • 4 -slot modular system • Network Equipment Building Standards (NEBS) Level 3 compliance • NPE-G 100 Processor – Three onboard Gigabit Ethernet ports – 1 GB of Synchronous Dynamic RAM (SDRAM) – 256 MB of removable Compact Flash memory – Better than 1 mpps processing performance • Redundant power supplies • Front-to-back airflow for optimal cooling © 2004 AARNet Pty Ltd 15

IPv 4 and IPv 6 • Native IPv 4 and IPv 6 (Dual Stack) network – Unicast and Multicast for both IPv 4 and IPv 6 – EFT IPv 6 Multicast (initially intra-domain only) – Line rate performance for IPv 4 and IPv 6 – Peering to both R&E and Commodity Internet w/ IPv 6 – Hexago IPv 6 Migration Broker to aid member and client IPv 6 deployment • DNS, AARNet Mirror and USENet News accessible over IPv 4 and IPv 6 • IPv 6 Performance Measurement • IPv 6 Flow Records? • Jumbo frames, 9000 byte © 2004 AARNet Pty Ltd 16

Some Issues • End to end performance measures – Desire to measure performance from member site – Provide connectivity reports on core services • Support Qo. S (diffserv) – Need to support Vo. IP and Video. IP traffic – Possibly introduce scavenger service • Support for large traffic flows, jumbo frames • Support for EVERYTHING with IPv 6? © 2004 AARNet Pty Ltd 17

Services • • DNS Cache and Secondary Servers Usenet News Hexago IPv 6 Migration Broker DDo. S Detection and Mitigation – Investigate appliances – Interest in automatic detection and filtering – Locate next to transit (and peering) links – IPv 6 Do. S/DDo. S? • AARNet Mirror • Vo. IP Gateways • NLANR and/or RIPE Test Traffic Measurement © 2004 AARNet Pty Ltd 18

Current Status (1) • National Transmission – Confirmation of POP sites – Testing STM-64 circuits – Build new Giga. POP sites – Obtain fibre between Giga. POPs and COs – Solution for Tasmania and Northern Territory • International Transmission – Planning progressing with US partner organisations on connecting “SX Trans. PORT” – STM-4 to Palo Alto should be enabled during February – Direct Asian links dependant on available funds and member demand © 2004 AARNet Pty Ltd 19

Current Status (2) • Commodity Internet Transit – Access Commodity Internet in Palo Alto • Connected to the PAIX fabric • Obtain transit from MCI/UUnet and NTT/Verio – Commodity IPv 6 Transit! • Peer with other organisations at PAIX – Add second commodity POP in Los Angeles • Need to determine – data centre location – backhaul from Morro Bay (San Luis Obispo) • Will use the same transit providers as at Palo Alto © 2004 AARNet Pty Ltd 20

Current Status (3) • Peering – Developing national and local (state) policies – A consideration for POP site location • Regional links – Investigate CWDM options – Possibly issue another RFP – Priorities are: • inland Sydney/Brisbane via the telescopes • coastal Sydney/Brisbane route • Sydney to Albury © 2004 AARNet Pty Ltd 21

IPv 6 Migration Broker (1) • What… – Hexago IPv 6 Migration Broker • http: //broker. aarnet. au/ – Tunnel Broker used by Free. Net 6 – User setup for 6 in 4 tunnels, via web form • allocations from AARNet’s 2001: 388: : /32 address space – Can be used just for end systems – But can also assign prefix for local LAN – No routing functionality, static routing only – Open access but targeted to “local”, Australian community, not just AARNet members & clients © 2004 AARNet Pty Ltd 22

IPv 6 Migration Broker (2) • Why? – Members & clients are not ready to fully deploy IPv 6 across their network but some interest within their organisation – Some common firewalls, eg PIX, don’t support IPv 6 • Tunnel allows traversal of firewalls • But doesn’t provide firewall function unless end point can do it © 2004 AARNet Pty Ltd 23

IPv 6 Migration Broker (3) • Experience… – Most configure account but don’t configure tunnel – Some setup tunnel but for whatever reason only use it for a short time… • Perhaps just looking at the Kame : -) • Maybe forgot to add to startup – Small number of users permanent fixture © 2004 AARNet Pty Ltd 24

Current IPv 6 Activity • Major IPv 6 Transit Links – APAN-JP – Abilene • Native IPv 6 connection – Australian National University, Canberra • Permanent IPv 6 Tunnels to: – 9 AARNet Members – 4 Commercial ISPs / Carriers – APNIC – New Zealand • Public 6 to 4 Relay • Migration Broker © 2004 AARNet Pty Ltd 25

IPv 6 Activities • AARNet’s IPv 6 Migration Broker – http: //broker. aarnet. au/ – Requires TSP software • IPv 6 Forum – Australian Chapter – Mike Biber, Chair AARNet IPv 6 Working Group – Promote awareness of IPv 6 in Australia • IPv 6 Tutorials in each State and Territory – John Barlow • IPv 6 Research, e. g. Monash University – http: //www. ctie. monash. edu. au/ipv 6/ • AARNet IPv 6 interface stsistics – http: //ipv 6. broadway. aarnet. au/mrtg/ipv 6/ • Seek out and deploy IPv 6 peering in Australia © 2004 AARNet Pty Ltd 26

Procket Networks and IPv 6 – Tony Li © 2004 AARNet Pty Ltd 27

www. aarnet. edu. au