e VLBI Research in the European VLBI Network

e. VLBI Research in the European VLBI Network Steve Parsley Joint Institute for VLBI in Europe Dwingeloo, Netherlands parsley@jiv. nl AMPATH, FIU, Miami, 31 January 2003

Outline • • • European VLBI Network (EVN) European Research Networks Fibre Connection to Dwingeloo i. Grid Demonstration EVN-NREN: Proof of Concept Project AMPATH, FIU, Miami, 31 January 2003

AMPATH, FIU, Miami, 31 January 2003

The European VLBI Network

EVN Telescopes

Correlator AMPATH, FIU, Miami, 31 January 2003

AMPATH, FIU, Miami, 31 January 2003

European Research Networks • National • NREN = National Research and Education Network • UKERNA, SURFnet, GARR, DFN, PSNC, NORDUnet …. . • International • GÉANT = Pan-European Research Network AMPATH, FIU, Miami, 31 January 2003

GÉANT: Access of NRENs to GÉANT NL DE BE 2. 5 HU IT FR 1. 2 G GR CZ AT G 62 UK CH SE 2 M 0 GEANT ES PL 34 M HR RO EVN telescope LV BG 45 M CY LT PT 310 M 55 SI M IE 1 LU IL SK EE

GÉANT: The connectivity at 10 Gbps 2. 5 Gbps EE LV 34 155 UK 155 LU NL 155 BE FR SE IE LT 45 PL SE - Po. P for Nordunet 155 DE 0 ES PT CZ SK 622 CH IT AT HU 622 34 622 AMPATH, FIU, Miami, 31 January 2003 GR HR BG 34 45 CY 155 SI IL 34 RO

20 km of new fibre 180 km to SARA via installed link 3 colours @1 Gbps from Dwingeloo to Amsterdam Internet Exchange Dwingeloo

Network SARA – ASTRON / JIVE SARA GX SNET-LAN Access to Nether. Light Gb. E ODF ASTRON / JIVE ISDN Cisco 1721 Cisco 2620 RS 232 ODF Gb. E Project e. VLBI Gb. E Access to SURFnet 5 (ar 5. amsterdam 1) Cisco 15252 DWDM Cisco 15216 EDFA (2 x) 4912 G Cisco 15252 DWDM Gb. E ASTRON router Amsterdam 1 AMPATH, FIU, Miami, 31 January 2003 Emmeloord 1 Dwingeloo 1

i Grid 2 oo 2 www. startap. net/igrid 2002 i. Grid Demonstration www. igrid 2002. org (COMING SOON) The International Virtual Laboratory 24 -26 September 2002 Amsterdam Science and Technology Centre (WTCW) The Netherlands Call for Applications with Insatiable Bandwidth Appetites! “We hereby challenge the international research community to demonstrate applications that benefit from huge amounts of bandwidth!” AMPATH, FIU, Miami, 31 January 2003

vlb Westerbork Jodrell Bank Future fibre Link (now it’s a truck) GEthernet over SURFnet Lambda SARA Shared IP over GÉANT World of VLBI Technique & Operations Science Other VLBI collaborators About VLBI AMPATH, FIU, Miami, 31 January 2003 Dwingeloo Live telescope pictures Image JIVE Correlator Fringe Display

vlbi. Grid Objectives • Trial use of UDP for VLBI data transfer across production research networks: • Data rates • Packet loss • Packet reordering • Exercise new link to JIVE • Demonstrate e. VLBI fringes • Publicise VLBI to network researchers AMPATH, FIU, Miami, 31 January 2003

i. Grid Network Routing NNW MAN i. Grid 2002 Amsterdam Server Super. JANET 4 Core Network Server SURFnet Core Manchester 1 Gbit Eth. 2. 5 Gbit Pos 10 Gbit AMPATH, FIU, Miami, 31 January 2003 DWDM link Geant Core PCEVN

Data Transfer Process: TCP Control Raid 0 Disc Ring. Buffer UDP Data Web Interface Raid 0 Disc

Receiving Packets Incoming packet ids 1 2 3 4 6 • Packets have Header with packet id + Data • Put header directly into control area • Put data directly into ring buffer • No extra copy • Assume UDP packets arrive in order none lost • Deposit data into next consecutive slot 1 2 3 Time 4 Move 6 Continue with 7 • Inspect Header • Move data onward to correct location • Record: • • The received inter-packet spacing 1 -way delay

i. Grid Results • UDP across GÉANT (server-server) : • ~500 Mbit/s • <6 E-5 lost packets • <1 E-5 reordered packets (all exchanges) • FTP to Dwingeloo: • ~40 Mbit/s • Normal fringes detected • PR: • Great success, new link used three days after first light • Much interest in VLBI data rates and volumes AMPATH, FIU, Miami, 31 January 2003

i. Grid Lessons Learned • 500 Mbit/s VLBI data transfer on the production network using a simple UDP based protocol is feasible. • Acceptable packet loss is achievable. • Packets may be re-ordered. A simple data moving strategy corrects for this. • Control/monitoring data should be on a separate, reliable channel. • End hosts must have sufficient power in both compute cycles and input/out capability. An initial specification would include: • 64 bit 66 MHz PCI bus • 2 GHz processor • Gigabit Ethernet interface: • Intel Pro 1000 • Sys. Konnect • Fast disks with high speed buses • Maxtor D 740 EIDE 133 AMPATH, FIU, Miami, 31 January 2003

Proof of Concept Project (Po. C) • Target: • Up to 5 radio telescope sites (not incl NL) connected in real-time to JIVE correlator • Duration: • Up to end of GÉANT contract (Nov 2004) • Support: • • • Best effort IP service transiting NRENs and GÉANT No significant upgrades to GÉANT (initially) Support 512 M and 1 G real time modes of operation Use existing NREN access ports Limited resilience AMPATH, FIU, Miami, 31 January 2003

Proof of Concept Participants • • • • DANTE Pan-European Network GARR Italian NREN UKERNA UK NREN PSNC Polish NREN DFN German NREN SURFnet Dutch NREN KTHNOC/NORDUnet Nordic NREN Manchester University Network application software JIVE EVN Correlator Westerbork telescope Netherlands Onsala Space Observatory Sweden MPIf. RGermany Jodrell Bank UK TCf. A Poland CNR IRA Italy AMPATH, FIU, Miami, 31 January 2003

Success Criteria (for Po. C) • NREN/GÉANT point of view: • Connect at least three sites to JIVE • Observe significant usage (time and BW) • EVN point of view: • Same-day imaging of 12 hour, 1 Gb/s observation • Real time network verification of transient phenomenon AMPATH, FIU, Miami, 31 January 2003

Traffic Engineering on GÉANT. . . AMPATH, FIU, Miami, 31 January 2003

GÉANT Nodes Affected by EVN Traffic SE NORDUnet JANET UK PL SURFnet CZ PSNC BE NL DFN DE 2 DE 1 FR 2. 5 G 10 G CH AT IT GARR

Failure Scenarios #1 SE NORDUnet JANET UK PL SURFnet CZ PSNC BE NL DFN DE 2 DE 1 FR 2. 5 G 10 G CH AT IT GARR

Failure Scenarios #2 SE NORDUnet JANET UK PL SURFnet CZ PSNC BE NL DFN DE 2 DE 1 FR 2. 5 G 10 G CH AT IT GARR

Potential Upgrades to GÉANT SE NORDUnet JANET UK PL SURFnet CZ PSNC BE NL DFN DE 2 DE 1 FR 2. 5 G 10 G CH AT IT GARR

Failure Scenarios #3 SE NORDUnet JANET UK PL SURFnet CZ PSNC BE NL DFN DE 2 DE 1 FR 2. 5 G 10 G CH AT IT GARR

Failure Scenarios #4 SE NORDUnet JANET UK PL SURFnet CZ PSNC BE NL DFN DE 2 DE 1 FR 2. 5 G 10 G CH AT IT GARR

Some Technical Issues • Packet re-ordering due to architecture of Juniper M 160 routers in GÉANT backbone. VLBI UDP application must fix this. • VLBI data rates are restricted to “octaves” • : - 256 M, 512 M, 1 G • “ 1 Gbps” e. VLBI flows are >1 Gbps • two GE channels will be needed to support "1 Gbps" from each telescope. AMPATH, FIU, Miami, 31 January 2003

NREN 1 GÉANT NREN 2 NREN 3 IP 1 IP 2 IP 3 IP 4 SURFnet JIVE AMPATH, FIU, Miami, 31 January 2003

Po. C Design • Separate, GE point-to-point connection for each telescope • GE link on /30 subnet between VLAN interface on the SURFnet access router and the GE NIC at Dwingeloo. • Add second GE NIC for “ 1 Gb/s” each carrying 512 Mb/s GE AMPATH, FIU, Miami, 31 January 2003 NREN GÉANT SURFnet JIVE Corr. GE

Local Loops - Status • • • Italy: • GARR : Backbone capacity - 2. 5 Gbps. Poland: • • Medicina radio 30 km from Bologna Good POP) PSNC: 10 GE linktelescope: from Torun to Poznan (location. POP. of GÉANT Germany: connectivity fibre) expected by end of 2003. (3 -4 km dig ready in June(dark 2003. to radio • DFN: Notelescope. ) backbone problems expected. Bonn ring on-line April’ 03. UK: • Torun radio telescope: 15 km, tobut telescope in place. 100 m connection to DFN POPTorun needed not planned. • 5 -7 km UKERNA: no problems to support the e. VLBI traffic on the Sweden: and terminal equipment within 3 months. backbone. 2. 5 GFibre backup access port GÉANT • • Super. JANET Effelsberg Radio Telescope: to telescope in to place NORDUnet: two, redundant 2. 5 Gbit/s links to GEANT inbar 5 km Theplanned. Netherlands: copper section. Stockholm available. Bank: significant problem with local loop be to JB (BT to • Jodrell SURFnet: 2. 5 Gbit/s border connection will soon upgraded • SUNET: 10 Gbit/s backbone with. Options 2 X 2. 5 Gbit/s. SRP links between quoted £ 1 M for the first year) being explored include 10 Gbit/s “piggy-back” on e. MERLIN, 30 Gbit/s infrastructure ~June 2003 Chalmer University and Stockholm. • JIVE: 190 km dark fibre link from JIVE to Amsterdam (SARA) in • Onsala place. Space Observatory: 40 km of fibre required. Installation costs of up to € 150 k have been quoted with € 20 k+ annual • Westerbork: 24 fibre link directly to Dwingeloo by June‘ 03 charges. AMPATH, FIU, Miami, 31 January 2003

Project Timeline AMPATH, FIU, Miami, 31 January 2003

NREN 1 GÉANT NREN 2 NREN 3 IP 1 IP 2 IP 3 IP 4 SURFnet JIVE AMPATH, FIU, Miami, 31 January 2003

UKLight Proposal • Initiative to attract funding to create: • UK point-of-access (Po. A) to the international test-bed • Intra UK optical test-bed • Draft projects: • • Development of a common optical control plane Secure Optical Community Services A project to use a dark fibre infrastructure An applications demonstration based upon very long baseline interferometry (VLBI) for Radio Astronomy AMPATH, FIU, Miami, 31 January 2003

Dedicated Lambda VLBI Network AMPATH, FIU, Miami, 31 January 2003

2 nd e. VLBI Workshop 15 -16 May 2003, Dwingeloo, Netherlands • • First Announcement 10 th Jan ‘ 03 Second Announcement 1 st March ‘ 03 Abstract Deadline 19 th March ‘ 03 Final Announcement 3 rd April ‘ 03 Registration Deadline 20 th April ‘ 03 Hotel Deadline 15 th April ‘ 03 Workshop 15 -16 th May ’ 03 http: //www. jive. nl/jive/evlbi_ws/meeting AMPATH, FIU, Miami, 31 January 2003