eVLBI and EXPRe S Arpad Szomoru 4 th
e-VLBI and EXPRe. S Arpad Szomoru 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Outline • • Current status, recent results Network test e-VLBI and the EXPRe. S proposal Conclusions, future 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Why e-VLBI ? • • Reliability – real-time feedback to the telescopes Sensitivity – sustained high data rates possible Logistics – No media management Rapid science results: • Geodesy • Precision spacecraft navigation 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
e-VLBI Proof-of-Concept Project • • • • DANTE/GÉANT Pan-European Network SURFnet Dutch NREN GARR Italian NREN UKERNA UK NREN PSNC Polish NREN DFN German NREN KTHNOC/NORDUnet Nordic NREN Manchester University Network application software JIVE EVN Correlator Westerbork telescope Netherlands Onsala Space Observatory Sweden MRO Finland MPIf. R Germany Jodrell Bank UK TCf. A Poland CNR IRA Italy 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
GÉANT: Access of NRENs to GÉANT NL DE BE 2. 5 HU IT FR 1. 2 G GR CZ AT G 62 UK CH SE 2 M 0 GEANT ES PL 34 M HR RO EVN telescope LV 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE BG 45 M CY LT PT 310 M 55 SI M IE 1 LU IL SK EE
POC targets: • for the EVN and JIVE • Standards: - Protocols, parameter tuning, procedures at telescope and correlator. • New Capabilities: - Higher data rates, improved reliability, quicker response. • for GÉANT and the NRENs • To see significant network usage with multiple Gbit streams converging on JIVE. • Minimum is three telescopes (not including Westerbork) • Must be seen to enable new science and not just solve an existing data-transport problem • Real-time operation is seen as the ultimate aim, buffered operation accepted as a development stage. 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Current status: • JIVE: 6 lambdas via Netherlight, each capable of 1 Gbps GE lines to correlator • 1 Gbps connections to Westerbork, Torun, Onsala, Jodrell LX Optics converters Cisco ONS 15252 • 155 Mbps to Arecibo, 128 to Cambridge Optical Splitters Fibre pair • Medicina being connected at 1 from. Gbps Amsterdam now, Metsahovi 2006 Correlator Interface 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Network testing • Use existing protocols on currently available hardware • TCP maximal reliability • Not really required • Sensitive to congestion • Lot of fine-tuning necessary • And possible • UDP connectionless • Unaccountable • Tailor made protocols? • Lambda switching • Internet weather • Hard to quantify • Hard to pinpoint bottlenecks 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
e-VLBI transfer tests • February 2005: network transfer test (BWCTL) employing various network monitoring tools involving Jb, Cm, On, Tr, Bologna and JIVE 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
e-VLBI transfer tests 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Test results Memory-memory Disk 2 net-net 2 disk In 2 netnet 2 out UDP TCP TCP Bench via patch 930 250 Idem, jumbo frames 960 Bench via Amsterdam 500 249 Bologna-JIVE 670 128 Jodrell-JIVE 50 70 Arecibo-JIVE 800 Onsala-JIVE-Haystack 456 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE 256 64 256 378 307 88 64 260 128 177 612 512 256 680 Idem, jumbo frames Torun-JIVE 544 341 867 256 360 Idem, jumbo frames Westerbork-JIVE TCP 71 256
POC results • Demonstration of feasibility • Identification of problems • Has led to closer ties with networking community and generated political interest • Has laid the foundation for the next step forward (EXPRe. S) 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
The Future… e-VLBI & EXPRe. S! • I 3 proposal to the EC (Communication & Network Development Call) • Ranked first out of 43 proposals; nearly fully funded to an amount of 3. 9 MEuro. EXPRe. S = EXpress Production Real-time e-VLBI Service 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
EXPRe. S aims • Upgrade EVN to e-EVN • Last mile problem at many telescopes • Including connections on a global scale • 16 Gb/s lines into Dwingeloo to JIVE correlator • Software in field and correlator to become real-time • And look beyond 1 Gb/s • More capacity on digital sampling, more bandwidth • As being implemented for e-MERLIN in UK • Hardware (PC-based) and protocols for transport • Correlator with more capacity: distributed correlation 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Expanding the e-VLBI Network 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
New Antennas 40 m antenna at Yebes, Spain Radio and mm frequencies Sardinia Radio Telescope 64 m; radio to millimeter 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Current “e-VLBI” Telescope Status Comments Cost (Meuro) JIVE 6 x 1 Gbps 16 x 1 Gbps upgrade …… WSRT (NL) 1 Gbps - - Onsala (SE) 1 Gbps 10 Gbps upgrade 0. 15 Torun (PL) 1 Gbps Jodrell Bank & Cambridge (UK) 2. 5 Gbps 10 Gbps upgrade (7 x 30 Gbps e-MERLIN**) 0. 12 Arecibo (USA) 155 Mbps 1 Gbps upgrade 0. 4 Medicina (IT) 1 Gbps Summer 2005 - Metsahovi (FI) 1 Gbps 2006 0. 33 - ** Crucial part of the plan is to combine e-MERLIN and e-VLBI Telescopes transparently. 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Telescope Status Comments Cost (Meuro) Yebes (ES) NYO 1 Gbps planned 2006/7 0. 53 VIRAC (LV) NYO 1 Gbps planned 2006/7 VLBI Capable 2006 ? 0. 53 Shanghai (CN) NYO 0. 1 - 1 Gbps planned Last mile: 25 km Poor connectivity between Europe & China (ORIENT) 0. 35 Miyun (CN) & Kunming (CN) NYO 0. 1 -1 Gbps planned Last mile: few km 0. 3 Urumqi (CN) NYO Fibre installed – 2 Mbps 0. 1 -1 Gbps planned 0. 1 Hartebeesthoek (SA) NYO 1 Gbps planned as part of SKA ambitions Poor connectivity between South Africa & Europe - TIGO (Chile) NYO Fibre installed - Noto (IT) NYO Last mile problem 3. 1 Sardinia (IT) NYO Last mile 1. 0 Effelsberg (DE) NYO Last mile 5 km but… 2. 0 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Towards “real”real-time • • Reliability of operation Robustness Feedback to stations Status display at stations and Jive Ease of operation Continuous monitoring Better protocols? Software consistency 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Protocol, buffer selection 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
FABRIC Future Array of Broadband Radio-telescopes on Internet Computing • Will need a new correlator for 4 Gb/s • Current EVN Mk 4 based on 16 x 1 Gb/s • Implemented on 1024 special chips • Next generation will most likely use standard CPUs • Current EVN Mk 4 processor equivalent 40 T-ops (2 bit) • LOFAR Blue. Gene ≈ 27 Tflops • 32 station x 4 Gb/s ⇒ 640 T-ops • And requirement to route 32 x 32 4 Gb/s input stream • Possible solution: distribute the computing • Use the Internet as the cross-switch • SETI@home gets 59 Tflops • Proposed to do pilot on the Grid 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Central astronomical scheduling Resource allocation from central control Transported data may be time/frequency slices Telescopes will do multicast transmission Output data stored in central archive, proprietary rights for users Compute nodes will be cluster size 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Conclusions, future • POC has been a success, but, progress has been both fast and slow • Bottlenecks hard to identify, packetloss does not equate congestion, fairness is a problem, relation packetloss— correlation poorly understood • Geant 2, Surfnet 6 will be hybrid networks • lambda switching • Bandwidth on demand • New protocols, software/hardware interfaces • 4— 10 Gbps transfer—new hardware? • e-VLBI is moving ahead; e-EVN will soon be a reality 4 th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
- Slides: 26