Progress on EXPRe S at JBO EXPRe S

Progress on EXPRe. S at JBO EXPRe. S Board Meeting, Utrecht, Jan 2008 Ralph Spencer and the JBO team

Contents • • • Outline of JBO project Manpower Protocol work Progress on i. BOBs Handover to Richard. Jan 2008 Progress with i. BOBs at Jodrell 2

Outline of Work at JBO • NA 4 Outreach work – Alastair Gunn - completed • SA 1 Connection of 4 e-MERLIN telescopes to JIVE (see under Fabric) • SA 2: The 10 Gbps data link equipment available, awaiting cross connection in Manchester between academic and commercial links • Fabric: Protocols for high speed networks –Stephen Kershaw : PFLDNet 2008 Manchester 5 -7 March http: //www. hep. man. ac. uk/PFLDnet 2008/about. Manchester. php • Fabric: Input to e-MERLIN correlator at 4 Gbps Jan 2008 Progress with i. BOBs at Jodrell 3

E-MERLIN---e-VLBI system Jan 2008 Progress with i. BOBs at Jodrell 4

January 2008 Network protocols Stephen Kershaw

TCP • Is TCP suitable for transferring realtime data? • If data is lost TCP will slow transmission and retransmit. • Demonstrate the feasibility of using TCP for real-time applications Jan 2008 Progress with i. BOBs at Jodrell 7

CUBIC TCP variants • TCP using different algorithms for the window/throughput reduction • Some variants allow more efficient use of bandwidth and appear more suitable for realtime applications Jan 2008 High. Speed Progress with i. BOBs at Jodrell 8

TCP variants • Orders of magnitude improvements for speed of recovery with many variants • Orders of magnitude reduction in buffering Note spikes due to bursts of data • Evident bursty behaviour – concerns for highly loaded packet switched networks Jan 2008 Progress with i. BOBs at Jodrell 9

Multicast reduces load on networks when sending to multiple hosts Test of multicast topology • Multicast UDP stream over JANET academic network • 100 2. 2 Mbps multicast IPTV streams originating from London, UK • End-hosts on Manchester University campus network • All Cisco network hardware, 100 Mbps interfaces • Measurement of inter-packet arrival times • Time from arrival of packet ‘n’ to arrival of packet ‘n+1’ • Expected ~6 ms for one 2. 2 Mbps stream Jan 2008 Progress with i. BOBs at Jodrell 10

Multicast • Latency spikes of up to 0. 5 s at intervals of 60 s • Unexpected behaviour may limit the application of this technology • Correlation between streams and between PCs shows a common source of delay Jan 2008 Progress with i. BOBs at Jodrell 11

Multicast Test of multicast topology - results • Multicast traffic routing is not as efficient as unicast on this network • Potential concerns for other networks • Does not look suitable for e. VLBI Jan 2008 Progress with i. BOBs at Jodrell 12

DCCP Stable kernel • Require tuning of virtual memory settings to prevent kernel panics Experimental kernels • Revisions of late November 2007 address some memory and performance issues – yet to be tested on our systems. • CCID 4 in development Performance • CCID 2 can attain 1 GE line-rate but it is not a turn-key thing Future work • Test 2. 6. 24 kernel and recent DCCP git tree – not mature enough yet for a definitive statement about suitability Jan 2008 Progress with i. BOBs at Jodrell 13

Progress on Digital Interfaces i. BOBs and all that Jonathan Hargreaves/Richard Hughes-Jones

The i. BOB System • As a transmitter: • Now able to transfer data at line rate to a test PC over a 10 GE (CX-4) link • Needs A/D connection and interface (Onsala/Metsahovi) • As a receiver • Needs to interface to VSI chip on station board • As a network test system • See also Richard’s talk • Fujitsu switch ordered Jan 2008 Progress with i. BOBs at Jodrell 15

JBO Receiver: Status Recent Progress • Sixteen packet FIFO implemented in the i. BOB’s FPGA • Even though packets are transmitted at 4 Gbps they might ‘bunch up’ and arrive at line speed. The FIFO is intended to smooth them out again • The 10 Gb transceiver can run at 200 MHz – enough to handle line speed, but the SRAM bandwidth is 167 MHz for both read and write. Read out at 4 Gbps consumes 64 MHz of that. More resource than needed • Data flows from the i. BOB to the Station Board defined (see next slide) • Data readout from the SRAM synchronised to the correlator clock, when present • When correlator clock is not present, i. BOB defaults to test mode using the internal clock • VSI chip will buffer and reclock data and route it to the Station Board input Next Tasks • Test the new code in hardware using another i. BOB to generate test packets • Write firmware for the VSI chip and either simulate or test on remote hardware Jan 2008 Progress with i. BOBs at Jodrell 16

JBO Receiver: Block Diagram Clock Control Lock 128 MHz clock to correlator 200 MHz System Clock 128 MHz . 10 GB RX FIFO ‘Yellow box’ on the Simulink design 16 k deep 4 GB/s CX 4 SRAM Control. Re-orders out of order packets Holds up to 16 packets in case short bursts arrive at line rate DATA 64 bit . Counts lost packets DATA 32 bit 512 k x 64 bit (512 packets = 9 ms at 4 Gbps) 2 x MDR 80 ‘VSI’ cables . DATA to SB Input Station Board . Jan 2008 Progress with i. BOBs at Jodrell DATA to SB Input Reroute it to the left and right polarisation station board (SB) inputs. DAT A SRAM i. BOB Buffer and reclock data. Correlator CLK DATA 32 bit Reads data out to the correlator at a steady 4 Gbps ADD R ‘VSI chip’ 17

Network Testing Device: Status Recent Progress • System clock increased from 100 MHz to 200 MHz to allow line speed data flows at 10 Gbps (JH) • Added jumper bank to select ethernet IP and MAC address (JH & RHJ) • All registers, test modes and results can be accessed via ethernet control/response protocol (RHJ) • PC based software provides user interface to the ethernet control link (RHJ) • Ethernet based control supersedes the Tiny. SH command line and is seen as a prototype for control of i. BOBs sending real data • Currently testing i. BOB to i. BOB transfers on bench Next Tasks • Count lost packets, out of order packets and duplicate packets • Update real time field correctly each second – currently it is fixed over a burst • Test over network Jan 2008 Progress with i. BOBs at Jodrell 18