Communication link strategiesingredients for FCC S Danzeca ENSTI

Communication link strategies/ingredients for FCC S. Danzeca (EN-STI) on behalf of the R 2 E Project logo area 6 th HL-LHC Workshop – Paris – 14/16 November 2016

Communication link • In order to control equipment, send data and receive data, the communication link is a fundamental part of an accelerator • The communication link requirements can be different depending on the application • Control safety electronics, measurements streaming, audio/video survey. They can have different requirements. • Each application can require different protocols and communication means • The communication link should work in radiation areas • Can we extrapolate the current needs to FCC? logo area S. Danzeca - Planning and strategy for HL-LHC R 2 E 2

Strategy § The needs for FCC are not yet clear and it can be complex to build just one solution pretending it will work in more than 30 years § The solution: study and develop multiple highend solutions which can be implemented with or without the same technology also in 30 years. § These solutions must be compatible with the radiation levels in the FCC locations where the electronics will be installed. § The feasibility study of the implementations today represents already an important steps in all the future developments logo area S. Danzeca - Planning and strategy for HL-LHC R 2 E 3

Communication link study § High-end solutions exploitable for the FCC § § Twisted pairs communication link (discarded) Ethernet based solutions Fiber optic based solutions Wireless solutions § For each of them we can study the basic building blocks and requirements § The building blocks are not necessary related to a technology or a chip but are means of effectively verify that link could be exploited (within certain limit) logo area S. Danzeca - Planning and strategy for HL-LHC R 2 E 4

Ethernet based solutions § Ethernet advantages: § Speed 10 -100 Mbit/s up to 1 Gbit/s § Topology: point to point and daisy-chain § Several protocols can be implemented using the same basic structure § Audio/video communication possible § Basic building blocks Core logo area S. Danzeca - Planning and strategy for HL-LHC R 2 E

Ethernet based solutions § Current status FPGA: § § § § Current status Ethernet PHY: § § § Several FPGAs/Microcontroller can be suitable for 10 -100 Mbit/s link. 10 -100 Mbit MAC can be implemented Artix SRAM based FPGA and Flash based Smart. Fusion 2 are under investigation for their tolerance and failure rate in radiation environment Artix FPGA is Latchup free but several mitigation schemes should be applied for making the link robust. This solution could work in low radiation areas. The SRAM solution is not dependent on the Artix but the development methodology and mitigation techniques can be ported to any tolerant SRAM FPGA. Flash based FPGA Smart. Fusion 2 has very low cross section for Latchup event. It can be used as an easy development tool for assessing the feasibility of the Ethernet link without applying complex mitigation scheme. It has the advantage of having hard-coded a ARM processor which can reduce the development time. Possible test of a very simple solution this year. No solutions yet for the 1 Gb/s link due to the necessity to have a radiation tolerant SERDES for the MAC implemented in the FPGA On the track to join as partner a consortium for the development of a radiation hard ARM PHY. The project name is Space Ethernet Physical Layer Transceiver (SEPHY) and it is an European project already started and leaded by TTECH and Arquimea. NDA signed, waiting for the inkind contribution. First tape out of the chip during this year. In parallel we tested a 1 Gb/s PHY under irradiation and we plan to make a radiation test on a 10/100 Mbit during this year. On going path: § § At the moment all the FPGA developments are based on a soft or hard core microprocessor which helped in verifying the functionalities of the link. But the soft, hard core are more prone to single events and the migration strategies are primitive. The need is to develop a more robust structure on the FPGA which can be more easily mitigated This solution can be ported to any FPGA also in the future. Resources needed logo area

Fiber Optic based solutions § Fiber optic solution advantages: § § Speed 5 -10 Gb/s (>10 Gb/s) Possible to cover very long distances (km) Slow control signalling possible Audio/video communication possible § The topology of the network can be only point to point. The daisy chain is possible but two links should be implemented on the same card § Basic building blocks taking as an example the GBTx CERN development logo area S. Danzeca - Planning and strategy for HL-LHC R 2 E

Ethernet based solutions § Current status Transceiver: § § § Current status Small Form Factor (SFP) Transceiver: § § § GBTX 4. 8 Gb/s Transceiver developed by EP is implemented in several boards and can be used as a baseline for all the developments. Long term support. Rad. Hard by design New version of GBTX for HL currently under development VTTx is a Rad. Hard by design physical layer developed by EP Current version provided by EP is multimode but a single mode is being developed for the accelerator sector thanks to a collaboration between EP and BE. Current status FPGA/ASIC: § Scalable Sensor Data Processor (SSDP) is a mixed-signal ASIC DSP processor § § § The design and development of the SSDP is an activity stemming from the European Space Agency and is supported by a consortium lead by Thales Alenia Space. We are including in the SSDP the e-link interface with which we will be able to talk with the GBTX and have a Rad. Hard by design DSP processor On going path: § § At the moment the SSDP need a followup with the activities that Thales is going to do on the prototype. First it will be necessary a verification of the inclusion of the elink in the design. Then the SSDP should be tested in our environment with in mind some intense computing application Resource needed logo area

Wireless solutions § Wireless solution advantages: § § § Wifi ~ 1 Gb/s Client layer Possible to cover long distance Partially no cabling required (not always true i. e 3 G in the LHC tunnel) Several devices in meshed/distributed network Audio/video communication possible § The basic structure is depending on the type of chosen network. § Meshed network can be exploited with Iot devices (we are going to make a test in CHARM during this year) § IT-CS, EN-STI and University of Malta have started a collaboration in order to have a PJAS that will gather the necessary requirement for a wireless network structure and specifications. Not only one type of network is exploitable! § The aim is also to assess the possible induced failures due to the radiation. § The following steps would be to find the solutions (client and partially master side) that can deal with the selected network type. logo area S. Danzeca - Planning and strategy for HL-LHC R 2 E 9

Conclusions and Outloooks • Several technologies can be exploited for FCC communication link. • Nowadays several important steps have been done in order to create the basic building blocks necessary for the development common solutions for: • Ethernet based link • Fiber optic based link • For the Ethernet Link currently on-going activities are related to the development of a core to handle the communication on the FPGA and verify the robustness. In addition, a Rad. Hard physical layer is being investigated with SEPHY consortium. • For the Fiber communication currently on-going activities are related to the inclusion of the elinks into the SSDP processor. • Currently this task need a follow up in order to verify the correct inclusion of the elink IP by EP in the SSDP core. • The wireless network can be a serious option to consider. The constraints and the requirements should be well set in order to start studying possible basic structures. logo area S. Danzeca - Planning and strategy for HL-LHC R 2 E 10

Thank you logo area S. Danzeca - Planning and strategy for HL-LHC R 2 E 11

HS Tester System architecture overview • High Speed tester is currently under development thanks to the collaboration between ENSTI, BE-BI, EP-DT • First test this year in CHARM testing 3 x. ADC at 144 Mb/s for a total of 432 Mbs CHARM Control room Radiation Area Front-End System Back-End System HDD Network OL 0 Test routine software DMAPCIe Interface OL 1 OL 2 OL 3 Server Computer logo area Felix Board VTRx High-Speed Optical Link (OL) Up to 4. 8 Gbps GBTx DUT 0 DUT 1 Rad-Tol FPGA GEFE System DUT… FMC Mezzanine DUT board S. Danzeca - Planning and strategy for HL-LHC R 2 E 12

Radiation Hardness Assurance § § § We are promoting a Radiation Hardness Assurance procedure for the new developments and for the systems already installed. A draft of the procedure document is at this link : https: //edms. cern. ch/document/1740220/1 Simple structure: § RHAPV: Project Validation (new project) -> report of the project information, radiation environment, radiation tests § RHACD: Check document (existing equipment) -> report the card changed and if they are conform with the RHAPS. § RHAPS: Process structure -> Pure RHA guideline which give information on the process and guide the user through the testing method and effectiveness. New Developments • Have to follow a radiation assurance procedure RHA • The criticality should be assessed • The system has to be tested in a representative radiation environment logo area System already installed • Their fault rate should be assessed • The relocation should be notified • The integration document will have a field pointing at the RHA document • Any system change should be notified S. Danzeca - Planning and strategy for HL-LHC R 2 E 13