Ice Cube DAQ Design and Performance Thorsten Stezelberger

Ice. Cube DAQ Design and Performance Thorsten Stezelberger LBNL VLVn. T 2 Workshop, Catania Nov 8 -11, 2005 9/16/2020 1

Overview VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • Ice. Cube • Design Requirements • Ice. Cube DAQ • Reliability, Quality Assurance • Lessons Learned • Performance Ice. Cube DAQ Design and Performance Thorsten Stezelberger 2

Ice. Cube VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • 70 -80 Strings • 60 DOMs per String • Ice. Top Air Shower Array on top of String Ice. Cube DAQ Design and Performance Thorsten Stezelberger 3

Design Requirements VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • • • 10 -15 Years lifetime goal Reliability / Robust Design Remotely controllable Failure must not propagate Resistant to Radio Frequency Interference Expected Hit rate ~0. 8 k. Hz Low power consumption < 5 W/DOM Surface temperatures -20°C to -70°C At-depth temperatures -10°C to -40°C Ice. Cube DAQ Design and Performance Thorsten Stezelberger 4

Physics Requirements VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania • Timing, 7 ns RMS • Waveform capture 4 ns RMS 300 MS/s for 400 ns, 40 MS/s for 6. 4µs • Charge Dynamic Range >200 PE/15 ns ~500 PE/15 ns • In Situ Calibration (internal and external) • In-Ice Hardware Local Coincidence 9/16/2020 Ice. Cube DAQ Design and Performance Thorsten Stezelberger 5

DAQ System Overview Ethernet VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania DAQ E l e c t r o n i c s 9/16/2020 Ice. Cube DAQ Design and Performance Thorsten Stezelberger 6

Timedomains VLVn. T 2 Workshop GPS Master Clock Nov 8 - 11, 2005 Catania Hub DOMs (60/string) SP (String Processor) DOM: “local time units” Hub: “ master time units” SP: transforms local time stamps to master time 9/16/2020 Ice. Cube DAQ Design and Performance Thorsten Stezelberger 7

DOMHub DOM Power Supplies VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania Power Distr. Card Chassis Fans 8 DOR Cards 9/16/2020 CPU Ice. Cube DAQ Design and Performance Thorsten Stezelberger Hard GPS Drive distr. 8

DOM Readout Card VLVn. T 2 Workshop DOM Communication DOM Power Input GPS Master Time Input Nov 8 - 11, 2005 Catania Main Cable Connector 9/16/2020 PCI interface Ice. Cube DAQ Design and Performance Thorsten Stezelberger DOM Power Distribution 9

Main Cable VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • 0. 9 mm Twisted Quad Copper Cable • 2. 5 km long (continuous in ice) + Surface Cable • 2 DOMs per wire pair • Power, Timing and Communications over the same wire pair • > 50 d. B suppression near end cross talk • > 30 db suppression far end cross talk Ice. Cube DAQ Design and Performance Thorsten Stezelberger 10

Digital Optical Module VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania Passive Base HV Supply Flasher Board Main Board Delay Board 9/16/2020 Ice. Cube DAQ Design and Performance Thorsten Stezelberger 11

DOM Main Board VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 Ice. Cube DAQ Design and Performance Thorsten Stezelberger 12

DOM Main Board Trigger (2) VLVn. T 2 Workshop Nov 8 - 11, 2005 Pulser 1 5 4 8 x 16 Delay x 2 x 0. 25 Catania x 2. 6 x 9 MUX OB -LED 10 b 40 MHz ADC CPU DP Ram 1 megabaud DOR 8 b 10 b ATWD f. ADC (n+1) (n– 1) 10 b ATWD 10 b FPGA and SOPC DAC LPF +/-5 V, 3. 3 V, 2. 5 V, 1. 8 V DC -DC Configuration Device 32 b LC 8 Mbit SDRAM 16 Mb 20 MHz Oscillator Corning Frequency Ctl (was Toyocom) 9/16/2020 Monitor & Control DACs & ADCs 8 b, 10 b, 12 b 16 b CPLD PMT Power Ice. Cube DAQ Design and Performance Thorsten Stezelberger Flash 4 Mb 8 b Flasher Board Flash 4 Mb 64 Bytes 13

DOM Main Board Trigger (2) VLVn. T 2 Workshop Nov 8 - 11, 2005 Pulser 1 5 4 8 x 16 Delay x 2 x 0. 25 Catania x 2. 6 x 9 MUX OB -LED 10 b 40 MHz ADC CPU DP Ram 1 megabaud DOR 8 b 10 b ATWD f. ADC (n+1) (n– 1) 10 b ATWD 10 b FPGA and SOPC DAC LPF +/-5 V, 3. 3 V, 2. 5 V, 1. 8 V DC -DC Configuration Device 32 b LC 8 Mbit SDRAM 16 Mb 20 MHz Oscillator Corning Frequency Ctl (was Toyocom) 9/16/2020 Monitor & Control DACs & ADCs 8 b, 10 b, 12 b 16 b CPLD PMT Power Ice. Cube DAQ Design and Performance Thorsten Stezelberger Flash 4 Mb 8 b Flasher Board Flash 4 Mb 64 Bytes 14

Altera Excalibur VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • FPGA 400, 000 Gates • ARM 9 CPU • Bus bridges between CPU and FPGA • Dual Ported Memory between CPU and FPGA • Power consumption 0. 5 – 0. 7 W Ice. Cube DAQ Design and Performance Thorsten Stezelberger 15

DOM Main Board Trigger (2) VLVn. T 2 Workshop Nov 8 - 11, 2005 Pulser 1 5 4 8 x 16 Delay x 2 x 0. 25 Catania x 2. 6 x 9 MUX OB -LED 10 b 40 MHz ADC CPU DP Ram 1 megabaud DOR 8 b 10 b ATWD f. ADC (n+1) (n– 1) 10 b ATWD 10 b FPGA and SOPC DAC LPF +/-5 V, 3. 3 V, 2. 5 V, 1. 8 V DC -DC Configuration Device 32 b LC 8 Mbit SDRAM 16 Mb 20 MHz Oscillator Corning Frequency Ctl (was Toyocom) 9/16/2020 Monitor & Control DACs & ADCs 8 b, 10 b, 12 b 16 b CPLD PMT Power Ice. Cube DAQ Design and Performance Thorsten Stezelberger Flash 4 Mb 8 b Flasher Board Flash 4 Mb 64 Bytes 16

ATWD Analog Transient Waveform Digitizer VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • ASIC • Switched Capacitor Array • 4 Channels x 128 Samples Deep • Sampling on Launch • Sample Speed 250 800 MSamples/s • Digitization, ~30µs/waveform • Power consumption 125 m. W Ice. Cube DAQ Design and Performance Thorsten Stezelberger 17

DOM Main Board Trigger (2) VLVn. T 2 Workshop Nov 8 - 11, 2005 Pulser 1 5 4 8 x 16 Delay x 2 x 0. 25 Catania x 2. 6 x 9 MUX OB -LED 10 b 40 MHz ADC CPU DP Ram 1 megabaud DOR 8 b 10 b ATWD f. ADC (n+1) (n– 1) 10 b ATWD 10 b FPGA and SOPC DAC LPF +/-5 V, 3. 3 V, 2. 5 V, 1. 8 V DC -DC Configuration Device 32 b LC 8 Mbit SDRAM 16 Mb 20 MHz Oscillator Corning Frequency Ctl (was Toyocom) 9/16/2020 Monitor & Control DACs & ADCs 8 b, 10 b, 12 b 16 b CPLD PMT Power Ice. Cube DAQ Design and Performance Thorsten Stezelberger Flash 4 Mb 8 b Flasher Board Flash 4 Mb 64 Bytes 18

Oscillator VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • Reference “time” of DOM • High Reliability Part • Screened for short term stability Ice. Cube DAQ Design and Performance Thorsten Stezelberger 19

DOM Main Board Trigger (2) VLVn. T 2 Workshop Nov 8 - 11, 2005 Pulser 1 5 4 8 x 16 Delay x 2 x 0. 25 Catania x 2. 6 x 9 MUX OB -LED 10 b 40 MHz ADC CPU DP Ram 1 megabaud DOR 8 b 10 b ATWD f. ADC (n+1) (n– 1) 10 b ATWD 10 b FPGA and SOPC DAC LPF +/-5 V, 3. 3 V, 2. 5 V, 1. 8 V DC -DC Configuration Device 32 b LC 8 Mbit SDRAM 16 Mb 20 MHz Oscillator Corning Frequency Ctl (was Toyocom) 9/16/2020 Monitor & Control DACs & ADCs 8 b, 10 b, 12 b 16 b CPLD PMT Power Ice. Cube DAQ Design and Performance Thorsten Stezelberger Flash 4 Mb 8 b Flasher Board Flash 4 Mb 64 Bytes 20

Local Coincidence VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania • Reduce Data Rate • Rate without LC ~1 KHz • Rate with LC <15 Hz • Dedicated Full Duplex LC connection • Transmit and Receive go through FPGA • LC functionality can be reprogrammed 9/16/2020 Ice. Cube DAQ Design and Performance Thorsten Stezelberger 21

Reliability, Quality Assurance VLVn. T 2 Workshop DOMs are not accessible after deployment Nov 8 - 11, 2005 Catania 9/16/2020 • By Design • Production • Testing Ice. Cube DAQ Design and Performance Thorsten Stezelberger 22

Design VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • • • Qualified Manufacturer List GIDEP (Government-Industry Data Exchange Program) No Wet Electrolytic Capacitors Derate Parts HALT (Highly Accelerated Lifetime Testing) Input Serial number; electronic, barcode and human readable • Robust Boot Mode • Little Redundancy in the DOM (Deployed Spares) • MTBF calculation not reliable (Temperature range) Ice. Cube DAQ Design and Performance Thorsten Stezelberger 23

Production VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • IPC 610 class 3 for Board production and loading • No Microwelds in PCB • Use of Industrial Temperature Parts • Tin-Lead Parts wherever possible (NO pure tin parts) • Close work with Board manufacturer and loading house for fast feedback • Digital picture of Boards after loading • ESD Precautions Ice. Cube DAQ Design and Performance Thorsten Stezelberger 24

Testing VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • • • Initial room temperature Test HASS (Highly Accelerated Stress Screening) 48 hour Main Board burn-in hot and cold Interface test Integrated DOM burn-in 21 days at various Temp. Test before Deployment at ~-25°C In Summary; Testing, more Testing and even more … All Test results are stored in Database ESD Precautions Ice. Cube DAQ Design and Performance Thorsten Stezelberger 25

Lessons Learned VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • Test Framework (bench, integration, before deployment and after deployment) • DOMHub can run standalone • Terminal Connection to DOM • Script Language/Interpreter on DOM Ice. Cube DAQ Design and Performance Thorsten Stezelberger 26

Performance VLVn. T 2 Workshop One String and 4 Ice. Top Stations are deployed Nov 8 - 11, 2005 Catania 9/16/2020 • Time Calibration • Noise/Discriminator Threshold • Muon Reconstruction Ice. Cube DAQ Design and Performance Thorsten Stezelberger 27

Time Calibration Reciprocal Active Pulsing VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania Verified using: • Flasher Boards • Ice. Top –Ice. Cube coincidences • AMANDA – Ice. Cube coincidences for 76 DOMs 9/16/2020 Ice. Cube DAQ Design and Performance Thorsten Stezelberger 28

Clock Stability VLVn. T 2 Workshop Time Calibration every 3. 5 s Nov 8 - 11, 2005 Catania 9/16/2020 Ice. Cube DAQ Design and Performance Thorsten Stezelberger 29

Noise/Discriminator Threshold VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • PMT Gain 107 • Noise in the Ice is ~800 Hz • Threshold as low as ~1/8 SPE Ice. Cube DAQ Design and Performance Thorsten Stezelberger 30

Muon Reconstruction VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 Ice. Cube DAQ Design and Performance Thorsten Stezelberger 31

Summary VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 • • DAQ Design Reliability One Ice. Cube string deployed Four Ice. Top Stations deployed String is working very well All 76 DOMs function well In Jan, Feb up to 10 more Strings deployed Full Array deployed by 2009 -10 Ice. Cube DAQ Design and Performance Thorsten Stezelberger 32

Thank You VLVn. T 2 Workshop Nov 8 - 11, 2005 Catania 9/16/2020 Ice. Cube DAQ Design and Performance Thorsten Stezelberger 33