Ice Cube 5000 optical modules on 86 strings

Ice. Cube 5000 optical modules on 86 strings at the South Pole to look for high energy muons produced by neutrino absorption in or under the ice The optical modules are placed between 1. 5 and 2. 5 km below the ice surface The glas envelope contains radioactive potasium that causes 500 Hz fake hits av. 13 hit in a 5 s window The system triggers on different conditions. One is to trigger on more than 32 hit in window, i. e. > 5 s above average August 24, 2011 IDAP kick-off meeting - Track engine

Track Engine Deployment Diagram August 24, 2011 IDAP kick-off meeting - Track engine

Ice. Cube: Track Engine Collaboration with particle physics (Ice. Cube) Clyde Robson – software Carl Wernhoff – firmware (FPGA) Hossein Kaviannipour – hardware/software interface New topological trigger for Ice. Cube Sort hit data from 5000 digital optical modules Look at most recent hits in overlapping 5 s windows Form pairs of hits, or "tracklets" Sort tracklets by direction Muons produce clusters of tracklets around true direction, noise hits have random direction More efficient than current triggers Flexible hardware implementation, allows future upgrade remotely from home institutes Successful hardware installation at the pole, Jan 2011 August 24, 2011 IDAP kick-off meeting - Track engine

Track Engine overview Global trigger Ethernet Track engine server Sort Engine August 24, 2011 PCIe Track Engine FPGA IDAP kick-off meeting - Track engine

Track Engine Implementation August 24, 2011 IDAP kick-off meeting - Track engine

Track Engine Deployment Diagram server Monitoring software Sample processing software August 24, 2011 Control software Sort engine IDAP kick-off meeting - Track engine

Track Engine hardware August 24, 2011 IDAP kick-off meeting - Track engine

Track Engine Deployment Diagram August 24, 2011 IDAP kick-off meeting - Track engine

Challenges with the hardware design working with integers (not floats) → careful truncations (MSB and LSB side) where possible → careful error approximation calculations August 24, 2011 IDAP kick-off meeting - Track engine

Challenges with the hardware design working with integers (not floats) → careful truncations (MSB and LSB side) where possible → careful error approximation calculations trigonometric functions → could be avoided in this case with mathematical tricks August 24, 2011 IDAP kick-off meeting - Track engine

Challenges with the hardware design working with integers (not floats) → careful truncations (MSB and LSB side) where possible → careful error approximation calculations trigonometric functions → could be avoided in this case with mathematical tricks parts of the algorithm hard to implement in logic/describe in HDL → auto-generated HDL code (placing pairs in bins) August 24, 2011 IDAP kick-off meeting - Track engine

Challenges with the hardware design working with integers (not floats) → careful truncations (MSB and LSB side) where possible → careful error approximation calculations trigonometric functions → could be avoided in this case with mathematical tricks parts of the algorithm hard to implement in logic/describe in HDL → auto-generated HDL code (placing pairs in bins) final design is very complex (10'000 lines of HDL) → thorough testing of subunits and whole design → custom test environment alone is 10'000 lines of sw code August 24, 2011 IDAP kick-off meeting - Track engine

Challenges with the hardware design working with integers (not floats) → careful truncations (MSB and LSB side) where possible → careful error approximation calculations trigonometric functions → could be avoided in this case with mathematical tricks parts of the algorithm hard to implement in logic/describe in HDL → auto-generated HDL code (placing pairs in bins) final design is very complex (10'000 lines of HDL) → thorough testing of subunits and whole design → custom test environment alone is 10'000 lines of sw code final product must be robust, design mustn't "freeze"/hang for any unexpected input data → (thorough testing as described above) → recording internal errors so they can be caught ("assertion-based design in hardware") → on-chip "watch-dog" monitorer capabel of resetting ciritical parts August 24, 2011 IDAP kick-off meeting - Track engine

Pro's and con's hw/sw Con's with a hardware solution: complex design expensive in terms of development time hardware competence is rare compared with software competence software code can be read and understood by most physicists - a hardware design can't August 24, 2011 IDAP kick-off meeting - Track engine

Pro's and con's hw/sw Con's with a hardware solution: complex design expensive in terms of development time hardware competence is rare compared with software competence software code can be read and understood by most physicists - a hardware design can't Pro's with a hardware solution superior performance not achievable in software cheap in terms of rack space and power consumption August 24, 2011 IDAP kick-off meeting - Track engine

Track Engine Deployment Diagram August 24, 2011 IDAP kick-off meeting - Track engine

Simplified schematic of the TE system August 24, 2011 IDAP kick-off meeting - Track engine

Com. Pack environment

Communication Package (Com. Pack) First design: Embedded system

Communication Package (Com. Pack) First design: Embedded system Platforms: Xilinx V 5 & V 6

Communication Package (Com. Pack) First design: Embedded system Platforms: Xilinx V 5 & V 6 Architectures: PPC, Microblaze

Communication Package (Com. Pack) First design: Embedded system Platforms: Xilinx V 5 & V 6 Architectures: PPC, Microblaze Operating Systems: xilinux, uclinux, petalinux

Com. Pack Embedded

Communication Package (Com. Pack) Second design: Standalone PCIe

SPD – Stockholm. University PCIe Driver Capabilities: Dynamic DMA buffer allocation

SPD – Stockholm. University PCIe Driver Capabilities: Dynamic DMA buffer allocation MSI & MSI-X support

SPD – Stockholm. University PCIe Driver Capabilities: Dynamic DMA buffer allocation MSI & MSI-X support Wait queue/Work queue

SPD – Stockholm. University PCIe Driver Capabilities: Dynamic DMA buffer allocation MSI & MSI-X support Wait queue/Work queue Standard interface (read/write)

SPD – Stockholm. University PCIe Driver Capabilities: Dynamic DMA buffer allocation MSI & MSI-X support Wait queue/Work queue Standard interface (read/write) Concurrency proof