T 13 Concentrator ASIC Jim Hirschauer L 3

T 13: Concentrator ASIC Jim Hirschauer L 3 Manager, 402. 04. 07 November 29, 2017

Scope § Specification, design, prototyping, production, and quality control of the concentrator ASIC. § The concentrator ASIC collects data from several HGCROC signaldigitizing ASICs and transmits it to the backend (off-detector) electronics. § The concentrator ASIC generates and transmits reduced granularity trigger primitives at 40 MHz, and serializes and transmits full granularity “DAQ” data at 750 k. Hz. § We have been calling the concentrator ASIC “ECON” for Endcap CONcentrator. T 13: Concentrator ASIC J Hirschauer, 2017 November 29 2

Conceptual Design T 13: Concentrator ASIC J Hirschauer, 2017 November 29 3

Requirements § Trigger data transmission at 40 MHz § DAQ data transmission at 750 k. Hz § Maximum latency 16 BX (additional 20 BX in HGCROC) § Impacts maximum depth of derandomizing buffer § Radiation tolerance: 1. 5 MGy, 1 e 16 1 Me. V-equiv neutrons/cm 2, SEU compliance § 65 nm TSMC process § SEU tolerant design including TMR, error detection/correction § Power <= 5 m. W / channel (4 -9 W per ASIC) § 65 nm TSMC process § power optimized design and algorithms T 13: Concentrator ASIC J Hirschauer, 2017 November 29 4

Conceptual Design : Input / Output § ROC ECON trigger § 2 -4 e’-links (1. 44 Gbps) § Fixed format § 8 -bit floating point (4 mant. + 4 exp. ) per trigger channel. § ROC ECON data § Single e’-link (1. 44 Gbps) § Variable format after zero suppression § ECON BE trigger § 3 lp. GBT’ (10 Gbps, 8. 96 Gbps useable). § Variable format after trigger algorithm § ECON BE data § 6 e-links feeding single otherwise unused uplink on lp. GBT (8. 96 Gbps useable). § Variable format after zero suppression T 13: Concentrator ASIC § FNAL engineers working with CERN to modify e-link/lp. GBT tech for ECON-specific use. § E-links overclocked to 1. 44 Gbps – extra 4/36 bits/BX used for alignment and error correction. J Hirschauer, 2017 November 29 5

Conceptual Design : trigger path § ROC ECON trigger § 8 -bits (4+4) per trigger channel. § Relatively straightforward § ECON trigger algorithm ECON BE data format § Ship with good granularity for regions of interest. § Ship energy sums for uninteresting regions 11 -bit floating point (7 mant. + 4 exp) § Out-of-time pileup correction in ECON BE bandwidth § ECON BE trigger 200 PU § 10 Gbps = 256 bits / BX 1 k § 208 bits / BX user § Require >3 trigger links per motherboard in busy regions of detector – split motherboards , include 2 ECON / motherboard, adjust trigger algorithm Motherboard § 16 bits for alignment Bits per BX § 32 bits for clock and error correction Layer T 13: Concentrator ASIC J Hirschauer, 2017 November 29 6

Conceptual Design : data path § Data formats § ADC readout for Q>0. 5 MIP § ADC+To. A for Q>10 f. C (~10% of ADC) § ADC+To. A+To. T for Q>150 f. C (~0. 1% of ADC) § Pre-ADC for QBX-1>2. 5 MIP (~30% of ADC) § Channel list for low occupancy regions, channel bit map for high occupancy regions § ECON BE data rates 200 PU § 10 Gbps = 256 bits / BX 9 k § 32 bits for clock and error correction § Data rates <10 kbit per L 1 A event – will fit into 1 DAQ link per motherboard. T 13: Concentrator ASIC Bits per BX § 192 bits user data @ 750 k. Hz = 10 kbit per event Motherboard § 32 bits for alignment and fast control Layer J Hirschauer, 2017 November 29 7

Conceptual Design : data path § Data formats § ADC readout for Q>0. 5 MIP ADC hit occupancy § ADC+To. A for Q>10 f. C (~10% of ADC) § ADC+To. A+To. T for Q>150 f. C (~0. 1% of ADC) § Pre-ADC for QBX-1>2. 5 MIP (~30% of ADC) T 13: Concentrator ASIC J Hirschauer, 2017 November 29 8

Conceptual Design : silicon layers § Motherboard layout for 8 th CE-E layer § ECON BE (trigger) data rates [Gbps] in black (red) for 200 PU sensor ch / module ROC / module sensor ch / ROC trigger ch / ROC data e-links / ROC trigger e-links / ROC module / concentrator ROC / concentrator sensor ch / concentrator input data e-links / concentrator input trigger e-links / concentrator total input e-links / concentrator T 13: Concentrator ASIC J Hirschauer, 2017 November 29 Fine Coarse 432 192 6 3 72 8 1 2 64 16 1 4 4 24 1728 4 12 768 24 48 72 12 48 60 9

Conceptual Design : mixed layers § Motherboard layout for 12 th CE-H layer § Motherboards for Si. PM readout reside at periphery of the cassette § Challenge: distance for ROC-ECON elink ~1. 5 m. ~1. 5 m T 13: Concentrator ASIC J Hirschauer, 2017 November 29 10

Fabrication and Testing § Fermilab ASIC group has extensive experience designing, prototyping, producing, and testing ASICs. § Radiation testing for cumulative effects (TID, NIEL) and single event effects at every stage prototype production – with dedicated test structures embedded in design. § Accelerated aging tests. § System Verilog simulation of full sensor ROC ECON BE system. § Robotic system for testing ~10 k ECON ASICs before assembly onto motherboards. § Critical challenge: full system testing with HGROC prototypes. T 13: Concentrator ASIC J Hirschauer, 2017 November 29 11

Interfaces § Physical space on motherboard § Space is not severely constrained § Logic levels § Well defined e-link and lp. GBT protocols § Supply voltages § Relatively simple to document § Data formats – most challenging interface § Custom formats for all 4 data paths. § Already working closely with design team for digital backend of ROC § Working to establish same connection with TPG BE collaborators. T 13: Concentrator ASIC J Hirschauer, 2017 November 29 12

Safety and Hazards § Ionizing radiation during ASIC radiation testing. § Follow FNAL and radiation facility protocols. § Mechanical hazards during operation of robot ASIC tester. § Follow FNAL protocols § System subjected to regular FNAL safety reviews. TXX: [Talk Title] [FI Last Name], 2017 November XX 13

R&D and Engineering T 13: Concentrator ASIC J Hirschauer, 2017 November 29 14

Institutional and Personnel Involvement § Jim Hirschauer (FNAL) § Led phase 1 upgrade of HCAL barrel/endcap. § Led production and testing of QIE 10/11. § Jim Hoff (FNAL digital ASIC engineer) § 20+ years experience in digital design for HEP. § Recently designed concentrator ASIC for DUNE (1. 2 gbps). § Sandeep Miryala (FNAL, recently NIKHEF) § 4+ years experience in high-speed digital design § Experience in high speed projects: GBT, lp. GBT, RD 53 § Drew Baden and Tullio Grassi (Maryland) § Extensive HEP DAQ/trigger experience in HCAL phase 0 and phase 1 upgrade. § Currently focusing on development of System Verilog simulation. § M. Klute (MIT), M. Velasco (Northwestern), F. Yumiceva (FIT) § Physics simulations for algorithm development, data rate determination, etc. TXX: [Talk Title] [FI Last Name], 2017 November XX 15

R&D Plans § 2018 : complete design for first prototype § Fully specify data formats, error handling, fast signals § Port e-link and lp. GBT designs to ECON § Implement simple trigger algorithm § 2019 : complete design for second prototype § Produce and evaluate first prototype § Continue design improvements in parallel § Implement final trigger algorithm § 2020 : launch production § Produce and evaluate second prototype in engineering run. § Final design adjustments § Ready for production in late 2020 T 13: Concentrator ASIC J Hirschauer, 2017 November 29 16

Risks § Additional engineering run is required § § 25% probability $250 k cost impact 6 -9 months schedule impact Mitigation: Comprehensive testing of first prototype, comprehensive full system simulation, early development of test infrastructure and procedures. § Additional production run is required § § 10% probability $1. 2 M cost impact 6 -9 months schedule impact Mitigation: Comprehensive testing of engineering run, comprehensive full system simulation, early development of test infrastructure and procedures, second engineering run. TXX: [Talk Title] [FI Last Name], 2017 November XX 17

Value engineering § ECON ASIC results from value engineering § Reduces numbers of optical links by factor of 13 to 8 k total links § $280 cost / link (VTTx, fiber, BE channel) § ECON saves >$27 M in link costs (ECON cost = $3. 7 M). § Exploit potential similarities in requirements for potential concentrator ASIC for MIP timing detector. T 13: Concentrator ASIC J Hirschauer, 2017 November 29 18

Milestones Milestone description Milestone Date Submit prototype Sep 2018 Submit engineering run Sep 2019 Submit full production Jun 2020 Complete quality control Jan 2021 T 13: Concentrator ASIC J Hirschauer, 2017 November 29 19

Conclusion and Outlook § Critical activities for first prototype in 2018 § § Integration of e-link / lp. GBT technology into ECON Finalization of specification based on physics simulations Initial power estimates Development of initial full system simulation in System Verilog § Keys to overall success: § Clarity and documentation of interfaces to ROC and BE § Full system simulation § Testing, testing … T 13: Concentrator ASIC J Hirschauer, 2017 November 29 20

Additional Information T 13: Concentrator ASIC J Hirschauer, 2017 November 29 21

Review Charge [Draft] 1. Are the physics requirements clearly stated and documented? Have the requirements been translated into technical performance requirements and specifications? Is the design technically adequate and is it likely to meet the technical requirements? 2. Can the design be constructed, inspected, tested, installed, operated and maintained in a satisfactory way? 3. Is there adequate supporting documentation to support the conceptual design? What is the current level of maturity of design, and is there a well-reasoned and documented R&D plan to continue the project to a baselining level of maturity? Do any new technologies need to be developed? Is the prototyping plan well-reasoned and adequate? 4. Are the risks (technical, cost, and schedule) of the selected base design approach and alternatives understood and documented, and are appropriate steps being taken to manage and mitigate these risks? 5. Have areas been identified where value engineering should be done? If value engineering has been performed, is it documented? What other designs, if any, were considered? Is the current design well matched to the technical specifications? 6. Are the project organization and lines of responsibility clearly defined and sufficient to ensure the successful engineering and design of the project? Are the design interfaces between different areas of the project, and between on -project and off-project areas understood, documented, and well defined to ensure a coordinated effort and an integrated design? Is there a documented plan in place for implementing configuration management to ensure changes to the technical requirements / specifications are controlled and communicated to all affected groups? 7. Is there proper attention and documentation of ESH&Q, including safety, hazards, quality assurance/quality control, and reliability? In particular, please review the methods used for requirements validation. 8. Is there a reasonable set of milestones that the project is managing to, both during the R&D and construction phase? T 13: Concentrator ASIC J Hirschauer, 2017 November 29 22