CERNIPMC Introduction ATLAS Upgrade IPMC Workshop CERN EPESEBE

CERN-IPMC: Introduction ATLAS Upgrade - IPMC Workshop CERN EP-ESE-BE Julian Mendez 10/10/2018 CERN-IPMC - Introduction 1

ATLAS Upgrade - IPMC Workshop OUTLINE q CERN IPMC – Introduction § § § x. TCA evaluation project – general overview CERN-IPMC: specifications CERN-IPMC: hardware CERN-IPMC: software configuration Demo: introduction of the CERN-IPMC environment q CERN-IPMC – Demonstration § § Introduction Adding a simple temperature sensor (LM 75): driver and instantiation Case of a multi-purpose sensor (LTC 2990) How does a sensor driver work? 10/10/2018 CERN-IPMC - Introduction 2

ATLAS Upgrade - IPMC Workshop OUTLINE q CERN IPMC – Introduction § § § x. TCA evaluation project – general overview CERN-IPMC: specifications CERN-IPMC: hardware CERN-IPMC: software configuration Demo: introduction of the CERN-IPMC environment q CERN-IPMC – Demonstration § § Introduction Adding a simple temperature sensor (LM 75): driver and instantiation Case of a multi-purpose sensor (LTC 2990) How does a sensor driver work? 10/10/2018 CERN-IPMC - Introduction 3

ATLAS Upgrade - IPMC Workshop OUTLINE q CERN IPMC – Introduction § § § x. TCA evaluation project – general overview CERN-IPMC: specifications CERN-IPMC: hardware CERN-IPMC: software configuration Demo: introduction of the CERN-IPMC environment q CERN-IPMC – Demonstration § § Introduction Adding a simple temperature sensor (LM 75): driver and instantiation Case of a multi-purpose sensor (LTC 2990) How does a sensor driver work? 10/10/2018 CERN-IPMC - Introduction 4

ATLAS Upgrade - IPMC Workshop XTCA EVALUATION PROJECT – GENERAL OVERVIEW q x. TCA evaluation project launched in 2011 in EP-ESE q Main objectives: § § § Technical evaluation of components for MTCA, MTCA. 4 and ATCA systems Technical evaluation of AC/DC converters Development of tools (H/W and S/W) for the testing of commercial components Conduct market surveys Report and share results q Longer term goal § Standardize MTCA and ATCA shelves and power supplies o Many options (backplanes, cooling, RTMs, power supply, …) § Propose acceptance test procedures § Propose a selected set of equipment to the experiments § Provide centralized support for these items 10/10/2018 CERN-IPMC - Introduction 5

ATLAS Upgrade - IPMC Workshop XTCA EVALUATION PROJECT – GENERAL OVERVIEW q Important number of different equipment evaluated, both in MTCA and ATCA q Modules were selected and purchase contract were established q An MMC module was designed for the AMC cards (m. TCA) MTCA Crates Vadatech VT 892 ATCA boards ATCA Crates ELMA 043 -012 MTCA. 4 Schroff MTCA. 4 + AC/DC CM 100 Schroff 14 -slot Schroff MTCA. 4 PMs ASIS 14 -slot Kontron AMC Carrier Comtel Load blade MCHs NAT DC 780 Vadatech UTC 010 Wiener AC/DC Vadatech UTC 001 NAT MCH Kontron AM 4904 AC/DC Power Supply Emerson Net. Sure 501 Power. One Aspiro Power. One Guardian Lineage Power CPS AMCs ELMA Load Board Kontron AM 5030 10/10/2018 ESD ADIO 24 CCT AM 31 CERN-IPMC - Introduction 6

ATLAS Upgrade - IPMC Workshop XTCA EVALUATION PROJECT – GENERAL OVERVIEW q An IPMC project was launched in 2015 – Based on the commercial solution from Pigeon Point System q Pigeon Point: § Part of the Pentair group § Provide solutions for the Hardware Platform Management modules: o Module Management Controller (MMC) o Intelligent Platform Management Controller (IPMC) o Shelf Manager (Sh. MM) § Pigeon Point shelf manager is used by almost all of the shelf manufacturers q A license for the IPMC solution was acquired by CERN EP-ESE-BE. It includes: § Documentation § Development tools § Hardware, firmware and software reference designs 10/10/2018 CERN-IPMC - Introduction 7

ATLAS Upgrade - IPMC Workshop XTCA EVALUATION PROJECT – GENERAL OVERVIEW q Pigeon solution was evaluated in 2015 using: § The starter kit IPMC mezzanine card § An Adapter card (CERN) to use the mezzanine with existing ATCA blade q Evaluation outcomes: ü ü ü ü Standard compliance (Polaris tester) Management of the Advanced. TCA blade Management of up to 4 AMCs E-Keying and Clock configuration OEM commands Non-intelligent RTM Ethernet interface Remote upgrade (HPM. 1) Pigeon Point IPMC mezzanine 10/10/2018 CERN-IPMC - Introduction 8

ATLAS Upgrade - IPMC Workshop XTCA EVALUATION PROJECT – GENERAL OVERVIEW q Considering: § The positive outcome of the PP IPMC evaluation (ensures the compliance with the standard) § The reasonable price of the solution Decision was taken to develop an IPMC mezzanine based on the Pigeon Point commercial solution. q Development project was started in 2016 q Key considerations: § Compatibility with already designed ATCA blades: o Use the same form factor as the IPMC already developed by LAPP o Stay pin compatible with the existing IPMC § Adapt the Pigeon Point solution to be General purpose and easily customizable 10/10/2018 CERN-IPMC - Introduction 9

ATLAS Upgrade - IPMC Workshop OUTLINE q CERN IPMC – Introduction § § § x. TCA evaluation project – general overview CERN-IPMC: specifications CERN-IPMC: hardware CERN-IPMC: software configuration Demo: introduction of the CERN-IPMC environment q CERN-IPMC – Demonstration § § Introduction Adding a simple temperature sensor (LM 75): driver and instantiation Case of a multi-purpose sensor (LTC 2990) How does a sensor driver work? 10/10/2018 CERN-IPMC - Introduction 10

ATLAS Upgrade - IPMC Workshop CERN-IPMC: SPECIFICATIONS q Role of the Intelligent Controller for Advanced. TCA blades: § Monitoring sensors (Voltages, temperatures …) § Controlling the system (Power management, port/clock activation …) § Ensuring properations (Compatibility between the boards, hot swap, redundancy …) 10/10/2018 CERN-IPMC - Introduction 11

ATLAS Upgrade - IPMC Workshop CERN-IPMC: SPECIFICATIONS FULLY TESTED q Supported features: § Advanced. TCA Rev. 3. 0 standard • • • Hot swap (FRU info. , handle switch, LEDs, Hardware address, etc. ) Sensor monitoring (SDR, measurement, events, etc. ) Rear Transition Module (intelligent and non-intelligent RTM) § AMC standard (up to 9 AMCs – including i. RTM) § Ethernet interface (RMCP/RMCP+, UDP, simplified TCP/IP) § Serial interface (So. L or debug interface) § I 2 C buses for sensor monitoring and power management § User I/Os (35 User I/Os + 16 IPM I/Os) § JTAG Master (Xilinx Virtual Cable daemon) 10/10/2018 CERN-IPMC - Introduction 12

ATLAS Upgrade - IPMC Workshop CERN-IPMC: SPECIFICATIONS q Functional block diagram: MICROSEMI SMARTFUSION SOC CORTEX ARM M 3 10/10/2018 CERN-IPMC - Introduction 13

ATLAS Upgrade - IPMC Workshop OUTLINE q CERN IPMC – Introduction § § § x. TCA evaluation project – general overview CERN-IPMC: specifications CERN-IPMC: hardware CERN-IPMC: software configuration Demo: introduction of the CERN-IPMC environment q CERN-IPMC – Demonstration § § Introduction Adding a simple temperature sensor (LM 75): driver and instantiation Case of a multi-purpose sensor (LTC 2990) How does a sensor driver work? 10/10/2018 CERN-IPMC - Introduction 14

ATLAS Upgrade - IPMC Workshop CERN-IPMC: HARDWARE q Typical use of the CERN-IPMC 10/10/2018 CERN-IPMC - Introduction 15

ATLAS Upgrade - IPMC Workshop CERN-IPMC: HARDWARE q Management interface comes from the ATCA backplane connector (Zone 1) § Provides all connections required between the IPMC and the shelf manager. q Direct routing from IPMC to zone 1 connector. q Capacitors and resistors required for hardware address signals q I 2 C buffers for IPMB-A/B buses (REQ 2. 298 and 2. 299) are implemented on the mezzanine. Typical case – Zone 1 connection 10/10/2018 CERN-IPMC - Introduction 16

ATLAS Upgrade - IPMC Workshop CERN-IPMC: HARDWARE q Handle switch is active low by default. § § Power up sequence starts when signal is at GND (10 k pull-up required). Active state can be easily modified in the software. q Blue LED is driven through a TPS 61070 booster (5 V) q Additional LEDs are driven by the controller: § § IO at 3. 3 V Maximum current: 30 m. A Typical case – LEDs and Handle switch 10/10/2018 CERN-IPMC - Introduction 17

ATLAS Upgrade - IPMC Workshop CERN-IPMC: HARDWARE q ATCA blade power management ensured by specific pins and Mgt I 2 C bus. q Additional power management devices can be added to protect against over-v. /cur. § Already protected by FUSES and IQ 65033 module § Power. Good signals monitoring can be disabled in the software Typical case – Power 10/10/2018 CERN-IPMC - Introduction 18

ATLAS Upgrade - IPMC Workshop CERN-IPMC: HARDWARE q JTAG to IPMC: § JTAG slave interface can be used to configure IPMC FPGA using a Flash. Pro 4 JTAG programmer. § E. g. : connection to a front panel connector Typical case – JTAG slave q JTAG Master: § JTAG master interface is made of GPIO running JTAG commands at 5 MHz. § Connectivity between the ATCA on-board devices and the IPMC is user’s responsibility. § Directly routed from the CERN-IPMC controller without any pull-up/down nor capacitors. 10/10/2018 CERN-IPMC - Introduction 19

ATLAS Upgrade - IPMC Workshop CERN-IPMC: HARDWARE q The Ethernet lines available on the IPMC are ready to be connected to: § RJ 45 connector § To a switch (e. g. TS 3 L 110) for dual connectivity (E. g. : front panel or backplane). q Required pull-ups and PHY transceiver are located on the IPMC mezzanine Typical case – Ethernet 10/10/2018 CERN-IPMC - Introduction 20

ATLAS Upgrade - IPMC Workshop CERN-IPMC: HARDWARE q UART port: § Connection to the payload is recommended: o Sending alerts (e. g. : sensor events) o Sending configuration (e. g. : using PI intf or Serial Over Lan) § Using the UART for the Debug interface (optional) 10/10/2018 CERN-IPMC - Introduction 21

ATLAS Upgrade - IPMC Workshop CERN-IPMC: HARDWARE q Support of up to 8 AMCs + 1 i. RTM q I 2 C buffers must be added for each slot on the IPMB-L bus § Prevent communication issues upon insertion/extraction q Supported power management devices: § § § TPS 2358/2458 TPS 2459 LTC 4222 10/10/2018 CERN-IPMC - Introduction Typical case – Power 22

ATLAS Upgrade - IPMC Workshop OUTLINE q CERN IPMC – Introduction § § § x. TCA evaluation project – general overview CERN-IPMC: specifications CERN-IPMC: hardware CERN-IPMC: software configuration Demo: introduction of the CERN-IPMC environment q CERN-IPMC – Demonstration § § Introduction Adding a simple temperature sensor (LM 75): driver and instantiation Case of a multi-purpose sensor (LTC 2990) How does a sensor driver work? 10/10/2018 CERN-IPMC - Introduction 23

ATLAS Upgrade - IPMC Workshop CERN-IPMC: SOFTWARE q No NDA needed. § Project can be customized and compiled remotely: core source code is kept private. q Project architecture: config. xml IPMC-sensors Drivers + compile. py script IPMC-user OEM commands Callback functions q Gitlab URL: https: //gitlab. cern. ch/ep-ese-be-xtca/ipmc-project 10/10/2018 CERN-IPMC - Introduction 24

ATLAS Upgrade - IPMC Workshop CERN-IPMC: SOFTWARE q Why does the software have to be modified? § User customizable features: o FRU Information (Device ID, Manufacturer info. , Product info. ) o LAN (MAC address, Default IP, slot specific IP, DHCP, Gateway, Netmask) o Modules (AMCs, i. RTM/Non-intelligent RTM) o Sensors o E-Keying o Power sequencing q Almost all of the modifications can be done using the config. xml file § Documented on git: https: //gitlab. cern. ch/ep-ese-be-xtca/ipmc-project 10/10/2018 CERN-IPMC - Introduction 25

ATLAS Upgrade - IPMC Workshop CERN-IPMC: SOFTWARE q General configuration: § § § Device details Board details Power information Handle switch states Serial interface configuration 10/10/2018 CERN-IPMC - Introduction 26

ATLAS Upgrade - IPMC Workshop CERN-IPMC: SOFTWARE q Power ON/OFF sequences: § List of step to be executed q The steps can be: § § § PSQ_ENABLE_SIGNAL(signal_name) PSQ_DISABLE_SIGNAL(signal_name) PSQ_TEST_SIGNAL_JUMP_IF_SET(signal_name, offset): PSQ_TEST_SIGNAL_JUMP_IFNOT_SET(signal_name, offset) PSQ_SET_TIMER(timer, timeout) PSQ_JUMP(offset) PSQ_JUMP_IF_TIMEOUT(timer, offset) PSQ_JUMP_IFNOT_TIMEOUT(timer, offset) PSQ_SET_RECOVERY_POINT(address) PSQ_END PSQ_FAIL q Signal name listed on the pinout documentation (here) 10/10/2018 CERN-IPMC - Introduction 27

ATLAS Upgrade - IPMC Workshop CERN-IPMC: SOFTWARE q LAN interface configuration: § MAC Address § Net. Mask § Gateway q IP can be set to be configured via DHCP or per slot (exclusive: IPAddr. List or Enable. DHCP) 10/10/2018 CERN-IPMC - Introduction 28

ATLAS Upgrade - IPMC Workshop CERN-IPMC: SOFTWARE q AMC/i. RTM ports can be enabled and connected to specific IPMC ports q E. g. : AMC slot 1 (0 x 72) is connected to IPMC port 1 with a max. current of 6 Amps. q Almost the same for i. RTM except that the site number is replaced by the I 2 C address. 10/10/2018 CERN-IPMC - Introduction 29

ATLAS Upgrade - IPMC Workshop CERN-IPMC: SOFTWARE q Sensors can be instantiated using the config. xml file § Additional details during the “CERN-IPMC – Demonstration” part of the workshop 10/10/2018 CERN-IPMC - Introduction 30

ATLAS Upgrade - IPMC Workshop OUTLINE q CERN IPMC – Introduction § § § x. TCA evaluation project – general overview CERN-IPMC: specifications CERN-IPMC: hardware CERN-IPMC: software configuration Demo: introduction of the CERN-IPMC environment q CERN-IPMC – Demonstration § § Introduction Adding a simple temperature sensor (LM 75): driver and instantiation Case of a multi-purpose sensor (LTC 2990) How does a sensor driver work? 10/10/2018 CERN-IPMC - Introduction 31

ATLAS Upgrade - IPMC Workshop DEMO: INTRODUCTION OF THECERN-IPMC ENVIRONMENT q Based on the CERN-IPMC Dev. Kit: Connectors Shelf manager Serial, Ethernet, JTAG Master IPMB-0 IPMC slot I 2 Cs Sensor, EEPROM 9 AMC ports Hardware address, handle switch, LED, etc. MMC AMC CPLD 35 User I/Os + 16 IPM I/Os Control Mgt u. C Serial Connector 10/10/2018 CERN-IPMC - Introduction 32

ATLAS Upgrade - IPMC Workshop DEMO: INTRODUCTION OF THECERN-IPMC ENVIRONMENT IPMC Tester Self Manager board 10/10/2018 CERN-IPMC - Introduction 33

ATLAS Upgrade - IPMC Workshop DEMO: INTRODUCTION OF THECERN-IPMC ENVIRONMENT q Project creation q Customize the CERN-IPMC software to run on the CERN-IPMC Dev. Kit q Compile the project q Load the binary using HPM. 1 (Ethernet) q Run the CERN-IPMC test (using the CERN-IPMC Dev. Kit) q Configure an Artix 7 FPGA using the JTAG Master interface q Documentation: https: //gitlab. cern. ch/ep-ese-be-xtca/ipmc-project 10/10/2018 CERN-IPMC - Introduction 34

Demo 10/10/2018 CERN-IPMC - Introduction 35

ATLAS Upgrade - IPMC Workshop SUMMARY q NDA signature is not needed. q CERN-IPMC is ready to be used and the package ready to be purchased. q The CERN-IPMC kit includes: § § The CERN-IPMC mezzanine card (200. - CHF) Access to the ipmc-project repository (https: //gitlab. cern. ch/ipmc-project) Access to the CERN-IPMC Web. App (https: //cern-ipmc. web. cern. ch) Access to the CERN-IPMC documentation (https: //cern-ipmc. web. cern. ch/doc) o Datasheet o Hardware o Software o Sensor q Forum is available: https: //cern-ipmc. web. cern. ch/forum 10/10/2018 CERN-IPMC - Introduction 36