Instruments and Sensors on the Grid The Grid

Instruments and Sensors on the Grid The Grid. CC Project Gaetano Maron Istituto Nazionale di Fisica Nucleare Laboratori Nazionali di Legnaro, Legnaro Italy G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

Overview • Bringing Instrument into the Grid • The Grid. CC Project • The Instrument Element • The Grid. CC Test-bed: Pilot applications • Conclusions G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

Bringing Instruments into the Grid G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

The Grid Technologies to extend the limit of a single computer (center) Computing Element Storage Element User Interface Computing Element G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006 Computing Element

Extending the Grid Concepts Instrument Element Satellite views to monitor the volcano Terrestrial probes to monitor The volcano activities Virtual Control and Monitor Room G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006 To model calculations and disaster predictions

The Grid. CC Project tions lcula el Ca r Mod fo Data ctions Predi Instruments Grid Computational Grid Virtual Control and Monitor Room G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

The Gridcc Project G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

General on the Grid. CC Project • It is a 3 years project. Started the 1 st September 04 • Funded by EU in the Frame Program 6 (contract 511382) • 10 Partners from 3 EU Countries + (Israel) • About 40 people engagged • www. gridcc. org G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006 Participant name Country Istituto Nazionale di Fisica Nucleare Italy Institute Of Accelerating Systems and Applications Greece Brunel University UK Consorzio Interuniversitario per Telecomunicazioni Italy Sincrotrone Trieste S. C. P. A Italy IBM (Haifa Research Lab) Israel Imperial College of Science, Technology & Medicine UK Istituto di Metodologie per l’Analisi ambientale – Consiglio Nazionale delle Ricerche Italy Universita degli Studi di Udine Italy Greek Research and Technology Network S. A. Greece

Grid. CC Application Fields • Experimental Sciences – Take control of a experiment from a distance (remote operation and control, data taking and data analysis): • High Energy, Nuclear and Solid State Physics • Electronic Microscopes • Telescopes • Monitoring and analysis of the territory (e. g. disaster analysis) – Meteorology – Geophysics • Bio-medics – Integration of remote operation, data taking, data analysis and data storage of sophisticated instruments like: • Mammography • Pet, TAC, NMR etc. • Industrial Applications – widely distributed controls • Electrical power grid • Public transportation G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

The Grid. CC Architecture Security Services Virtual Control Room Information and Monitoring Services (IMS) Global Problem Solver Storage Element (SE) Collaborative Services (CS) Compute element (CE) Web Service Interface WMS Work Management System Execution Wf. MS Service Wf. MS Work Flow Mng System AS Agreement Service G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006 Instrument Storage elements Element Instrument (SE) (IE) Storage Element (IE) (SE) WMS AS Information System (IS) AS allows to negotiate advance Reservation on the IE, SE and CE

The Instrument Element G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

IE Requirements Web Services Storage Element Computing Element Instrument Element Any Protocol or physical connection Sensor Network Grid Instrument 1: Provide a uniform access to the physical device Computing Element Instrument Element 2: Allow a standard grid access to the instruments. (Low bandwidth) 3: Allows the data acquired from the Instruments to be published to subscribers (high bandwidth) 4: Allows standard access to the other Grid Element 5: Provide hard (reservation of IE) and soft (statistical prediction) guarantees of the IE’s methods execution times G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

Device Virtualization Model 1. 2. 3. 4. Instrument Parameters hold configuration information Attributes hold instrument variables Control Model hold actions XML Based Language to allow the device to describe itself Attributes Control Model Voltmeter XML Based Language • Parameters: Maximum Voltage, Minimum voltage • Attributes: measured Voltage • Commands: Perform a measure G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

Instrument Element interaction paths Data Virtual Control Room Subs cribe rs IE Status Execution Wf. MS Service IE SE Commands VIGS DP Grid Interaction IMS WMS AS 1) A Virtual Instrument Grid Service (VIGS) interface has been defined. The VIGS provides a Web Service acccess to the instrumentation 2) The Data Publishing (DP) channel disseminates the acquired information to the data subscribers 3) A SRM/SE interface is provided to allow file movement with other Grid Element, like SE, CE and other IEs. 4) Information messages (logs, errors, etc. ) are published from the IE (via IMS) G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006 s, r o r , Er nitors s g Lo s, Mo te Sta Storage Element (SE) Compute element Computing element (CE) Element (CE) VIGS Virtual Instrument Grid Service IMS Information and Monitor Service SE Storage Element DP Data Publishing

Instrument Element Interconnections Fast Data Publishing Log Display Data Consumer Existing Grid Elements VCR Control Panel IE Control and Status VCR Fast Data Display IE Computing Element (CE) IE Information and Monitor System G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006 Log Persistency

Instrument Element Architecture Problem Solver IMS SRM/SE Instrument Grid. FTP SE Resource Service • The term Instrument Element describes a set of services that provide the needed interface and implementation that enables the remote control and monitoring of physical instruments. Element Instrument Manager Data Mover Data Flow DP Access Control Manager VIGS IMS State Flow Error Flow Monitor Flow Control Flow Data Collector IMS Proxy Control Manager Real Instruments G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006 Control Manager Event Processor FSM Engine Input Manager Resource Proxy

IE middleware technologies • Tomcat + Axis (and Java) are the main technologies of the IE • All the services are deployed on a single or multiple instances of Tomcat, according to the needs of the application • Message oriented middleware (Pub/Sub) is based on the Java Messaging System (JMS). The following implementations are used in the project – Sun – Narada Brokering – RMM - JMS (IBM), see below for more details • My. SQL and Oracle are used as Data Base for the RS • Wf. MS engine based on BPEL • EGEE g. LIte is the reference framework for – WMS – CE (g. Lite CREAM is used where possible) – SE + SRM • A modified version of STORM is used as SRM for the IE G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

Access Control Manager Instrument Element Implementations The IE components are typically implemented into a fully equipped Machines (e. g. dual core cpus, large memory, large disks, etc). This is true for RS, IMS and PS. For IM (and DM) there are 2 possibilities, according to the application type: • IM implemented in a fully equipped machine • IM embedded into the instrument that should be controlled Instrument Element Resource Service Instrument Manager Inf & Mon Service Problem Solver Data Mover IMS RS IM IM G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

VIGS Custom Board DP Xilink Virtex IV SE PPC 405 FPGA Linux Java. VM Grid. CC IM Web Service IMS Custom Electronics 1 Gbps Ethernet Instrument Element on a Chip Grid on a Chip USER INTERFACE G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

IE Data Producer Typical Use Case: 1 to N IE G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006 Data Subscribers (Monitor, local storage, etc. ) Fast Data Publishing Same data are sent to several subscribers. Multicast protocols can have a benefic impact In term of performance • J MS Provide a standard set of API that standardize this communication system • Many Commercial and academic implementation of this API exist in both C/C++ and Java (Narada. Brokering, Sun, IBM, Sonic. MQ etc ) • No one has a good Multicast implementation • Gridcc (IBM Haifa lab) has implemented a Reliable Multicast protocol (RMM) JMS compliant • RMM-JMS works within a LAN, but an efficient bridge technology has been developed to allow inter-LAN multicast communication

RMM-JMS Performance one to many case 60 MByte/s IE 500 kmsg/s G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

RMM-JMS Performance round trip time (RTT, Latency) • Two machines with a single publisher and a single subscriber on each one • Average round trip time computed over 1000 samples G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

Grid. CC Pilot Applications G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

Main IE Pilot Applications: Power Grid Virtual Control Room Instrument Manager Power Grid V. O Instrument Element Gas . . . G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006 Solar

Main Grid. CC Pilot Applications: Control and Monitor of high energy experiments In collaboration with the CERN CMS/Tri. DAS group G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

The CMS Data Acquisition 2 107 electronics channels 40 MHz • O(104 ) distributed Objects to – control – configure – monitor • On-line diagnostics and problem solving capability • Highly interactive system (human reaction time fraction of second) 100 Hz G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006 • World Wide distributed monitor and control

CMS Prototype G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

CMS Prototype: IEs at work - Grid. CC middleware used for CMS MTCC (Magnet Test and Cosmic Challenge) - 11 Instrument Elements with a hierarchical topology - Instruments are in these case Linux hosts where the cms on-line software is running - More than 100 controlled hosts CMS Instrument Elements TOP Det 11 Detector GTPe 8 1 DAQ IE Instrument Managers DAQ Trigger - First phase of data taking successfully ended - Second phase starting in few weeks TTS Filter. Farm Fed. Builder G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006 Ru. Builder

Main Grid. CC Pilot Applications: Remote Operation of an Accelerator Elettra Synchrotron Demo available in the Demo session G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

The other Grid. CC pilot applications • Meteorology (Ensemble Limited Area Forecasting) • Device Farm for the Support of Cooperative Distributed Measurements in Telecommunications and Networking Laboratories • Geo-hazards: Remote Operation of Geophysical Monitoring Network G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

Conclusion (I) • The Grid. CC project is integrating instruments into the “classic” computational/storage Grids. • Novel concepts introduced by Grid. CC are: – The Instrument Element, allowing a virtualisation of the instruments to be controlled and their insertion in a Grid – The Virtual Control Room, providing an highly interactive environment with IEs/CEs/SEs. VCR even provides cooperative tools to allow the cooperation (logbook, chat, videoconf. , etc. ) between remote users – hard (reservation of IE) and soft (statistical prediction) guarantees of the IE’s methods execution times – Fast Data Publication via a Message Oriented middleware (RMM-JMS) to distribute data and information from an IE to the world wide Grid. • The IE is highly customizable and can be adapted in different environments. G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

Conclusions (II) • The IE can be shrunk down into a chip allowing grid enabled embedded control of the instrumentation • Several heterogeneous pilot applications are deploying and running the IE middleware • We support and encourage the adoption of our middleware in other projects/experiments. Direct help can be provided since now, tutorial will be available by the end of 2006 G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006

Question? • Thx for your time More information: www. gridcc. org On-line Demo at: http: //sadgw. lnl. infn. it: 2002/IEFacade Acknowledgement: The Grid. CC project is supported under EU FP 6 contract 511382. G. Maron, EGEE 06 Conference, CERN, 25 -29 September, 2006