Tango polling system Tango workshop Archamps 08102005 5

  • Slides: 20
Download presentation
Tango polling system Tango workshop Archamps (08/10/2005) 5 -Oct-05 1

Tango polling system Tango workshop Archamps (08/10/2005) 5 -Oct-05 1

Tango workshop Introduction to Tango polling n n Every Tango device server process has

Tango workshop Introduction to Tango polling n n Every Tango device server process has a polling thread Its rule is to ¨ Poll attribute and/or command (without input parameters) ¨ Store result in a data cache (called polling buffer) managed as a circular buffer ¨ Detect events and throw them to the CORBA notification service 5 -Oct-05 2

Tango workshop Introduction to Tango polling n n Each device has its own polling

Tango workshop Introduction to Tango polling n n Each device has its own polling buffer Polling buffer depth is tunable ¨ By device (default is 10) ¨ By command/attribute for finer tuning n Client get their data from ¨ The real device ¨ The last record in the polling buffer ¨ The polling buffer and in case of error from the real device ¨ The choice is done by a Tango API client call 5 -Oct-05 3

Tango workshop Introduction to Tango polling n Configuring polling means ¨ Defining which command/attribute

Tango workshop Introduction to Tango polling n Configuring polling means ¨ Defining which command/attribute must be polled ¨ Defining polling periods n n Every polled object (command or attribute) has its own polling period Two way to configure polling ¨ From the Tango database (using Jive) ¨ Directly in code (the way used by Pogo) 5 -Oct-05 4

Tango workshop Introduction to Tango polling n According to the way polling is configured

Tango workshop Introduction to Tango polling n According to the way polling is configured (db or code), polling starts ¨ At device server process start-up time if configured from database ¨ At the first request to read data from polling buffer when configured by code 5 -Oct-05 5

Tango workshop The Polling has to be tuned n Nothing is magic! ¨ Don’t

Tango workshop The Polling has to be tuned n Nothing is magic! ¨ Don’t believe that the polling thread is able to poll everything at high frequency simply by decreasing polling period parameter ¨ The hardware access time has to be taken into account ¨ If you poll 5 objects (command or attribute) with 100 ms response time each, don’t try to poll them faster than 500 ms…(That’s life…) 5 -Oct-05 6

Tango workshop Polling tuning rule n Reserve some time for clients other than the

Tango workshop Polling tuning rule n Reserve some time for clients other than the polling thread to access your device ¨ The polling thread should not access the device for more than 50/60 % of time ¨ In the previous example, set polling period at 1 second leaving device free for external client during 50 % of time 5 -Oct-05 7

Tango workshop Checking polling status n The Tango device server process administration device has

Tango workshop Checking polling status n The Tango device server process administration device has one command to check polling ¨ Its name is Dev. Poll. Status ¨ This command returns n n Polling buffer depth Polling period Time needed to execute last command or to read attribute Delta time between the last recorded data (to check if it follows polling period) ¨ Jive 5 -Oct-05 displays the result of this command 8

Tango workshop 5 -Oct-05 9

Tango workshop 5 -Oct-05 9

Tango workshop De-tuned polling n It may happen that the hardware response time dramatically

Tango workshop De-tuned polling n It may happen that the hardware response time dramatically increase (disconnected serial line. . ) ¨ The polling thread is out of synchronization ¨ It tries to return on time by DISCARDING some work. Att 1 reading time Att 1 Att 2 Att 3 Att 4 Time Att 2 and Att 3 WILL NOT be polled !! 5 -Oct-05 10

Tango workshop De-tuned polling n Without discarding some works, we could saturate the device

Tango workshop De-tuned polling n Without discarding some works, we could saturate the device preventing any device access for the external world Att 1 reading time Att 1 Att 2 reading time Att 3 reading time Att 4 Time The device is not accessible any more for external world 5 -Oct-05 11

Tango workshop De-tuned polling n It could even be worse: ¨ The polling thread

Tango workshop De-tuned polling n It could even be worse: ¨ The polling thread is out of sync. ¨ Several polled objects are not polled any more ¨ The data stored within the polling buffer get older and older ¨ They even become too old (4 times the polling period) to be returned by the API. The API generates device access to return data to client instead of getting data from polling buffer ¨ We generate additional hardware access !! 5 -Oct-05 12

Tango workshop Tango kernel polling tuning n Tango kernel tries to automatically tunes the

Tango workshop Tango kernel polling tuning n Tango kernel tries to automatically tunes the polling thread ¨ At device server process startup time for polling configured from the database ¨ Every 500 poll times ¨ After a new object has been added to the polling thread list of polled object 5 -Oct-05 13

Tango workshop Tango kernel polling tuning n Tuning at device server startup time ¨

Tango workshop Tango kernel polling tuning n Tuning at device server startup time ¨ We don’t know attribute reading time nor command execution time. ¨ The polling tops are equally spread in the smallest polling period n Ex : 3 attributes polled at 600 m. S, 1000 m. S and 3000 m. S. The startup tuning will start polling at ¨ ¨ ¨ n T 0 for attribute 1 T 0 + 200 m. S (600/3) for attribute 2 T 0 + 400 m. S ((600/3) * 2) for attribute 3 This is done by a temporary thread dedicated to this issue 5 -Oct-05 14

Tango workshop Tango kernel polling tuning n Regular kernel polling tuning ¨ We now

Tango workshop Tango kernel polling tuning n Regular kernel polling tuning ¨ We now know command execution time or read attribute time ¨ The polling tops are spread in the smallest polling period according to their previous response time 5 -Oct-05 15

Tango workshop Tango kernel polling tuning n Ex : One device with 3 attributes

Tango workshop Tango kernel polling tuning n Ex : One device with 3 attributes polled n n n Att 1 with polling period = 600 m. S needing 400 m. S to be read Att 2 with polling period = 1000 m. S needing 40 m. S to be read Att 3 with polling period = 3000 m. S needing 10 m. S to be read Dead time is 150 m. S (600 – (400 + 40 + 10)) ¨ Att 1 polled at T 0 ¨ Att 2 polled at T 0 + 450 m. S (400 + (150 / 3)) ¨ Att 3 polled at T 0 + 540 m. S (400 + 40 + 2 * (150 / 3)) ¨ n Note that in this case, device is free for external access only 29% of time (10 + 3(40) + 5(400) = 2130 m. S compared to 3000 m. S) 5 -Oct-05 16

Tango workshop Event system and polling n n Event system needs polling Don’t forget

Tango workshop Event system and polling n n Event system needs polling Don’t forget to take this into account when defining polling period for attribute/command ¨ As soon as one application subscribes to one event on attribute, the event system ask polling thread to poll this attribute with a default polling period set to 1 sec if not already polled 5 -Oct-05 17

Tango workshop Event system and de-tuned polling n Tango release 5. 3 ¨ If

Tango workshop Event system and de-tuned polling n Tango release 5. 3 ¨ If the polling thread is out of sync and discards some attribute reading ¨ If one event has been subscribed for this attribute ¨ An event with error “Out of sync” is forced informing client that polling needs tuning 5 -Oct-05 18

Tango workshop Possible improvement (V 5. 4 ? ) n n Add an attribute

Tango workshop Possible improvement (V 5. 4 ? ) n n Add an attribute to device server process admin device with read value set to polling thread sleeping time during the last 5 seconds (or something like) In case of de-tuned polling, modify automatic tuning that it does not discard always the same polled object (with permutation of the polled list object) 5 -Oct-05 19

Tango workshop Possible improvement (V 5. 4 ? ) n Stop attribute polling for

Tango workshop Possible improvement (V 5. 4 ? ) n Stop attribute polling for attribute polled only for event detection when no more clients are interested in event (With the help of the event heartbeat) 5 -Oct-05 20