SLC Aware IOC Database Service Design Debbie Rogind

SLC Aware IOC Database Service Design Debbie Rogind Nov 11, 2004

Topics n DBS Use Cases n Design Considerations n DBS Initialization n Data Structures n Download Sequence n ST 0 Conversion n Receive/Process Setpoints and other messages from Alpha n Process/Send SLC IOC udpates to Alpha n DB Input/Output Utilities

DB Service Use Cases n DBEX - “Look like” a tcp micro n Same message formats, traffic, use proxy n Req and Accept database (“IPL”) n Accept updated setpoints from Alpha n Provide updated data from IOC n Supertype 3 (ST 3) readbacks n Supertype 2 (ST 2)setpoints (new) n n Accept and reply to diagnostic inquiries Accept and Process DBEX up/down messages n Console Users n Utilities to edit, dump, get type of various slc data n Application code n n n Utilities to find, get, put, update slc data Coded “data flow dependancies” ? ? ? What else

Design Considerations n Port as much existing micro code as n n possible Messages from VMS control system are little-endian, VMS format, and packed The VMS supertype header structure is packed – no “pad” exists between 4 byte boundaries Support ASCII representations for primary, secondary names and units Slc IOC needs supplementary data definition: n n n primary ASCII to # secondary (secn) ASCII to # mappings for given primary datum width (for I or Z)

Topics n DBS Use Cases n Design Considerations n DBS Initialization n Data Structures n Download Sequence n ST 0 Conversion n Receive/Process Setpoints and other messages from Alpha n Process/Send SLC IOC udpates to Alpha n DB Input/Output Utilities

DB Thread Initialization n slc. Exec starts Phase 1 message Q’ed threads db. Send, db. Hdlr n slc. Exec starts Phase 2 db. Recv after all Phase 1 actives are T n All db threads n n n Create cmlog handle Init resources n Message queue n Sockets n Memory heap n Timers, semaphores Set active = T n Wait (at message Q or socket) n db. Hdlr thread initializes shared db thread globals n slc. Exec sends “DB_DOWNLOAD” to db. Send

Topics n DBS Use Cases n Design Considerations n DBS Initialization n Data Structures n Download Sequence n ST 0 Conversion n Receive/Process Setpoints and other messages from Alpha n Process/Send SLC IOC udpates to Alpha n DB I/0 Utilities

DB Service Globals for db threads, initialized by db. Hdlr: Name Type Represents dbnodes_p[5] dbsuptype_tu Pointers to superblocks 0 -3 db. Recv mallocs, writes pointers & data while other threads are blocked at db download event. After download event pointers read-only (ptr to ST 0 & data deleted; rest never change) dict_p void * Pointer to dictionary; new char-based gphash table created for housing supertype 0 data (secondary data & location) and valid ASCII names. gphash has its own mutex protection for add/delete/find dictionary utilities. dbex_up db. Version epics. Boolean char * Status of dbex up (T) or down (F) Db major / minor version w dbex_up msg db. RWMutex epics. Mutex. Id Activiated for reads or writes from/to supertype 1 -4 data. db. Recv (for Alpha VMS updates), and dbl* I/O utilities access dbhilo_updates [N_JOBS] Dbhilo_job_ts * Each “job service” array contains pairs of offsets to upper and lower bounds of modified ST 2/3 data. Acc’ed by utility rte dbhilo_update called from dblput. dbhilo. Mutex [N_JOBS} epics. Mutex. Id Activiated for reads or writes from/to dbhilo_updates[ ]. Acced by db. Send, and private utility dbhilo_update called by dblput db. Ack. Event [N_JOBS} epics. Event. Id Job service threads wait at this event until their updated data has been acknowledged by Alpha • download. Event, db. Exists flag is created by slc. Exec • db. Hdlr. Thread signals download. Event, sets db. Exists flag after db download / ST 0 conversion

Supertype 0 Byte Stream to/from Alpha fwd_hdr_ts proxy header dbsuptypehdr_ts supertype header hash length array listhead ptrs array node counts links – next UPS, unit, collided nodes len - ups + nsub’s catn - prim # unit - unit # nsub - # of secns UPS block 0 secn block [0 – nsub]: supn | subn fmt | slen sptr = offset in supn … …. UPS block X secn block [0 – nsub]

Supertype 1 -3 Byte Stream fwd_hdr_ts proxy header dbsuptypehdr_ts supertype header data data ….

Supertype Header & Data Name Data type Represents ip_port_u len user cmd ip_port_tu int 4 u user_field_ts int 1 u crc int 1 u Lower half of the ip address, and the port number Message bytecount - this fwdheader User defined; chunk count for large buffers; Fwd_server command, e. g. 8 bit crc over header; currently set to 0 x 55 fwd_hdr_ts dbsuptypehdr_ts network netlsn id <<<<< len micr blkbeg blkend datw[ ] Note or int 2 u potential byte int 4 u Int 4 u int 4 u Array of int 2 u dbsuptype_tu = union Type of data alignment mistmatch here >>>>> Size of superblock Microname Copy to location in superblk Copy from location in superblk type-specific data; packed; max size is NETBUFLEN minus sizeof(dbsuptypehdr_ts)/2

Buffer/Q Structures n Rcvbuf, sndbuf get cast to dbsndrcvbuf_ts ndb. Recv converts to native. SB -VMStoh. Sup. Hdr() Name Data type dbsndrcvbuf_ts proxy_hdr supblk native. SB Represents Receive and send buffers fwd_hdr_ts dbsuptype_tu dbsuptypehdr_ts Proxy header Superblock header + data Native superblock header structure added for slc ioc n. DB Thread Queue message structure Name Data type dbmsgmail_ts msgheader native. SB dbdiag Represents Q messages msgheader_ts dbsuptypehdr_ts dbdiagmsg_ts Msg header Native superblk hdr Diagnostic data n db. Send uses native. SB n. Converts prior to sending hto. VMSSup. Hdr()

Secondary Data n The secondary data is discussed in later slides (after ST 0 Conversion and the “new” dictionary is discussed)

Topics n DBS Use Cases n Design Considerations n DBS Initialization n Data Structures n Download Sequence n ST 0 Conversion n Receive/Process Setpoints and other messages from Alpha n Process/Send SLC IOC udpates to Alpha n DB I/0 Utilities

DB Service Download Block Diagram

db. Recv. Thread – Download (“IPL”) Sequence n Rcvbuf receives Supertype 0 -3 messages from DBEX at dbex socket n VMStoh. Sup. Hdr(rcvbuf. native. SB, rcvbuf. supblk) n Sends “DB_DBEX_ACK” to db. Send Q to ack DBEX (if req’ed) n Validates data and proper sequence of download (dbmicromail) n Allocates super block memory (dbsuperalloc) from heap; populates dbnode_p[0] n Copies rcvbuf (dbmsgcopy) to alloc’ed memory; drops proxyhdr n Lock / unlock db. RWMutex around memcpy n Send “DB_CONVERT” to db. Hdlr n Upon recv’ing last ST 3 block

db. Send. Thread – Download (“IPL”) Sequence n Receives “DB_DOWNLOAD” message from slc. Exec n Sends “ack” to DBEX socket n n n Formats sndbuf. native. SB hto. VMSSup. Hdr(sndbuf. supblk, msg. native. SB) Pre-fixes proxy_hdr Sends (dbdownloadme) n Receives “DB_DBEX_ACK” messages from db. Recv for each ST 0 -3 block received n Sends “ack” to DBEX socket n n uses msg. native. SB. id; hto. VMSSup. Hdr(sndbuf. supblk, msg. native. SB) Pre-fixes proxy_hdr Sends

Topics n DBS Use Cases n Design Considerations n DBS Initialization n Data Structures n Download Sequence n ST 0 Conversion n Receive/Process Setpoints and other messages from Alpha n Process/Send SLC IOC udpates to Alpha n DB I/0 Utilities

db. Hdlr. Thread n db. Hdlr is responsible for n Initializing the DB Globals upon init n Converting the ST 0 data to dictionary format n Releasing DB resources upon exit n Resetting global values upon exit n n n db. Exists download. Event Thread specific – see General Thread Shutdown n Receives DB_CONVERT message to initiate the ST 0 Conversion process n After conversion n Sets db. Exists = T n Signals download. Event n Receives TEST_STOP message n Refer to General Thread Shutdown

db. Hdlr Block Diagram

Secondary Data type Name Data type slc. Subblk_ts secn supn len format sptr width Represents SLC Secondary data type int 1 u int 4 u int 1 u Secondary number Supertype number Word length (V=variable) (I, A, R, Z, S, T) Word offset into supn block 1, 2, 4, 8 bytes; data width Gphash find - gph. Find (“name”) - returns GPHENTRY*: Name Data type GPHENTRY node name pvtid user. Pvt Represents SLC Secondary data type ELLNODE Const char * Void * Link node “primary unit secondary” Pointer to slc. Subblk_ts Not used

db. Hdlr. Thread ST 0 Conversion Sequence n Receives “DB_CONVERT” message from db. Recv n Replace ST 0 num-oriented hash table with string based hash table dictionary n Step 1: n Walk thru all structures in ST 0 to swap words and longs (if arch. is not little endian) n Step 2: n Read PRIMARY. MAP file and make entries into dictionary, each combining n Secondary data from ST 0 n File data – data width, ASCII names n Hash on “primary unit secondary” n Hash on “primary” n Links ALL* units § For primary name error checking n Could also hash on “primary secondary” § For faster secondary name error checking n Structure is optimized for run-time operations

db. Hdlr. Thread ST 0 Conversion Sequence n Correlates file with ST 0 data PRIMARY. MAP: Prim catn Sec QUAD 1 BMON QUAD 1 KTIM secn fmt width # 135 R 4 1 31 T 8 1 … n For each unique primary name in file Make “primary” entry call dbunits. ST 0 (ld_p, catn) if (status OK and ld_p. len>0) for each unit# in list dbget. Ups. ST 0(uptr, catn, unit) link unit in “primary” (for ALL*) for every secondary in file dbget. Sec. ST 0(sptr, uptr, secn) store “primary unit sec” with combined ST 0 & file data; + link sec in unit entry (for ALL*)

db. Hdlr. Thread ST 0 Conversion Sequence n n Delete ST 0 Signal download. Event, sets db. Exists flag n Comments n Translated dblistu. a 38 to C n Uses memory manager (free. List) to manage new slc. Subblk_ts w combined data Optimized for: dblists in thread’s init: n dblist(dblist_p, prim 1, unit 1, sec 1) dblist(dblist_p, prim 2, unit 2, secn) etc. And “on the fly” access, such as operator

Topics n DBS Use Cases n Design Considerations n DBS Initialization n Data Structures n Download Sequence n ST 0 Conversion n Receive/Process Setpoints and other messages from Alpha n Process/Send SLC IOC udpates to Alpha n DB I/0 Utilities

Setpoints from Alpha

db. Recv. Thread – Alpha Updates to slc ioc n Receives Supertype 2 (setpoint) update messages from DBEX n n Sends “DB_DBEX_ACK” msg to db. Send, if req’ed. Lock db. RWMutex Copies super block recv’ed (dbmsgcopy) to slc ioc ST 2 block Unlock db. RWMutex n Other DBEX messages n Receives diagnostic req from DBEX n Sends “DB_DIAG_RPY” to db. Send Q to ack DBEX n Receives “DXDN”, “DXUP” messages from DBEX n n n Resets/sets dbex_up flag Stores database version in global db. Version If DBEX becoming available, send “DB_UPDATE_ALL” message to db. Send Q to update all jobs (TBD)

db. Send. Thread –Alpha Updates to slc ioc n Receives DB_DBEX_ACK messages from db. Recv for Alpha ST 2 updates n Sends “ack” to DBEX socket (same as “IPL” sequence) n Receives DB_DIAG_RPY message from db. Recv n n n n uses msg. native. SB. id VMStoh(sndbuf. diag, msg. diag) Process diagnostic data hto. VMSSup. Hdr(sndbuf. supblk, msg. native. SB) hto. VMS(sndbuf. diag, msg. diag) Pre-fixes proxy_hdr Sends to dbex socket (dbmdiag_sendrpy) n Receives TEST_STOP message n Refer to General Thread Shutdown

Topics n DBS Use Cases n Design Considerations n DBS Initialization n Data Structures n Download Sequence n ST 0 Conversion n Receive/Process Setpoints and other messages from Alpha n Process/Send SLC IOC udpates to Alpha n dblput n DB I/0 Utilities

Writing Data from slc ioc (dblput)

Topics • • • DBS Use Cases Design Considerations DBS Initialization Data Structures Download Sequence – ST 0 Conversion • Receive/Process Setpoints and other messages from Alpha • Process/Send SLC IOC udpates to Alpha – Dblput – SLC IOC DB Updates to Alpha • DB I/0 Utilities

slc ioc DB Updates to Alpha

db. Send. Thread – slc ioc updates to Alpha n Receives DB_UPDATE messages from various “job threads” to send update n n n Lock dbhilo_mutex[job] – prevent further updates for this job Compress job’s list of pending updates, if possible (microdbsendc) Lock db. RWMutex n n memcpy data from ST block indicated by sptr pair into sndbuf (microdbsendb) Unlock db. RWMutex Increment msg seq # (1 -256), write in proxyhdr. user_byte n n Read dbhilo_update[job] sptr pair store seq #, “job id” locally Send sndbuf (already in VMS format) to dbex socket

db. Send. Thread – CTL socket After sending data to dbex socket, loop waiting at CTL socket for Ack from DBEX n Further updates for this job are blocked; other job updates q’ed n Timeout after ~15 sec (less if !dbex_up) n n Wait 1 sec at ctl socket Check for slc. Exec “stop flag” CTL loop exit conditions: 1)Successful Ack (valid msg seq #) n Clear dbhilo_update[job] n If (! dbex_up), set dbex_up flag 2) Invalid msg seq # n If (! dbex_up), set dbex_up flag ; no clear 3) Timeout – no clear 4) Socket error n Set “lost connection flag” ; no clear n Loop waiting for connection n Check for slc. Exec “stop flag” Unlock dbhilo_mutex[job] Signal ack. Event[job] n Upon dbex_up / connection up n Send updates for all jobs

Topics n DBS Use Cases n Design Considerations n DBS Initialization n Data Structures n Download Sequence n ST 0 Conversion n Receive/Process Setpoints and other messages from Alpha n Process/Send SLC IOC udpates to Alpha n DB Input/Output Utilities

DB I/O Utilities n Threads use dballoc(), dbfree(), and db macros for list management n dballoc() / dbfree() use mem management n Re-entrant n Return 4 byte unsigned VMS cond code n Except dbexists returns epics. Boolean n All device args (prim, unit, secn) are 4 chars; blank padded, null-terminated; prim/secn are left justified; unit is right-j? n dblist (dblist_p, “prim”, ”unit”, ”secn”) n Unit can = “ALL*” dblunits (dbdata_p, “prim”) dblget (dbdata_p, dblist) dblput (dbdata_p, dblist) dballoc (dbdata_p, num. Bytes, num. Elements) n dbfree (dbdata_or_list_p) n dbunit 2 string (int. Unit, “unit”) n dbexists (void) n n

DB I/O Utilities – cont. n micro_dbsend(job) n All threads call to send their updates to Alpha n Threads wait for ack or timeout n DB I/O utils wait on download. Event prior to access n n Calling threads are held off until after db is downloaded; timeout in case of failure Shell commands check db. Exists() before calling dbl* utils n Error conditions n A dblist (dblist_p, prim, unit, secn) of either non-existent primary, unit, or secondary returns an error, except if unit = ALL* n A dbunits (dblist_p, prim) of non-existent primary returns an error

DB I/O functionality n Dblist (dblist_p, prim, unit, secn) n Calls gph. Find (“prim unit secn”) n Enters ptr to entry into dblist_p n Dblist (dblist_p, prim, ALL*, secn) n Calls gph. Find (“prim”) n Traverses linked list of prim/unit; for each entry, traverses linked list of secns to find; gets ptr from matching entry’s data n Enters ptr to entry in dblist_p n Dblunits (dbdata_p, prim) n Calls gph. Find (“prim”) n Traverses linked list of prim/unit; for each entry, enters (4 char) unit(s) into dbdata_p

DB I/O functionality n Dblput (dbdata_p, dblist_p) n Converts native typed input dbdata to VMS typed data and stores to dblist location n locks / unlocks db. RWMutex around memcpy of data to ST block n Calls dbhilo_update (job, hi, lo) n n inserts hi, lo offset pairs into dbhilo_update[job] locks / unlocks dbhilo_mutex[job] n Dblget (dbdata_p, dblist_p) n sets/resets db. RWMutex around memcpy from ST block data n Converts retrieved data to native n Inserts in dblist

Shell commands Shells: EPICS iocsh, RTEMS Cexp, vx. Works n All previous dbl* utils can be called from ioc shell after parsing console input. n dbexists() will be checked prior to calling underlying dbl* utility - no waiting for user n dbgettype (“prim”, “secn”) n dbdump (“prim”, “unit”, “secn”, “<filename>”) n dbdumpunits (“prim”, “<filename>”) n dbedit (“prim”, “unit”, “secn”, …) n dbdumphash (“<filename>”) n dbdumpdatabase (“<filename>”)