Common Services Implementation PDR March 21 2005 Tucson

Common Services Implementation PDR March 21, 2005 Tucson, AZ

Planned Releases* *Common Service Software Release Roadmap (TN-0026 Rev. A) Release 1. 3 Release 1. 2 Release 1. 1 Release 1. 0 Beta Panguitch 1 -3 Sacramento 1 -2 Martir 1 -3 Alpha

Progress Report • Martir 2 is nearly complete – Update code to reflect documentation – Further timing/robustness testing – Bug fixes and cleanup

Infrastructure In Place • CVS repository • Software development tree – Makefile structure – Java build/release/test environment – 3 rd party tools integrated • Document tree • Bug tracking facility

ASDT Building a ATST Software Development Tree $ cvs co -d atst $ cd atst $ export ATST=`pwd` $ cp admin/site. config. template admin/site. config $ vi admin/site. config # set site-specific variable definitions $ admin/create. Devel --make-all

Completed Java Work • Initial implementations and tests for – – – Container Component Toolbox Service Access Helpers Service Tool Helpers Attribute Tables

Remaining Java Work • Initial implementations for – – Container Manager Component Manager Controller Device • Complete interface support for – Container – Controller – Device

Remaining Java Work • Data Structures – Command/response – Configurations • Simulated controllers/devices • Tests – Container deployment – Command lifecycle

Remaining Design Issues • • • Requirements for a notification service Controller/Device specifications Container/Component lifecycle management Callbacks in commands Exception support

Anticipated C++ Work • Infrastructure – Makefiles for target OS – Integration of multiple OS’s in the ASDT • Software – Port Common Services Java classes to C++ – Realistic device simulators • Technical Issues – Avoiding name collisions – Providing support for RTOS’s

Anticipated C++ Work • Testing – – – Infrastructure Timing/robustness Command lifecycle Container/Component lifecycles Interoperability

Postgre. SQL • • • Object-relational database management system Highly scalable SQL compliant Open source 15 year development history Support for Unix/Linux, Windows

Postgre. SQL • APIs for C++, Java, Python, Perl, Tcl, … • Many commercial and open source products • Used in SOLIS – DB administration experience – Java DB servers with CORBA interface too slow – Direct DB connections perform better

Performance Tests • Log service tools – Simple – Buffered – over a set time interval or size • • Postgre. SQL version 7. 4. 2 Java JDK 1. 4. 2 Custom JDBC that optimizes block inserts 100, 000 messages sent by test Component

Computers • maunder – file server – Dual CPU (2. 8 GHz) Xeon – RAID-5 disk array (log database + system files) – Gigabit Ethernet • weaver – desktop computer – Dual CPU (1. 8 GHz) Athlon – IDE disk (log database + system files) – Gigabit Ethernet • tapestry – home computer – Dual CPU (2. 0 GHz) Athlon – IDE disk (log database) – SCSI disk (system files)

messages/sec Local Database

messages/sec Remote Database

ICE • Internet Communications Engine – Distributed object programming technology (middleware) – Highly-scalable – Fast – Versatile – Configurable – Open source

Supported Systems

ICE Trivia • • Zero. C staff includes many CORBA gurus Company is growing Customers – Hewlett-Packard, Boeing, … Regular release updates (quarterly)

ICE vs. CORBA • • No “design by committee” Leaner, yet more flexible specification language More language mappings Extensible metadata facility for Java No Any type Simplified core Ice protocol – More efficient than IIOP – Supports data compression and batched requests

ICE vs. CORBA • • Security was not an afterthought C++ mapping is simpler and is based on STL Scalability simplified Versioning for persistent data Tool for distributing software updates Typed event service Facets – interface inheritance and aggregation

ICE vs. CORBA • Asynchronous message invocation (AMI) – Present in few CORBA implementations – Request a callback with server request – Client thread not blocked • Asynchronous message dispatch (AMD) – No CORBA equivalent – Server-side equivalent of AMI • Support for UDP

Performance Tests • • • Averages 4 runs of 100, 000 messages Each message is an array of bytes Java client/server Synchronous message invocations No return data TCP with “one-way” and “two-way” proxies

Performance Tests • Computers – maunder – file server • Dual CPU (2. 8 GHz) Xeon – dorado – desktop computer • Dual CPU (2. 8 GHz) Xeon • 100 Mb Ethernet • Ice 1. 5. 1 • Java JDK 1. 4. 1

Messages/second Throughput Message size (bytes)

Messages/second Throughput Message size (bytes)

Latency Measurements • Zero. C (estimates) vs. ATST – dorado/maunder on gigabit Ethernet – Java SDK 1. 4. 2 – Ice 2. 0. 0 • Demarey, et al. – Benchmarking the Round-Trip Latency of Various Java-Based Middleware Platforms – Java SDK 1. 4. 2 – ICE 1. 5. 1 – Single CPU (2. 0 GHz) Intel Pentium 4 – Stable testing environment

Microseconds Round-Trip Latency *Benchmarking the Round-Trip Latency of Various Java-Base Middleware Platforms

Containers • Two basic functions – Manage the lifecycle of Components – Provide access to services • Custom class loader avoids namespace collisions • Tool. Box provides access to services • Tool. Box. Loader loads tools into the Tool. Box

Component. Class. Loader • Extends java. lang. Class. Loader • Loads a component and all of its referenced classes • Each component is loaded using a different instance of Component. Class. Loader • Interfaces are delegated

Component. Class. Loader Constructors Component. Class. Loader() Component. Class. Loader(java. lang. Class. Loader parent, java. lang. String search. Path) Component. Class. Loader(java. lang. String search. Path)

Component. Class. Loader Public Methods java. lang. Class load. Class(java. lang. String name) throws java. lang. Class. Not. Found. Exception java. lang. Class load. Class(java. lang. String name, boolean resolve) throws java. lang. Class. Not. Found. Exception

Component. Class. Loader Protected Methods java. lang. Class find. Class(java. lang. String class. Name) throws java. lang. Class. Not. Found. Exception

Tool. Box • • • Provides access to ATST services One is assigned to each Component Loaded with “tools” by a Tool. Box. Loader “tools” provide access to the services Holds component information Convenience class used within Common Services software

Abstract. Tool. Box. Loader Interface ITool. Box. Loader Data protected ITool. Box. Admin toolbox Protected methods Object load. In. Component. Namespace(String class. Name) void load. Tool. Box(ITool. Box. Admin new. Tool. Box) abstract void populate()

ITool. Box. Admin Methods used in following example void set. Connection. Service(IConnection. Service. Tool tool) void set. Health. Service(IHealth. Service. Tool tool) void set. Event. Service(IEvent. Service. Tool tool) void set. Log. Service(ILog. Service. Tool tool)

Ice. Tool. Box. Loader Implement populate() public class Ice. Tool. Box. Loader extends Abstract. Tool. Box. Loader { protected void populate() { // Load in service access tools … } private static ILog. Service. Tool log. Tool = null; … (other shared tools) }

Ice. Tool. Box. Loader Load the log service tool first if (null == log. Tool) { log. Tool = new Print. Log. Service. Tool(); log. Tool. chain. Tool(new Buffered. Log. Service. Tool()); } toolbox. set. Log. Service(log. Tool);

Ice. Tool. Box. Loader Load the ICE service tools if (null == cs. Tool) { cs. Tool = new Ice. Connection. Service. Tool(); } toolbox. set. Connection. Service(cs. Tool); if (null == es. Tool) { es. Tool = new Ice. Event. Service. Tool(); } toolbox. set. Event. Service(es. Tool);

Ice. Tool. Box. Loader Load an unshared tool IHealth. Service. Tool health. Service. Tool = (IHealth. Service. Tool) load. In. Component. Namespace( "atst. cs. services. health. Logging. Health. Service. Tool"); toolbox. set. Health. Service(health. Service. Tool);

Connection Service • Provides “connections” between components – Transparent to applications – Language/platform independent • Ice. Pack – Similar to the CORBA naming service – Ice. Connection. Service. Tool implements • register/unregister – handled by Container • connect/disconnect – Component developers use the App class

Connection Service Example // Get a connection atst. cs. interfaces. IRemote. Device mount = (atst. cs. interfaces. IRemote. Device) App. connect(“atst. tcs. mount”); // Make calls as if the object were local mount. set(params); // params is an Attribute. Table // Release a connection App. disconnect(“atst. tcs. mount”);

Event Service • Provides asynchronous/decoupled messaging – Publishers send “events” to a named event stream – Subscribers register to receive events – Connection is transparent • Ice. Storm – publish/subscribe service – Ice. Event. Service. Tool implements • subscribe/unsubscribe • post – Component developers use the Event class

Posting Events Example // Long integer Event. post(“atst. sys. sub. motor 1. pos”, -345782); Other types Event. post(String name, double value) Event. post(String name, String value) Event. post(IAttribute. Table value)

Receiving Events Create a “listener” public class Event. Listener extends atst. cs. services. event. Event. Callback. Adapter { public void callback(String source) { try{ String value = get. String() } catch(Illegal. Conversion. Exception { // do something }}}

Receiving Events Subscribe Event. Listener string. Listener = new Event. Listener(); Event. subscribe(“atst. somedevice. status”, string. Listener); // Do other work while waiting for status updates … // Unsubscribe when done listening Event. unsubscribe(“atst. somedevice. status”, string. Listener);

Ice. Storm Performance • Test – Attribute. Table containing a 128 byte message – 2 million events sent per run – Throughput computed after 200, 000 received • Configuration – maunder, dorado, gigabit Ethernet – ICE 2. 0. 0, Java SDK 1. 4. 2

Ice. Storm Performance Mean throughput over 6000 events/second

Log Service • Very efficient – Direct connections using JDBC – Optional buffering capability – Optional console printing capability • Many predefined categories • Message automatically tagged with timestamp, source and category • Component developers use the Log class

Log Class Methods boolean Int void Log. is. Enabled() Log. is. Debuggable(String category, int level) get. Debug. Level(String category) note(String category, String message) warn(String category, String message) severe(String category, String message) debug(String category, int level String message)

Log Class Methods void String note(String message) warn(String message) severe(String message) debug(int level String message) get. Stack. Trace(Throwable ex)

Log Service Examples // Warning with optional stack trace Log. warn(“startup mode unspecified – “ +Log. get. Stack. Trace()); // Error – stack trace automatic with severe messages Log. severe(LOG. INIT, “servo controller is not responding”); // Debug message Log. debug(2, “Initialization complete”);

Health Service • Component health is monitored by the Container • Component developers must implement String health. Check() Returns: Health. GOOD Health. BAD Health. ILL Health. UNKNOWN

Health Service Example for 2 D motor device public class Motor 2 D extends Device { String health. Check() { String motor 1 Health = Health. get. Health(“atst. tcs. mount. ra”); String motor 2 Health = Health. get. Health(“atst. tcs. mount. dec”); if(Health. GOOD. equals(motor 1 Health) && Health. GOOD. equals(motor 2 Health)) return Health. GOOD; else … }}

Minor Services • Archive Service – Provides high performance archiving of Attributes – Automatically records timestamp and source along with data • Example // Temperatures contained in (String name, String[] values) Log. note(“archiving system xyz temperatures”); Archive. archive(temperatures);

Minor Services • Property Service – Maintains metadata about Attributes in a persistent store – Accessed through the atst. cs. services. Property class • Method IAttribute. Table get. Meta. Data(String attribute. Name)

Minor Services • Monitor Service – Provides support for monitoring key values within a Component – Uses the Property Service to obtain metadata for the “monitored” attribute – May use the event service to publish an event when the monitored value changes by some prescribed amount

ID Service • Used internally to the ATST common services • Provides identification strings that are guaranteed to be unique across all of ATST • Monotonically increasing • Not necessary sequential • Very fast …

ID Service Example // Get a unique name String object. Adapter = atst. cs. services. Id. DB. get. Id(“object. Adapter"); Generated ID object. Adapter. 00004290. 0000001

ID Service • User supplied prefix + generated ID • Generated ID – prefix - unique ID from the atst. iddb database – suffix – local counter

Component • • Foundation for all applications Lifecycle is managed by Container No main function Developers inject functionality by providing implementations of various methods • Most applications will extend Controller/Device

Component Methods to be overridden void _init(IAttribute. Table args) void _startup(IAttribute. Table args) void _shutdown() void _remove() String health. Check() IAttribute. Table _get(IAttribute. Table params) void _set(IAttribute. Table params)

Controller • Multi-threaded Component that manages Configurations • Implements the command/action/response model – command thread – scheduler thread – action thread(s) • Maintains a queue of configurations to be executed

Controller Methods to be overridden void _start(atst. cs. interfaces. IConfiguration config) void _stop(String config. ID) void _pause(String config. ID) void _resume(String config. ID) void _action(atst. cs. interfaces. IConfiguration config)

Controller Attributes for configuring a controller threadmodel {FIXED, GROWABLE} numthreads fullthreadaction {QUEUE, REJECT} activethreads schedlist timeout

Controller Configuration attributes of interest timeout starttime stoptime.

Controller Generated events atst. cs. af. controller. Config. State. SCHEDULED atst. cs. af. controller. Config. State. RUNNING atst. cs. af. controller. Config. State. ABORTED atst. cs. af. controller. Config. State. DONE

Controller Callback methods to be overridden void _done(atst. cs. interfaces. IConfiguration config) void _abort(atst. cs. interfaces. IConfiguration config)

Device • Meant for use with hardware • Adds a state machine – – IDLE BUSY PAUSED FAULT

Remaining Work • Controllers/Devices – Develop base Java implementations – Extend to create simulated devices • Use for realistic lifecycle testing