A RESTful Web Service Interface to the ATLAS

A RESTful Web Service Interface to the ATLAS COOL Database Shaun Roe CHEP'09 Prague A RESTful Web Service. . . (Shaun Roe, Atlas) 1

http: //lcgapp. cern. ch/doxygen/COOL_2_6_0/doxygen/ht COOL (see talk by A. Valassi, this conference) COOL: A database schema and API designed to be technology neutral (Oracle/My. SQL/SQLite) Data structure is hierarchical: Data entry is to a particular folder, folder channel and with a specific Interval Of Validity (IOV) IOV and tag =>These form a unique set of coordinates for the datum or resource Originally in C++, its methods are exposed in Python, High Voltage values from the in the ‘Py. Cool’ module. Detector Control System of Atlas Semi. Conductor Tracker CHEP'09 Prague A RESTful Web Service. . . (Shaun Roe, Atlas) 2

REST POST: Create resource GET: Retrieve resource PUT: Update DELETE: Delete resource REpresentationa l State Transfer e. g. sending a form e. g. A browser normally GETs a web page Roy Fielding’s dissertation: RESTful A resource is defined by a unique URL noun What you ‘do’ to that resource is defined by the http method: methodverb ‘ 404 Not Found…’ The reply is given via http codes. The format of the answer that you ‘Accept: text/html’ want can be further defined by http headers CHEP'09 Prague A RESTful Web Service. . . (Shaun Roe, Atlas) 3

Motivation Rewind to 2006… People want data from Cool on the web! Solutions: • Use their own SQL from PHP/Python… • Use a shell script to set up environment, then run the Cool api behind some custom code • Use a cron job to generate local html files. . and people want to insert data from the web, too. CHEP'09 Prague A RESTful Web Service. . . (Shaun Roe, Atlas) 4

Aim: Universal Web Service Identify resources with url e. g. http: //server/database/schema/dbname/some/folder/path/timespan/0100/channels/23 ‘Accept: text/xml’ Use xml as the data transport format. ‘GET’ retrieves the values ‘POST’ creates the channel in a folder ‘PUT’ updates the channel ‘DELETE’ deletes the channel from the folder Multi-channel uploads/downloads should be possible. DB Modification should be protected. Additional resources to be matched to a url: Database nodelist, folder description, folder payload definition, list of channels in a folder, list of tags; it should be possible to create and delete databases, folders. A RESTful Web Service. . . (Shaun Roe, CHEP'09 Prague Atlas) 5

Data format: XML <channels server="ATLAS_COOLPROD" schema="ATLAS_COOLOFL_DCS" dbname="COMP 200" folder="SCT/DCS/CHANSTAT" since="12262716000000" until="122627412500000" tag="" version="single”> <channel id="138950656" since="122627058100000" until="122627412500000"> <value name="LVCHSTAT_RECV">193</value> <value name="STATE">17</value> </channel> <channel id="138950656" since="122627412500000" until="122627772800000"> <value name="LVCHSTAT_RECV">193</value> <value name="STATE">17</value> </channel> XML Schema Description </channels> Java. Script Object Notation Definition of an XSD schema allows validation of data format on upload, or type-ahead and documentation tooltips in an editor: Namespaces ? What about JSON ? (see later) (o. Xygen editor) CHEP'09 Prague A RESTful Web Service. . . (Shaun Roe, Atlas) 6

Implementation: Cherry. Pywww. cherrypy. org Cherry. Py is a Python application server. It is very easy to install and use. The ‘Routes’ module (from Ruby) allows automatic or manual mapping of Python classes and methods to URLs: d. connect('folder_payload', '/: server/: schema/: dbname/*folder. Path/payload', controller=rest. Con troller, action='folder. Payload’) maps this http: //myserver. cern. ch/cooldb/ATLAS_COOLPROD/ATLAS_COOL_SCT/DCSP 200/SCT/DCS/HV/ topayload this def folder. Payload(self, server, schema, dbname, folder. Path): The http headers and the methods are also available and can be used to route the URL to the appropriate Python method; other modules for security, caching etc are available. CHEP'09 Prague A RESTful Web Service. . . (Shaun Roe, Atlas) 7

http: //curl. haxx. se/ Examples, curl Command line examples Create a folder: curl 'http: //server/DEVDB 10/ATLAS_SCT_COMMCOND_DEV/ACALTEST' -u user: pwd -F definition=@folderdef. xml -X POST Create channels: curl 'http: //server/DEVDB 10/ATLAS_SCT_COMMCOND_DEV/ACALTEST/tes t/gain/channels' -u user: pwd -F definition=@calib. xml -X POST Insert values: curl 'http: //server/DEVDB 10/ATLAS_SCT_COMMCOND_DEV/ACALTEST/tes t/gain/timespan/0 -9223372036854775807/channels’ -u user: pwd -F payload=@calib. xml -X PUT c. URL (and libcurl, to use from C, C++, Perl) are standard installations. Other languages similar libraries: PHP – curl, A RESTful Webhave Service. . . (Shaun Roe, CHEP'09 Prague Atlas) … all are standard Java – java. net. url, Python – urllib 2 8

‘In the wild’ examples /** * @author formica */ public class Cherry. Py. Client { Java private String name = "user. name"; private String psswd = "mypass"; private String readnode. Url = "http: //server/ATLAS_SCT_COMMCOND_DEV/ROETEST/Branch/Trial 1/payload"; private String writenode. Url = http: //server/ATLAS_SCT_COMMCOND_DEV/ROETEST/Branch/Trial 1/timespan/0 -100; public String cool. Get. Cherry. Py() { try { URL url = new URL(this. readnode. Url); java. net. URLConnection conn = url. open. Connection(); Input. Stream is = (Input. Stream) conn. get. Input. Stream(); Buffered. Reader in = new Buffered. Reader(new Input. Stream. Reader(is)); String input. Line = in. read. Line(); is. close(); return input. Line; } catch (Exception e) { PHP e. print. Stack. Trace(); } $header=array('Accept: text/xml'); return "none"; // } $ch = curl_init(); Python http. Headers={'Accept': 'text/xml'} text. Data=None request=urllib 2. Request(url, text. Data, http. Headers) u=urllib 2. urlopen(request) xml. Doc. String = u. read() return xml. Doc. String CHEP'09 Prague curl_setopt($ch, CURLOPT_HTTPHEADER, $header); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $format=requested_format(); $http_result = curl_exec($ch); if ($format != 'text/xml'){ $dom=domxml_open_mem($http_result); } curl_close($ch); A RESTful Web Service. . . (Shaun Roe, Atlas) 9

Example: Run List Search for runs matching criteria, display results; Uses an ‘Ajax’ request to get the Cherry XML is inserted directly in the XHTML page, and associated with an XSLT stylesheet to format it as a table e. Xtensible Stylesheet Language for Transformations Client-side CHEP'09 Prague Local server side A RESTful Web Service. . . (Shaun Roe, Atlas) 10

Plans, conclusion Use of Cherry. Py allowed rapid development of a web service interface to Cool which greatly simplified conditions access and adheres to ‘RESTful’ principles. It was used during 2008 as the engine for the ‘run list’ query page, for various dcs retrieval services, and for insertion of detector status and calibration data; its adoption is growing as a part of web information collations in the detector subsystems. Future extensions include extra methods for tag viewing, and more flexible output format (e. g. JSON, zipped format…) CHEP'09 Prague A RESTful Web Service. . . (Shaun Roe, Atlas) 11