Grid Computing Infrastructure Software Slides for Grid Computing

Grid Computing Infrastructure Software Slides for Grid Computing: Techniques and Applications by Barry Wilkinson, Chapman & Hall/CRC press, © 2009. Chapter 7, pp. 201 -213. For educational use only. All rights reserved. Sept 13, 2009 7 -1. 1

Grid computing software Gone through several development cycles. Started before grid computing standards became accepted. Clearly need standardized protocols and interfaces for wide adoption of grid computing. 7 -1. 2

Standards Bodies Key players for Internet and World Wide Web: • IETF (Internet Engineering Task Force) – Formed in 1985 for Internet standards inc. previous TCP/IP • W 3 C consortium (Worlds Wide Web Consortium) – founded by Tim Berners-Lee shortly after he conceived World Wide Web in early 1990’s – Standardization of Web-related technologies • OASIS (Organization for the Advancement of Structured Information Standards) • Began as SGML Open in 1993 to promote SGML • Became OASIS in 1998 to focus on structured information standards including newly developed XML • DMTF (Distributed Management Task Force) • Created in 1992 for IT systems management infrastructure 7 -1. 3

Organizations for Grid computing • Open Grid Forum (OGF) – Principal Grid computing body – Began as Grid forum in 1998 – Merged with e. Grid (European Grid) forum and Asia. Pacific Grid community to form Global Grid Form (GGF) in 2001 • GGF meets three times a year around world • GGF 1 (March 2001), GGF 2, GGF 3 … – GGF became OGF in 2006 and continued meetings. • GGF 18 became OGF 18 – OGF 24 met in Sept. 2008 in Singapore. – Meetings grown to a very large endeavour today, with participants across world attending. 7 -1. 4

Standards in Web Services World • • XML introduced (ratified) in 1998 SOAP ratified in 2000 Web services introduced 2000 Subsequently, many standards developed: – WSDL – WS-* where * refers to names of one of many standards – Mostly by W 3 C organization 7 -1. 5

Using web services • It would be natural to use web services for grid computing components. • Provides a easily identifiable interface through WDSL documents. • Has Internet addressing (URLs). 7 -1. 6

(Stateless) Web Services” • “Pure” web services are stateless. • They do not remember information from one invocation to the next. • That can work fine for many web service applications. • They do not need to know what happened with a previous invocation by another client. 7 -1. 7

Simple previous Web service example • Web service Math. Service with a method inc to add 1 to the supplied argument and return the result: Fig 7. 1 7 -1. 8

“Stateful” web service example • With a method to add a value to an accumulating value and return its value: Fig 7. 2 7 -1. 9

Stateful Web service as a front-end to a database Fig 7. 3 7 -1. 10

• In this example, the client accesses a service to perform actions such as access a database. • Key aspect here is that the state is contained in a resource that can be separate from the web service. • Web service becomes a front end to the resource. • Come back to this concept later. 7 -1. 11

Open Grid Services Architecture (OGSA) • Originally proposed by Foster et al. in paper: “The Physiology of the Grid” (see reading list) • Anounced as Grid computing standard at GGF 4 in Feb 2002. • Defines standard mechanisms for creating, naming, and discovering service instances. • Addresses architectural issues relating to interoperable services for grid computing. • Does not give details of implementation. • Requires stateful services but does not say how that will be achieved. 7 -1. 12

Open Grid Service Infrastructure (OGSI) • Introduced in 2002 -3 (final draft June 2003). • First attempt to standardize how stateful Web services will be implemented. • Modifies WDSL to enable state to be specified, using a language called GWSDL (“Grid Web Definition Language”), • Introduced term Grid service. • OGSI included inheritance, addressing mechanism, notifications, and message passing mechanism. • Implemented in Globus toolkit version 3 (GT 3). 7 -1. 13

Open Grid Service Infrastructure (OGSI) • Not found acceptable by community at large because: – It significantly modified pure Web service approach – Required new tools – Too object oriented in approach – Too much specified in one standard. It would be better broken down in a series of specifications 7 -1. 14

WS-Resource Framework (WSRF) • Web service community also working on approaches to stateful Web services. • Web and Grid communities merged on WSResource Framework approach. • Specification developed by OASIS in 2004 – Replace OGSI and makes implementation of a stateful Web service acceptable. • Specifies how to make Web service stateful and other feature, without drifting from the original Web services concept. 7 -1. 15

Grid computing standards Figure from “An ‘Ecosystem’ of Grid Components”, 2004, Grid Research Integration Deployment and Support Center, http: //www-unix. gridscenter. org/r 6/ecosystem/ecology. php 7 -1. 16

From “The Globus Toolkit 4 Programmer’s Tutorial” by Borja Sotomayor. 7 -1. 17

Grid service • Broad meaning -- any service that conforms to interface conventions of a Grid computing infrastructure • Narrow (current) meaning -- a service that conforms to WSRF • Term seems to have lost favor – maybe because better to think of services in a Grid environment simply as regular Web services 7 -1. 18

Stateful Web Services • Obtained in WSRF by having a web service front- end to a stateful “resource. ” 7 -1. 19

Web Service Resource Framework (WSRF) Resource Web Service Client Resource properties Holds information retained between accesses. 7 -1. 20

WSDL file with WSRF • Serves same purpose as WSDL file in web services – to define the service interface. • A significant addition in the WSDL file is to specify the resource. 7 -1. 21

Container WSDL file includes specifying resource Resource Web Service Client Resource properties Holds information retained between accesses. 7 -1. 22

“Stateful” web service example revisited • With a method to add a value to an accumulating value: Fig 7. 4 7 -1. 23

Service Interface • If service implements operations on WSRF resource properties, WSDL will include definitions relating to resource property. • Resource Properties declared in WSDL file because it describes how the properties must be exchanged in XML (between clients, services, etc. ) 7 -1. 24

Contents of Math. xml. . . <!-- RESOURCE PROPERTIES --> <xsd: element name="Value" type="xsd: int"/> <xsd: element name="Last. Op" type="xsd: string"/> <xsd: element name="Math. Resource. Properties"> <xsd: complex. Type> <xsd: sequence> <xsd: element ref="tns: Value" min. Occurs="1" max. Occurs="1"/> <xsd: element ref="tns: Last. Op" min. Occurs="1" max. Occurs="1"/> </xsd: sequence> </xsd: complex. Type> </xsd: element>

Client Code Examples Math. Port. Type math = locator. get. Math. Port. Type. Port(endpoint); math. add(10); update. RP(endpoint, Math. Constanst. RP_VALUE, “ 100”); print. Resource. Properties(math);

Client Code Examples value. RP = math. get. Resource. Property( Math. QNames. RP_VALUE); value = value. RP. get_any()[0]. get. Value(); System. out. println("Value RP: " + value);

Resource Properties from the Command Line C: > wsrf-query -s http//localhost/wsrf/ services/examples/core/rp/Math. Service <ns 1: Math. Resource. Properties xmlns: ns 1=”http: //www. globus. org/namespaces/ examples/math. Service_instance_rp”> <ns 1: Value>100</ns 1: Value> <ns 1: Last. Op>ADDITION</ns 1: Last. Op> </ns 1: Math. Resource. Properties>

Resource Properties from the Command Line C: >wsrf-get-property -s http//localhost/wsrf/services/examples/ core/rp/Math. Service {http: //www. globus. org/namespaces/ examples/math. Service_instance_rp}Value <ns 1: Value xmlns: ns 1=”http. . . ”>100</ns 1: Value>

Other key issues Addressing • WSRF service needs an addressing mechanism that includes access to the resource • Pure web services typically use URIs (URLs) • WSRF service addressing defined in WSaddressing standard – Uses an “endpoint reference” 7 -1. 30

Endpoint Reference EPR Includes both: – Service address (URI) and – Resource identification (called a “key”) 7 -1. 31

Endpoint Reference (EPR) • Service URI + resource identifier (key) Fig 7. 5 7 -1. 32

EPR - Web service without an associated resource <wsa: Endpoint. Reference> <wsa: Address> http: //www. cs. uncc. edu/my. Web. Service. jws </wsa: Address> </wsa: Endpoint. Reference> where <Address> tag is required. prefix wsa is WS-Addressing namepace (http: //www. w 3. org/2005/08/addressing/wsdl) 7 -1. 33

EPR - Web service with an associated resource <wsa: Endpoint. Reference> <wsa: Address> http: //www. cs. uncc. edu/my. Web. Service. jws </wsa: Address> <wsa: Reference. Parameters> <myrp: ID> 234 </myrp: ID> </wsa: Reference. Parameters> </wsa: Endpoint. Reference> Resource ID specified within <Reference. Parameters>. Not specified within WSDL document itself. Separated out and obtained when EPR obtained 7 -1. 34

WS-Addressing Terms Endpoint – the destination where the web service can be accessed. Endpoint reference (EPR) – describes destination: includes destination location as URI, but can have other parameters like a key 7 -2. 35

Endpoint reference, EPR Like all WS-* specs, defined in terms of XML. Defined as complex type. Can contain: • Address (a URI) (required) • Reference Properties • Reference Parameters • Port type • Service name • Policy elements Corresponds to port. Type and service of WSDL document 7 -2. 36

SOAP message WS-Addressing standard requires that contents of: <wsa: Address> and <wsa: Reference. Properties> must appear in SOAP’s header: <soap: Envelop> <soap: Header>. . . <wsa: To>http: //www. cs. uncc. edu/axis/abw/Myservice. jws</wsa: To> <resource. ID>234</resource. ID>. . . </soap: Header> <soap: Body>. . . </soap: Body> </soap: Envelop> 7 -2. 37

WSRF Framework • Actually collection of four specifications: – WS-Resource. Properties • Specifies how resource properties are defined and accessed – WS-Resource. Lifetime • Specifies mechanisms to manage resource lifetimes – WS-Service. Group • Specifies how to group services or WSResources together – WS-Base. Faults • Specifies how to report faults 7 -1. 38

WS-* Standards continued Also: • WS-Notification – Collection of specifications that specifies how configure services as notification producers or consumers • WS-Addressing (where EPR is defined) – Specifies how to address web services. – Provides a way to address a web service/resource pair 7 -1. 39

Concrete example Globus 4. 0 Web Service Assignment This assignment is based upon a GT 4 service described by Sotomayor and Childers (2006). (Textbook describes a generic GT 4 service. ) 7 -1. 40

Assignment Goals • Show stateful WSRF Web services can be created and deployed in Globus 4. 0. • Assignment is to be done on your own computer (or a lab computer). • Requires you to install Globus 4. 0 core and associated software – all available for download. 7 -1. 41

Installing software Will need to have or install: • • JDK 1. 4. 2+ Ant 1. 5. 1+ Python 2. 4+ Globus ws-core-4. 0. + and set environment variables (paths) accordingly as explained in assignment write-up. Enjoy! 7 -1. 42

Testing installation Start Container Command: globus-start-container -nosec specifies “no security”, which simplifies assignment (no need for user certificates) Will show list of deployed services. 7 -1. 43

Standard list of deployed services in core Fig 7. 7 7 -1. 44

Purpose of Service To store an integer called value, which can be acted upon by three methods to: • Add argument a to value, and • Subtract argument a from value • Get value These methods given. Further methods will be implemented. Service is stateful (value retained between accesses). 7 -1. 45

Java Interface for service public interface Math { public void add(int a); public void subtract(int a); public int get. Value. RP(); } 7 -1. 46

Resource Properties In code provided, actually two resource properties: • Value -- an integer acted upon by the operations: – add – sub – get. Value. RP • “Last operation performed” -- a string holding name of last operation done, “ADDITION” or “SUBTRACTION”, which is not used in assignment. 7 -1. 47

Resource Properties Resource Math Web Service Client Resource properties “value” (integer) “last operation performed” (string) 7 -1. 48

Steps in Assignment • Preliminary set-up – Install GT 4 and associated software • Download provided files: – – WSDL service interface file Stateful Math web service code (in Java) WSDD Deployment Descriptor file Client code (java) to exercise service • Build Math service – Package all files into an archive file (. gar) • Deploy Math service into GT 4 container • Write and compile the client • Start container and execute client. Easy! 7 -1. 49

Final Step in Assignment • Add functionality to service – Add multiple operation to Math service – Requires you to modify WSDL (by hand), service code, and client code. – For that you will need to understand the files – Assignment Appendix gives details 7 -1. 50

WSDL file • Serves same purpose as WSDL file in Web services – to define the service interface. • A significant addition in the WSDL file is to specify the resource. 7 -1. 51

Globus-specific features of WSDL • Resource properties – specified in port. Type attribute wsrp: Resource. Properties • WSDL preprocessor used to include WSRF definitions (port. Type attribute wsdlpp: extends). • Bindings (how abstract interface maps to concrete protocol messages) – not needed because automatically inserted by GT 4 when built. 7 -1. 52

WSDL file Name of service being implemented <definitions name="Math. Service” target. Namespace=“http: //www. globus. org/namespaces/ examples/core/Math. Service_instance” xmlns="http: //schemas. xmlsoap. org/wsdl/" xmlns: tns=http: //www. globus. org/namespaces/ Namespaces examples/core/Math. Service_instance xmlns: wsdl="http: //schemas. xmlsoap. org/wsdl/" xmlns: wsrp="http: //docs. oasis-open. org/wsrf/2004/06/ wsrf-WS-Resource. Properties-1. 2 -draft-01. xsd" xmlns: wsrpw="http: //docs. oasis-open. org/wsrf/2004/06/ wsrf-WS-Resource. Properties-1. 2 -draft-01. wsdl" xmlns: wsdlpp="http: //www. globus. org/namespaces/2004/10/ WSDLPreprocessor" xmlns: xsd="http: //www. w 3. org/2001/XMLSchema"> <wsdl: import namespace= "http: //docs. oasis-open. org/wsrf/2004/06/ wsrf-WS-Resource. Properties-1. 2 -draft-01. wsdl" location=". . /wsrf/properties/WS-Resource. Properties. wsdl" /> 7 -1. 53

<types> <xsd: schema target. Namespace="http: //www. globus. org/namespaces/ examples/core/Math. Service_instance" xmlns: tns="http: //www. globus. org/namespaces/examples/core/ Math. Service_instance" xmlns: xsd="http: //www. w 3. org/2001/XMLSchema"> add <!-- REQUESTS AND RESPONSES --> method, <xsd: element name="add" type="xsd: int"/> argument <xsd: element name="add. Response"> int, returns <xsd: complex. Type/> a complex </xsd: element> type <xsd: element name="subtract" type="xsd: int"/> <xsd: element name="subtract. Response"> <xsd: complex. Type/> </xsd: element> <xsd: element name="get. Value. RP"> <xsd: complex. Type/> </xsd: element> <xsd: element name="get. Value. RPResponse" type="xsd: int"/> 7 -1. 54

<!-- RESOURCE PROPERTIES --> <xsd: element name="Value" type="xsd: int"/> <xsd: element name="Last. Op“ type="xsd: string"/> <xsd: element name="Math. Resource. Properties"> <xsd: complex. Type> <xsd: sequence> <xsd: element ref="tns: Value“ min. Occurs="1“ max. Occurs="1"/> <xsd: element ref="tns: Last. Op“ min. Occurs="1“ max. Occurs="1"/> </xsd: sequence> </xsd: complex. Type> </xsd: element> </xsd: schema> </types> 7 -1. 55

<!- M E S S A G E S --> Input and <message name="Add. Input. Message"> output <part name="parameters" element="tns: add"/> message for </message> add method <message name="Add. Output. Message"> <part name="parameters" element="tns: add. Response"/> </message> <message name="Subtract. Input. Message"> <part name="parameters" element="tns: subtract"/> </message> <message name="Subtract. Output. Message"> <part name="parameters" element="tns: subtract. Response"/> </message> <message name="Get. Value. RPInput. Message"> <part name="parameters" element="tns: get. Value. RP"/> </message> <message name="Get. Value. RPOutput. Message"> <part name="parameters" element="tns: get. Value. RPResponse"/> </message> 7 -1. 56

<!-- P O R T T Y P E --> <port. Type name="Math. Port. Type" wsdlpp: extends="wsrpw: Get. Resource. Property" wsrp: Resource. Properties="tns: Math. Resource. Properties"> <operation name="add"> <input message="tns: Add. Input. Message"/> <output message="tns: Add. Output. Message"/> </operation> <operation name="subtract"> <input message="tns: Subtract. Input. Message"/> <output message="tns: Subtract. Output. Message"/> </operation> <operation name="get. Value. RP"> <input message="tns: Get. Value. RPInput. Message"/> <output message="tns: Get. Value. RPOutput. Message"/> </operation> </port. Type> </definitions> 7 -1. 57

Service Code The code has two major parts: • Resource properties • Service code (methods) which are combined into one file for this assignment. 7 -1. 58

Service – Resource Properties public class Math. Service implements Resource, Resource. Properties { private Resource. Property. Set prop. Set; /* Resource Property set */ private int value; Resource properties private String last. Op; public Math. Service() throws Remote. Exception { /* RP Constructor */ this. prop. Set = new Simple. Resource. Property. Set( Math. QNames. RESOURCE_PROPERTIES); /* Create RP set */ try { /* Initialize the RP's */ Resource. Property value. RP = new Reflection. Resource. Property( Math. QNames. RP_VALUE, "Value", this); this. prop. Set. add(value. RP); Resource set. Value(0); Property Resource. Property last. Op. RP = new code Reflection. Resource. Property( Math. QNames. RP_LASTOP, "Last. Op", this); this. prop. Set. add(last. Op. RP); set. Last. Op("NONE"); } catch (Exception e) { throw new Runtime. Exception(e. get. Message()); } 7 -1. 59 }

Service – Resource Properties methods /* Get/Setters for the RPs */ public int get. Value() { return value; } public void set. Value(int value) { this. value = value; } public String get. Last. Op() { return last. Op; } public void set. Last. Op(String last. Op) { this. last. Op = last. Op; } 7 -1. 60

Service code - methods. . /* Remotely-accessible operations */ public Add. Response add(int a) throws Remote. Exception { value += a; add last. Op = "ADDITION"; return new Add. Response(); method } public Subtract. Response subtract(int a) throws Remote. Exception { value -= a; last. Op = "SUBTRACTION"; return new Subtract. Response(); } public int get. Value. RP(Get. Value. RP params) throws Remote. Exception { return value; }. . 7 -1. 61

Deploying a GT 4 service • GT 4 container uses Apache Axis • Basic steps for deploying a service similar in concept to that described for Apache Axis. 7 -1. 62

Deployment files • server-config. wsdd (Web Service Deployment Descriptor) - contains information about the web service. • jndi-config. xml (JNDI configuration file) - contains information about the resource management. 7 -1. 63

<? xml version="1. 0" encoding="UTF-8"? > <deployment name="default. Server. Config" xmlns="http: //xml. apache. org/axis/wsdd/" xmlns: java="http: //xml. apache. org/axis/wsdd/providers/java" specifies xmlns: xsd="http: //www. w 3. org/2001/XMLSchema"> where service <service name="examples/core/first/Math. Service" will be provider="Handler" use="literal" style="document"> located. <parameter name="class. Name"value="org. globus. examples. services. core. first. impl. Math. Service"/> <wsdl. File>share/schema/examples/Math. Service_instance/ Math_service. wsdl </wsdl. File> <parameter name="allowed. Methods" value="*"/> <parameter name="handler. Class“ value="org. globus. axis. providers. RPCProvider"/> <parameter name="scope" value="Application"/> <parameter name="providers" value="Get. RPProvider"/> <parameter name="load. On. Startup" value="true"/> </service> 7 -1. 64 </deployment> WSDD file

ant (Another Neat Tool) • A build tool used for building service – forming package of files for deployment – a gar file (grid archive file) • ant similar to make program but dependencies specified using XML configuration files. • Windows version of assignment uses Python. 7 -1. 65

Generating GAR file with Ant From http: //gdp. globus. org/gt 4 tutorial/multiplehtml/ch 03 s 04. html 7 -1. 66

GT 4 build command globus-build-service. sh first where first is a file that contains required bash and ant files. Windows Python version globus-build-service. py first 7 -1. 67

Deploying Service Deployment done using GT 4 command: globus-deploy-gar gar_file using the gar file created by globus-buildservice. 7 -1. 68

Start Container Command: globus-start-container -nosec Math. Service will be listed as one of the services that are available once the container has started. 7 -1. 69

GT 4 container Mathservice 7 -1. 70

Java client to invoke service public class Client { public static void main(String[] args) { Math. Service. Addressing. Locator locator = new Math. Service. Addressing. Locator(); try { String service. URI = args[0]; Endpoint. Reference. Type endpoint = new Endpoint. Reference. Type(); endpoint. set. Address(new Address(service. URI)); //service endpt ref. Math. Port. Type math; math = locator. get. Math. Port. Type. Port(endpoint); // Get Port. Type math. add(10); // Perform an addition math. add(5); // Perform another addition System. out. println("Current value: " + math. get. Value. RP(new Get. Value. RP())); // Access value math. subtract(5); // Perform a subtraction System. out. println("Current value: " + math. get. Value. RP(new Get. Value. RP())); // Access value } catch (Exception e) { e. print. Stack. Trace(); } } 7 -1. 71 }

Set CLASSPATH environment variable To set CLASSPATH (Windows), use provided batch file: %GLOBUS_LOCATION%etcglobus-devel-env. bat Then compile. 7 -1. 72

Execute Client Execute client: javac -classpath buildclassesorgglobusexamplesservicescor efirstimpl: %CLASSPATH% orgglobusexamplesclientsMath. Service_insta nceClient. java You will see the following result (hopefully): Current value: 15 Current value: 10 7 -1. 73

Undeploy sevice Deploy service in preparation to modifying it. Command: globus-undeploy-gar with named gar file. 7 -1. 74

Extend Functionality of Service Add a multiply method to your Math Service. Repeat all the steps to test it. 7 -1. 75

Acknowledgment This assignment is derived from the book: “Globus Toolkit 4 Programming Java Services” by Borja Sotomayor and Lisa Childers, Morgan Kaufmann, 2006 see: http: //gdp. globus. org/gt 4 -tutorial/ for on-line version. 7 -1. 76

Multiple-choice questions 7 -1. 77

What is a stateless Web service? (a) A Web service that cannot remember prior events (b) A Web service without local variables (c) A Web service that is not associated with a particular state in the country (d) A Web service that can remember prior events SAQ 7 -1. 78

What is a non-transient service? (a) An instance of a service that does not receive data (b) An instance of a service that outlives its client (c) An instance of a service that generates stateful data (d) An instance of a service that generates stable data SAQ 7 -2 7 -1. 79

What is the Open Grid Services Architecture (OGSA)? (a) A standard for defining a structure for interoperable Grid computing services but not its implementation (b) A standard for defining a structure for interoperable Grid computing services that includes its implementation (c) An open source implementation of Grid computing middleware (d) A Grid computing environment that is open to all SAQ 7 -3 7 -1. 80

What is the WS-Resource Framework (WSRF)? (a) A way of connecting a Web service to a resource without specifying its implementation (b) A framework for Web service reliability (c) A standard that provides for Web services embedded within a computing resource (d) A standard that provides for stateful Web services SAQ 7 -5 7 -1. 81

When one issues the command: globus-start-container -p 8081 -nosec what does one see? (a) 10 pages of errors messages (b) Nothing because the -p option with port 8081 cannot be used with -nosec (no security) (c) A list of services deployed by all users in the container (d) A message that states whether port 8081 can be used without Globus security (e) A list of the services that only you have deployed in the container SAQ 7 -6 7 -1. 82

Name one basic difference between a Web service and a WSRF Grid service. (a) No differences (b) A Grid service is accessed from a Grid (c) A Grid service is designed to be stateful (d) The connections between Web services are in the form of a Web and the connections between Grid services are in the form of a Grid SAQ 7 -8 7 -1. 83

Questions 7 -1. 84