Integrating Peer to Peer Networks and Web Services

Integrating Peer to Peer Networks and Web Services with the Grid Part of GGF 4 Tutorial Toronto February 17 2002 Grid Programming with Web Services, Software Components and Portal Development Tools PTLIU Laboratory for Community Grids Geoffrey Fox, Marlon Pierce, Shrideep Pallickara, Choonhan Youn Computer Science, Informatics, Physics Indiana University, Bloomington IN 47404 http: //grids. ucs. indiana. edu/ptliupages/presentations/ggf 4 feb 02 gcf@indiana. edu 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 1

Classic Multi-Tier Architecture Client Tier IIOP Middle Tier Servers Back-end Tier Raw Resources Object Broker Relational Database HTTP Web Server RMI Specialized Java Server SOAP + WSDL 11/2/2020 Web Service uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" “Flat” File System Old and New Useful Backend Systems 2

Object Web View of the World I n “Everything electronic” is a resource • • n n Computers Programs Data (from sensors to this presentation to email to databases) People “Everything electronic” is a distributed object “All External Interfaces” are defined in XML whether this is a property (data-structure) or a method (service, function, subroutine) • We can assume that a data-structure property has getproperty() and setproperty(value) methods to act as interface • All resources have interfaces n All resources are linked by messages with format and transport protocol to be discussed but as interfaces defined in XML, message structure must be specifiable in XML 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 3

Object Web View of the World II n All resources have a URI which can be a URL (location) to URN (property tag) • I could be ssn: //abc/de/fghk or my homepage http: //www. w 3. org/Addressing/ 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 4

The 3 Critical XML Interfaces n n n Data (sort of dumb) Information User Rendering Raw Data Resources (Virtual) XML Data Interface Server Processing Information/Knowledge (Virtual) XML Information Interface Render to XML Display Format (Virtual) XML Rendering Interface Clients 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 5

What is a Virtual XML Interface n n We can specify interfaces in XML but we are not required to implement in XML. Example 1: We aren’t likely to change syntax of mail Reply-to: Geoffrey Fox <gcf@indiana. edu> To: Geoffrey Fox <gcf@grids. ucs. indiana. edu> Subject: A Test for Tutorial A simple mail message Geoffrey Fox gcf@indiana. edu FAX 8128567972 Phones Cell 315 -254 -6387 Home 812323919 n n But we could specify and indeed store in XML with transport done using conventional SMTP. So conventional mail is easy to give a virtual XML specification for with name: value becoming <name>value</name> but this is not only way 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 6

Mail in XML n <mailasxml uri=“gxos: //mail/users/gcf/sent/2002/february/290” > <smtpheaders> <reply-to email=“gcf@indiana. edu” >Geoffrey Fox</reply-to> <to email=“gcf@grids. ucs. indiana. edu” >Geoffrey Fox</to> <subject>A Test for Tutorial</subject> </smtpheaders> <smtpbody> <message whitespace="collapse"> A simple mail message</message> <signature personuri=“ssn: //123/45/6789” whitespace=“preserve” > Geoffrey Fox gcf@indiana. edu FAX 8128567972 Phones Cell 315 -254 -6387 Home 812323919 </signature> </smtpbody> </mailasxml> n Such an interface could be used by “Messaging as a Web service” which could perhaps integrate e-mail, text chat, instant messenger, voicemail 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 7

Quick and Dirty mail as XML n n If your boss orders you to produce a quick messaging web service then you might instead define: <kwikmailasxml whitespace=“preserve” > Reply-to: Geoffrey Fox <gcf@indiana. edu> To: Geoffrey Fox <gcf@grids. ucs. indiana. edu> Subject: A Test for Tutorial A simple mail message Geoffrey Fox gcf@indiana. edu FAX 8128567972 Phones Cell 315 -254 -6387 Home 812323919 </kwikmailasxml> n n This is allowed but of course relies on an SMTP parser not an XML parser to decode message Our messaging web service would allow either form but put them on different ports. Internally we would convert to the common XML representation 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 8

XML in a Nutshell n n So we have seen. xml files – XML Instances or the equivalent of Objects in Java There are corresponding. xsd or Schema files which are the equivalent of Classes in Java • Schema happen to be defined in XML syntax • Schema totally replace DTD’s • Originally Schema defined “things”/”properties”/”data” n At times in this tutorial, we will use. wsdl files which define Web Service Interfaces • The wsdl specification separates into components that specify structure of service which is equivalent to “Abstract Method or Interface” and those that specify actual service (equivalent of program and its methods) 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 9

XML Rendering Interfaces n The simplest example is XHTML which is HTML in an XML Syntax • See http: //www. w 3 schools. com/xhtml/default. asp • <BR> in HTML MUST become <br/> • <p> must become <p>Material in Paragraph</p> etc. n Another good example is SVG (Scalable Vector Graphics) which is 2 D Vector Graphics and can represent • • n Macromedia Flash Power. Point and Star. Office Adobe Illustrator/Photoshop etc. See http: //grids. ucs. indiana. edu/courses/xinformatics/svg Here is a snippet of SVG from above link illustrating text, matrix transformations, positioning and styles <text transform="matrix(1 0 0 1 220. 687 216. 7256)" style="fill: #66 FF 00; stroke: none; "> <tspan x="5" y="0">Spring Semester 2001 MWF 2: 30 pm - 3: 20 pm </tspan> </text> 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 10

XML in a Nutshell Concluded n n We can browse some examples at: http: //grids. ucs. indiana. edu/courses/xinformatics/educexample and on the following page Note Schema are labeled by URI’s – their namespaces which allow one to build class libraries n • WSDL files are also referenced by their URI’s (namespaces) in the XML (SOAP) message communicating between 2 Web services • Note version number in URI http: //grids. ucs. indiana. edu/courses/xinformatics/xmlschemaspring 02. ppt n http: //www. w 3. org/TR/xmlschema-0/ are n n reasonable links Always remember that XML defines (potentially complicated) objects and it can specify both data and methods this could have profound implications for architecture and implementation of Enterprise infrastructure Use Castor (http: //castor. exolab. org/) to link XML & Java “automatically” 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 11

A Sample of XML n n n n <? xml version="1. 0"? > <okc version="3" xmlns="http: //grids. ucs. indiana. edu/okc/schema/admin/ver/3"> <event> <comment> This is a Test </comment> <sender email="gcf@grids. ucs. indiana. edu">Geoffrey Fox</sender> <distribution> Community Grids Project Reports </distribution> <organization></organization> <update createuri="gxos: //okc/newsgroups/cgreports/$next“ /> <keywords></keywords> <subject></subject> <message/> <filingdate> 2/6/2002 </filingdate> <cg: message. Type xmlns: cg=“http: //grids. ucs. indiana. edu/okc/schema/cg/ver/1” > Test </cg: message. Type> Schema are here </event> </okc> http: //grids. ucs. indiana. edu/schemas/commgrids-v 1. xsd http: //grids. ucs. indiana. edu/schemas/okc-v 3. xsd 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 12

SOAP over HTTP Structure n n SOAP defines a message with envelope, header and body The SOAP standard also defines a binding to HTTP which is equivalent to methodology for delivering message The SOAP body can specify data as well as fault processing information SOAP can be transported over standard mail by just putting XML in body of mail – see HTTP Delivery with address http: //xml. apache. org/soap/faq_chawke_smtp. html http: //www. w 3 c. org/2000/xp/Group/1/11/16/soap 12 -primer. html#SMTP 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 13

SOAP Example from W 3 C I n We take November 16 tutorial: http: //www. w 3 c. org/2000/xp/Group/1/11/16/soap 12 -primer. html n n This is a SOAP Message – defining a reservation request Header has key meta-data of application (namelist m: and instructions to SOAP (intermediate actor) processors Start Envelope Start Header 11/2/2020 End Header uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" Start Body 14

SOAP Example (Concluded) Start Body n This has the SOAP Body with actual request 11/2/2020 End Body uri="gxos: //gridform/ggf 4/tutorial 2/fox" End Envelope email="gcf@indiana. edu" 15

SOAP Binding to SMTP from W 3 C n n Same example as before This is very straightforward! 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 16

Remarks on SOAP Structure n If you look at networking protocols, at each layer one specifies a header and a body • HTTP has a header telling system where to send information and some high level specification of what to get • In example, we see that body of HTTP message is full SOAP envelope and this envelope has SOAP Header and Body interpreted by “application” • At a lower level in protocol stack, HTTP SOAP TCP/IP and ATM etc. have their own headers and bodies • It is the standard Russian Doll situation – each doll has a body (contained inside it) which is the full doll of SOAP Body higher level protocol 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 17

SOAP Attachments n n This uses same approach as email or HTTP and is a general way of including binary data in XML – but not understood by most parsers! Very important? 11/2/2020 XML links to MIME Attachment using absolute or relative URI’s uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 18

What is a Web Service I n n A web service is a computer program running on either the local or remote machine with a set of well defined interfaces (ports) specified in WSDL In principle, computer program can be in any language (Fortran. . Java. . Perl. . Python) and the interfaces can be implemented in any way what so ever • Interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining) but n n The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python Web Services separate the meaning of a port (message) interface from its implementation 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 19

What is a Web or Grid Service II? n n The Web is distributed system allowing communities to access seamlessly heterogeneous resources from heterogeneous clients • Resources are web-pages, instruments, Object repositories, Simulation codes running on supercomputers …. A Service is a generic application or capability respecting standards (general web and application specific) allowing multiple providers to compete on a given service • For a simulation to be a web service, we want it to have standard interfaces so be run from standard “portals” and access “standard format” data Portal is customizable User interface Middle Tier Broker Back end Capability Resource 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 20

n Role of Web Services Define interfaces of web applications so that computer-computer interactions are enabled • Defines virtual XML for all system and application services n n WSDL is XML versions of Class and Method definitions SOAP is XML version of message UDDI or WSIL catalogs WSDL based services enabling precise linkage of them WSFL and WSCL are candidate linkage languages WSDL interfaces Security Catalog WSDL interfaces 11/2/2020 Payment Credit Card uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" Warehouse shipping 21

Raw Resources Raw Data (Virtual) XML Data Interface Web Service (WS) WS WS WS etc. XML WS to WS Interfaces WS WS WS (Virtual) XML Knowledge (User) Interface Render to XML Display Format Clients 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" (Virtual) XML Rendering Interface 22

Some Sort of XML Protocol Stack n n Protocols are built in a layered fashion – one on top of another At higher (highest) level a protocol becomes an API (Application Interface) • Some reserve protocol to denote the representation of a process involving some negotiation but to my naïve view there is no consensus as to difference between an API and a protocol n We are expanding application (top) layer of OSI protocol stack 11/2/2020 UDDI or WSIL WSFL WSDL SOAP or RMI HTTP or SMTP or IIOP or RMTP TCP/IP Physical Network uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 23

Details of Application Layer n n n n UDDI finds where programs are • remote( (distributed) programs are just Web Services WSFL links programs together WSDL defines interface (methods, parameters, data formats) SOAP defines structure of message including serialization of information HTTP is negotiation/transport protocol TCP/IP is layers 3 -4 of OSI Physical Network is layer 1 of OSI 11/2/2020 UDDI or WSIL WSFL WSDL SOAP or RMI HTTP or SMTP or IIOP or RMTP TCP/IP Physical Network uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 24

Role of UDDI with WSDL n UDDI plays a role like find or grep under UNIX or Google on the web or yellow pages in the real world • A critical difference is that UDDI aims at finding resources matching precise information • Suppose we were a shopping cart service and told by a customer that we need to contact the Suchand. Such online credit card system • We would not only need to find www. suchandsuch. com but also identify the exact URL’s for “approval” and “payment” channels – we would need to get the syntax correct for exchanging data n For instance, we would need to know if system used US Canadian or HK dollars • So UDDI knows about WSDL Interfaces and this knowledge enables programs with such interfaces to link together n n UDDI and WSDL together enable dynamic applications Note “service” just means web-based application built according to agreed interaction rules – such “plug and play” software usually called “components” 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 25

Software Engineering and WSDL/SOAP? n n Software Engineering is a field whose goal is agreed by most to be important • Namely to codify the building of software so that it is more re-usable, modular, easier to maintain and easier to build with teams • However there is not so much agreement as to how to do this and if the field has established itself WSDL is part of component-based software engineering methodology of the web As the Web could be a set of processes and threads on a single PC or Palmtop, WSDL can in fact provide software engineering for conventional sequential or parallel computing. WSDL/SOAP addresses same issues as calling sequences and library methodology in conventional programming 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 26

WSDL and SOAP n We will use the Gateway Computing Portal as an example (http: //www. gatewayportal. org) • It is largely built using CORBA with a Java Server Pages front end • http: //community. ucs. indiana. edu: 8004/GCWS/Batch. Script. Gen/Main. jsp n Several capabilities have been interfaced using WSDL • Job Submission (11 Methods including execute local and remote command, copy files etc. as well as Submit Job) • Manage Web. Flow Session (67 Methods) • Generate Batch Script (just 1 method but two implementations developed – one at SDSC and one at Indiana – with UDDI to manage) • Each is one service – could have used finer grain services • Sample files are at http: //grids. ucs. indiana. edu/ptliupages/presentations/ggf 4 feb 02 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 27

Gateway Architecture n As needed Do. D approved Kerberos/Secure. ID security, does not usually link to standard Grid technology (Globus) Pre WSDL Post WSDL backend Globus or submit to local queue system CORBA Corba middle tier SECIOP with Java modules JSP SSL 11/2/2020 Add Castor Java to XML Will replace historical CORBA By EJB WSDL SOAP wrapper JSP Servlet uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 28

Basic Portal SOAP Interaction Client Web Browser (HTML) SOAP Server P r HTTP(s) P A O e ov T T H Service Components S User Interface Web Server The deployment descriptor maps the SOAP request to the correct service component (Apache SOAP specific). UI Server Components (including SOAP Client) 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 29

Server-Server Communication for Basic SOAP Requests n n n We have two servers: • User interface (UI) server. Includes components for basic functionality (like managing user sessions) plus SOAP client modules. • SOAP server. Converts SOAP requests into calls to the underlying service. User interacts with UI to make requests for services. UI server constructs SOAP request and acts as the SOAP client. For example, if UI server is Tomcat, developer uses Apache SOAP’s Java API implementation to create Java. Bean components to act as SOAP clients. SOAP server executes the request (generate batch script) and returns response. Note that the UI’s SOAP component must match the interface on the SOAP server. This is where WSDL will come in. UI server formats display for user. 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 30

SOAP and Gateway Portal I n n n Having specified service in WSDL, the run-time is implemented in SOAP which is “just” an XML header (info needed by transport – empty here) and body Here is SOAP transported by HTTP message This is exec. Local. Command WSDL operation to run one particular command (ls) on current Web. Flow directory HTTP Header Argument of operation 11/2/2020 Specify ls as SOAP Envelope and body uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 31

HTTP Header SOAP Envelope and body SOAP and Gateway Portal II n 11/2/2020 And this is the result of ls sent back to client in SOAP over HTTP uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 32

WSDL Abstractions n n WSDL abstracts a program as an entity that does something given one or more inputs with its results defined by streams on one or more outputs. Functions are defined by method name and parameters methodname(parm 1, parm 2, … parm. N) • Where parameters are “Input” “Output” or both n In WSDL, we will have a Web Service which like a (Java or CORBA Program) can be thought of as a (distributed) object with many methods • Instead of a function call, the “calling routine” sends an XML message to the Web Service specifying methodname and values of the parameters • Note name of function is just another parameter 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 33

7 Primitives in WSDL n n n n types: which provides data type definitions used to describe the messages exchanged. message: which represents an abstract definition of the data being transmitted. A message consists of logical parts, each of which is associated with a definition within some type system. operation– an abstract description of an action supported by the service. port. Type: which is a set of abstract operations. Each operation refers to an input message and output messages. binding: which specifies concrete protocol and data format specifications for the operations and messages defined by a particular port. Type. port: which specifies an address for a binding, thus defining a single communication endpoint. service: which is used to aggregate a set of related ports 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 34

Discussion of 7 WSDL Primitives n n n n types specify data-structures which are equivalent to arguments of methods message specifies collections of types and is equivalent to set of arguments in a method call. Note that it is an “abstract method” in Java terminology operation is a a collection of input output and fault messages; there are 4 types of operation one-way(service just receives a message), requestresponse(RPC), solicit-response, notification (services pushes out a message) port. Type represents a single channel that can support multiple operations. It is “abstract” as specified as a set of operations. It is equivalent to a “interface or abstract class” in Java binding tells you transport and message format for a porttype (which can have multiple bindings to reflect say performance-portability trades) port combines a binding and an endpoint network address (URL) and is like a “class instance” service consists of multiple ports and is equivalent to a “program” in Java 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 35

Basic Portal WSDL Interaction Client Web Browser (HTML) SOAP Server P r HTTP(s) P A O S User Interface Web Server e ov T T H Service Components WSTK compiles WSDL into client stubs that can be used by UI components UI Server Components (including WSDL clients) 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 36

WSDL-Based Portal Services n n n Basically the same as SOAP server-server interaction described previously. We can use Web Services Toolkit (WSTK) from IBM to generate java client stubs from the WSDL interface. These stubs can be used to create “local” java objects in the UI server that use SOAP RPC to talk to server. So the WSTK creates CORBA-like remote objects. We still use SOAP+HTTP under the hood to make the method calls. 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 37

WSDL Definitions and Namespaces <? xml version="1. 0" encoding="UTF-8" ? > <definitions name="Batch. Script. Service" target. Namespace="http: //yourserver/Batch. Script. Service" xmlns="http: //schemas. xmlsoap. org/wsdl/" xmlns: soap="http: //schemas. xmlsoap. org/wsdl/soap/" xmlns: tns="http: //yourserver/Batch. Script. Service" xmlns: xsd="http: //www. w 3. org/1999/XMLSchema“ > Default Schema (null namespace) is WSDL soap namespace is SOAP binding of WSDL tns namespace is this WSDL file xsd namespace defines (ancient) XML Schema 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 38

WSDL Message Example <message name="submit. Request"> <part name="xmljob" type="xsd: string"/> </message> <message name="submit. Response"> <part name="response" type="xsd: string"/> </message> For the batch script service, we pass the XML description of the job as a string and get back the script as a string. In general, any XML primitive or complex types can be used in messages. We could improve our service by defining a Batch. Script complex type. 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 39

WSDL port. Types Example <port. Type name="Batch. Script. Service. Port. Type"> <operation name="batch. Gen"> <output message="tns: submit. Response" name="submit. Response"/> <input message="tns: submit. Request" name="submit. Request"/> </operation> </port. Type> A port. Type corresponds to a Java class, so if we compile this WSDL to make client stubs, we will generate a Batch. Script. Service. Binding. java class. 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 40

WSDL SOAP Binding Example <binding name="Batch. Binding" type="tns: Batch. Script. Service. Port. Type"> <soap: binding style="rpc" transport="http: //schemas. xmlsoap. org/soap/http"/> <operation name="batch. Gen"> <soap: operation soap. Action=""/> <input> <soap: body use="encoded“ namespace="urn: Batch. Script. Service" encoding. Style="http: //schemas. xmlsoap. org/soap/encoding/"/> </input> <output> <soap: body use="encoded" namespace="urn: Batch. Script. Service“ encoding. Style="http: //schemas. xmlsoap. org/soap/encoding/"/> </output> </operation> note binding’s “type” attribute points back </binding> to the port. Type tag by name 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 41

WSDL Ports and Services <service name="Batch. Script. Service"> <documentation>BS stands for Batch Script </documentation> <port binding="Batch. Binding” name="Batch. Port"> <soap: address location= "http: //yourserver/soap/servlet/rpcrouter/"/> </port> ports are concrete implementations of </service> and point back to a particular </definitions> port. Types binding (by name). They also point to the specific location of a server that implements the service. A service is a collection of one or more ports. 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 42

Portal UDDI Interaction Client Web Browser (HTML) P (1) r HTTP(s) P A O e ov UDDI Server T T H S (2) User Interface Web Server SOAP over HTTP (3) SOAP Server UI Modules include both WSDL and UDDI clients 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 43

SOAP+UDDI+WSDL n n n UI server now includes both WSDL and UDDI client components. UI’s UDDI components are, for example Java. Beans using UDDI 4 J API to interact with the UDDI. The UDDI client components can be used to build administrator HTML/JSP interfaces to add, modify UDDI entries on the server. UDDI client components can also be used to build search interfaces for regular users. Note the UDDI client implementation uses SOAP for RPC to talk to the server. 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 44

Steps 1 -3 on Figure 1. 2. 3. User requests that the UI server performs a search for a particular service. UI server now acts as a client to the UDDI server, gets back the search results, plus URLs for the SOAP servers and the WSDL for the desired service. User selects the desired service provider from UI’s display. UI then becomes client to the SOAP server and executes the service. 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 45

Virtual XML for MPI I n n In days gone by, we used to study: MPI_SEND(buf, count, datatype, dest, tag, comm) [ IN buf] initial address of send buffer (choice) [ IN count] number of elements in send buffer (nonnegative integer) [ IN datatype] datatype of each send buffer element (handle) [ IN dest] rank of destination (integer) [ IN tag] message tag (integer) [ IN comm] communicator (handle) Now our SPMD code is a Web service running in some large scale parallel machine – perhaps integrated with an Apache server running on node • Need to support distributed and shared memory n We want to define all interfaces in WSDL – at least all of them which could be “external” ports 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 46

Virtual XML for MPI II n datatype becomes an XML Schema specifying structure of transmitted data – could either be a URI for Schema or its realization • There would be a built-in schema suitable for handling standard Fortran/C datatypes n n buf is a handle to an XML instance of Schema datatype count is still an integer (xs: integer where xs is Schema namespace xmlns: xs=http: //www. w 3. org/2001/XMLSchema) • For some bindings count is implied by buf n n n dest becomes a URI with special notation such as legacympi: //procnum/local/nnn for processor numbers tag becomes an instance of (new) mpitag Schema comm becomes an instance of (new) mpicommunicator Schema 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 47

HPCC and WSDL Backend Resources Simulation Component Data Component HPC Engine Persistent Storage MPP MPI. . Parallel I/O … Data base XML based Interfaces WSDL Middle Tier Java C++. . Proxy SOAP IIOP RMI …. . Java C++. . Proxy Client 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 48

Virtual XML for MPI III n n So one defines mpi. wsdl with all 100 or so methods defined as (abstract) operations • We will define several bindings including the classic HPCC method calls for Fortran C C++ and Java with either distributed memory or shared memory mode • For C++ and Java we would implements bindings for either “object” or “packed consecutive byte” interpretation of buf Note one can use both static and run-time compilation to map XML specified services into particular implementations MPI_INIT() might be implemented with JXTA (peer-to-peer advertisments) to find nodes; the communicator might be transmitted by SOAP while buf transmission uses MPICH This brings Web Services to MPI but also MPI to web services 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 49

Compiling for WSDL Single Address space Web Application 1 Web Application 2 Web Server stack Method 2 Method 1 Shared Memory 11/2/2020 WSDL Messages Distributed System uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 50

How to Integrate XML/Web Services into an Enterprise I n n You are the CIO of a little known country’s (call it Quagmire) education system and been tasked to position it as a 2005 leader in technology enhanced education Your initial review reveals a multitude of incompatible existing information systems distributed over Quagmire’s 50 campuses Your cradle to grave community education mandate (which got the President – your uncle – elected) implies integration of all aspects of education in your information system You note that performances issues are much less important in this application than interoperability (it currently takes months to get anything done so what's an extra millisecond or so at a WSDL interface) , so even in current primitive state Web services look attractive 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 51

Enterprise Web Services II n n n We start a requirements analysis and a catalog of existing “raw resources”. This will lead to the clean internal XML(WSDL) interfaces QIF and to the external XML interfaces (QEF) to the raw resources Quagmire Internal Framework (QIF) We find that The Grid community has defined computing interfaces which we will adopt to support infrastructure needed for use of simulations and data repositories in curriculum We find that Microsoft, the Liberty Alliance and the Grid have Security schemes but are relieved that with a little bit of XSLT (transformations) we can map between them We join the IMS consortium and adopt their standards for learning meta-data and user related information We adopt the AICC and ADL SCORM standards for “learning objects” (course pages) 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 52

Enterprise Web Services III n n n We define Quagmire educational agents which use QIF plus RDF/DAML/OIL to be able to capture educational goals We expect commercial standards compliant solutions will be available for most other needed Educational web services We find that we must interface with a dreadful collection of external raw resources coming from legacy People. Soft SAP Oracle and Microsoft products • As our new model is object (XML) and open standards based we build a new QII (Quagmire Information Infrastructure) supporting QIF objects n We analyze structure of legacy systems and their relation to QIF and define an external framework QEF which captures much of the raw resources • This Schema is extensible to capture quirks of each legacy system • We build a set of Web services that convert QEF into QIF n We inform vendors that we will only install QIF compliant systems in all future purchases 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 53

Some Science Web Services n These build on general (community) web services 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 54

Some General Grid Web Services 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 55

Education as a Web Service n n n n Can link to Science as a Web Service and substitute educational modules “Learning Object” XML standards already exist from IMS/ADL http: //www. adlnet. org – need to update architecture Web Services for virtual university include: Registration Performance (grading) Authoring of Curriculum Online laboratories for real and virtual instruments Homework submission Quizzes of various types (multiple choice, random parameters) Assessment data access and analysis Synchronous Delivery of Curricula Scheduling of courses and mentoring sessions Asynchronous access, data-mining and knowledge discovery Learning Plan agents to guide students and teachers 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 56

RDF in a Nutshell Subject n n n n Predicate Object The essence of RDF is a statement which has a subject and predicate and an object The “subject” has the property “predicate” with value “object” The Web Page has the title “RDF Model and Syntax” Mary had a husband called John Statements can specify metadata but more generally any facts are expressible in this way One can associate multiple objects with a subject and so build up a general node of a tree with parent as subject and objects as children. DAML and OIL extend RDF with more powerful relations and ontologies 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 57

RDF Tree Fragment Class General Same Predicate: Student Subject Agatha Same or Different Predicates Objec t Henry email Objec t Fred@jim Different Predicates Fred Nick name Freddie grade B- Specific Objec t 11/2/2020 Objec t uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 58

An Example of RDF and Dublin Core n <rdf: RDF xmlns: rdf=“http: //www. w 3. org/1999/02/22 -rdf-syntax-ns#” xmlns: dc=“http: //purl. org/metadata/dublin_core#”> n <rdf: Description about=“http: //www. dlib. org”> n <dc: Title>D-Lib Program - Research in Digital Libraries</dc: Title> <dc: Description>The D-Lib program supports the community of people with research interests in digital libraries and electronic publishing. </dc: Description> <dc: Publisher>Corporation For National Research Initiatives</dc: Publisher> <dc: Date>1995 -01 -07</dc: Date> n <dc: Subject> • <rdf: Bag> <rdf: li> Research; statistical methods</rdf: li> <rdf: li> Education, research, related topics</rdf: li> <rdf: li> Library use Studies </rdf: li> </rdf: Bag> n </dc: Subject> <dc: Type>World Wide Web Home Page</dc: Type> n <dc: Format>text/html</dc: Format> n <dc: Language>en</dc: Language> n </rdf: Description> </rdf: RDF> 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 59

XML Skin Message Or Event Soft Based ware Inter Connection XML Defined Resources connected by XML defined messages Implementation of resource and connection may or may not be XML Resource XML Skin Resource Data base Peer to Peer Community Grid 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 60

P 2 P Grid Architecture n n n Note proposed architecture – resources communicating via messages – is already peer-to-peer as we consider everything democratically – clients, data sources, backend resources Peer-to-peer systems (P 2 P) include Napster, Gnutella and Avaki discussed in another tutorial JXTA (http: //www. jxta. org) is attractive as has very clear architecture and primitives such as peers (users), peer groups , advertisements (XML messages), pipes (message queues), rendezvous (repository of advertisements and other places to find things) JXTA Peers communicate via messages with XML topics used to match service providers with service users Different mechanism than UDDI for registering and discovering resources 11/2/2020 R e s o u rc e s Resources Queued Events Resources uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" R e s o u rc e s 61

Classic Grid Architecture Resources Database Composition Neos Netsolve Security Portal Middle Tier Brokers Service Providers Portal Typically separate Clients Servers Resources 11/2/2020 Clients Users and Devices uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 62

Database Resources Peer-to-Peer Grid Event Broker Resources 11/2/2020 Resources uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 63

Semantic Grid & Digital Brilliance n n n Peer to Peer networks teach us that we can build “small worlds” where distance between nodes is logarithmic in number of nodes Consider a Grid of WSDL services linked (through UDDI) together • This is spirit of Semantic web – RDF/DAML/OIL metadata enables meaningful linkage We do not need to link everybody but only to establish “small world” routes Physics analogies suggest that phase transitions will occur when “enough” nodes are linked – one will get nodes to align in the direction of new knowledge This suggests ways of quantifying value of metadata induced linkages and areas where one “should” add more WSDL specifications 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 64

Publish/Subscribe as a Web Service n We can implement messaging subsystem (between WSDL resources) with either direct messages or by a queued system where you publish messages to queues and subscribe as receiver to particular queues • Natural asynchronous collaboration model which is in fact fast enough for synchronous collaboration n There are many different publish/subscribe models • JMS is a cluster of central servers • JXTA is a very dynamic Peer to Peer model where pipes are queues and topics (metadata) are service advertisements n Implement JMS API with JXTA protocol – different WSDL bindings here have different fault tolerance/reliability semantics • Could use JMS as long distance “carrier” between JXTA peers • JXTA provides higher performance than JMS for nearby recipients n n Pallickara built an intermediate dynamic message broker subsystem which can link JMS and JXTA see: http: //grids. ucs. indiana. edu/ptliupages/projects/narada/ 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 65

Single Server P 2 P Illusion Data base 11/2/2020 JMS Broker uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 66

Broker Network Software multicast P 2 P: Brokers are clients Broker Resource Broker Data base Broker 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 67

Community P 2 P Grid Data base JMS Server JXTA Rendezvous 11/2/2020 GXOS Server Generate “Automatically” GXOS Server uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 68

Narada JXTA/JMS Integration n n http: //grids. ucs. indiana. edu/ptliupages/projects/narada/ Is a network of event brokers which can reliably deliver XML specified events Using open. JMS selection module, becomes a distributed or conventional Java Message Service Linking special JXTA proxies, it can link JXTA communities 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 69

Event Web Service Destination Source Matching Routing (Virtual) Queue Web Service 1 WSDL Ports 11/2/2020 Filter workflow Web Service 2 WSDL Ports uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 70

Some Research Issues for P 2 P Grid n n What happens to programming languages when data structures are defined in XML How do we manage a sea of virtual XML? • Register, find and link objects • This is “distributed operating system of the world” ? n n What happens to databases when everything is an Object defined in XML and transformed by Java? How and when do we compile virtual XML • Convert slow XML message to super fast method call on stack n n n How do we implement services such as Security and collaboration over a range of grain sizes Supporting all “grain sizes” we get some sort of dynamic fractal world which looks like XML objects exchanging XML messages at all scales • Not well supported by centralized services (P 2 P problem) Semantic Grid: as metadata increases, objects link together forming digital brilliance – a phase transition in information space 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 71

Collaborative Web Resources n n Collaboration is “just” sharing objects What about Collaborative Web Services ? • You can in some cases do this automatically just by multicasting messages from service to clients • This is achieved by service publishing messages and clients subscribing n Many applications do not expose all state changes • E. g. when I edit Power. Point slide, Power. Point does not tell the world by sending an (XML) message n n Solved by shared event collaboration model and requires one to view user interface as a “port” in WSDL sense and treat “event handlers” (mouseover, click etc. ) as messages in WSDL Groove Networks does use XML front end to COM interfaces • More elegantly can use W 3 C DOM for (the few) documents (SVG is one) and “universal event handlers” n Interesting research area 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 72

Collaboration as a Web service I 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 73

Future Collaboration Web Service 11/2/2020 uri="gxos: //gridform/ggf 4/tutorial 2/fox" email="gcf@indiana. edu" 74