Integrating Peer to Peer Networks and Web Services
Integrating Peer to Peer Networks and Web Services with the Grid Part of GGF 4 Update Track Toronto February 18 2002 Grid Programming with Web Services, Software Components and Portal Development Tools PTLIU Laboratory for Community Grids Geoffrey Fox, Marlon Pierce, Shrideep Pallickara, Choonhan Youn Computer Science, Informatics, Physics Indiana University, Bloomington IN 47404 http: //grids. ucs. indiana. edu/ptliupages/presentations/ggf 4 feb 02 gcf@indiana. edu 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 1
Gateway and Web Services n We will use the Gateway Computing Portal as an example (http: //www. gatewayportal. org) • It is largely built using CORBA with a Java Server Pages front end • http: //community. ucs. indiana. edu: 8004/GCWS/Batch. Script. Gen/Main. jsp n Several capabilities have been interfaced using WSDL • Job Submission (11 Methods including execute local and remote command, copy files etc. as well as Submit Job) • Manage Web. Flow Session (67 Methods) • Generate Batch Script (just 1 method but two implementations developed – one at SDSC and one at Indiana – with UDDI to manage) • Each is one service – could have used finer grain services • Sample files are at http: //grids. ucs. indiana. edu/ptliupages/presentations/ggf 4 feb 02 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 2
Gateway Architecture n As needed Do. D approved Kerberos/Secure. ID security, does not usually link to standard Grid technology (Globus) Pre WSDL Post WSDL backend Globus or submit to local queue system CORBA Corba middle tier SECIOP with Java modules JSP SSL 9/15/2021 Add Castor Java to XML Will replace historical CORBA By EJB WSDL SOAP wrapper JSP Servlet uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 3
9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 4
9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 5
9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 6
WSDL Message Example <message name="submit. Request"> <part name="xmljob" type="xsd: string"/> </message> <message name="submit. Response"> <part name="response" type="xsd: string"/> </message> For the batch script service, we pass the XML description of the job as a string and get back the script as a string. In general, any XML primitive or complex types can be used in messages. We could improve our service by defining a Batch. Script complex type. 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 7
WSDL port. Types Example <port. Type name="Batch. Script. Service. Port. Type"> <operation name="batch. Gen"> <output message="tns: submit. Response" name="submit. Response"/> <input message="tns: submit. Request" name="submit. Request"/> </operation> </port. Type> A port. Type corresponds to a Java class, so if we compile this WSDL to make client stubs, we will generate a Batch. Script. Service. Binding. java class. 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 8
WSDL SOAP Binding Example <binding name="Batch. Binding" type="tns: Batch. Script. Service. Port. Type"> <soap: binding style="rpc" transport="http: //schemas. xmlsoap. org/soap/http"/> <operation name="batch. Gen"> <soap: operation soap. Action=""/> <input> <soap: body use="encoded“ namespace="urn: Batch. Script. Service" encoding. Style="http: //schemas. xmlsoap. org/soap/encoding/"/> </input> <output> <soap: body use="encoded" namespace="urn: Batch. Script. Service“ encoding. Style="http: //schemas. xmlsoap. org/soap/encoding/"/> </output> </operation> note binding’s “type” attribute points back </binding> to the port. Type tag by name 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 9
9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 10
WSDL Ports and Services <service name="Batch. Script. Service"> <documentation>BS stands for Batch Script </documentation> <port binding="Batch. Binding” name="Batch. Port"> <soap: address location= "http: //yourserver/soap/servlet/rpcrouter/"/> </port> ports are concrete implementations of </service> and point back to a particular </definitions> port. Types binding (by name). They also point to the specific location of a server that implements the service. A service is a collection of one or more ports. 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 11
What is a Virtual XML Interface n n We can specify interfaces in XML but we are not required to implement in XML. Example 1: We aren’t likely to change syntax of mail Reply-to: Geoffrey Fox <gcf@indiana. edu> To: Geoffrey Fox <gcf@grids. ucs. indiana. edu> Subject: A Test for Tutorial A simple mail message Geoffrey Fox gcf@indiana. edu FAX 8128567972 Phones Cell 315 -254 -6387 Home 812323919 n n But we could specify and indeed store in XML with transport done using conventional SMTP. So conventional mail is easy to give a virtual XML specification for with name: value becoming <name>value</name> but this is not only way 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 12
Mail in XML n <mailasxml uri=“gxos: //mail/users/gcf/sent/2002/february/290” > <smtpheaders> <reply-to email=“gcf@indiana. edu” >Geoffrey Fox</reply-to> <to email=“gcf@grids. ucs. indiana. edu” >Geoffrey Fox</to> <subject>A Test for Tutorial</subject> </smtpheaders> <smtpbody> <message whitespace="collapse"> A simple mail message</message> <signature personuri=“ssn: //123/45/6789” whitespace=“preserve” > Geoffrey Fox gcf@indiana. edu FAX 8128567972 Phones Cell 315 -254 -6387 Home 812323919 </signature> </smtpbody> </mailasxml> n Such an interface could be used by “User Messaging as a Web service” which could perhaps integrate News Groups, e-mail, text chat, instant messenger, voicemail 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 13
Virtual XML for MPI I n n In days gone by, we used to study: MPI_SEND(buf, count, datatype, dest, tag, comm) [ IN buf] initial address of send buffer (choice) [ IN count] number of elements in send buffer (nonnegative integer) [ IN datatype] datatype of each send buffer element (handle) [ IN dest] rank of destination (integer) [ IN tag] message tag (integer) [ IN comm] communicator (handle) Now our SPMD code is a Web service running in some large scale parallel machine – perhaps integrated with an Apache server running on node • Need to support distributed and shared memory n We want to define all interfaces in WSDL – at least all of them which could be “external” ports 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 14
Virtual XML for MPI II n datatype becomes an XML Schema specifying structure of transmitted data – could either be a URI for Schema or its realization • There would be a built-in schema suitable for handling standard Fortran/C datatypes n n buf is a handle to an XML instance of Schema datatype count is still an integer (xs: integer where xs is Schema namespace xmlns: xs=http: //www. w 3. org/2001/XMLSchema) • For some bindings count is implied by buf n n n dest becomes a URI with special notation such as legacympi: //procnum/local/nnn for processor numbers tag becomes an instance of (new) mpitag Schema comm becomes an instance of (new) mpicommunicator Schema 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 15
Virtual XML for MPI III n n So one defines mpi. wsdl with all 100 or so methods defined as (abstract) operations • We will define several bindings including the classic HPCC method calls for Fortran C C++ and Java with either distributed memory or shared memory mode • For C++ and Java we would implements bindings for either “object” or “packed consecutive byte” interpretation of buf Note one can use both static and run-time compilation to map XML specified services into particular implementations MPI_INIT() might be implemented with JXTA (peer-to-peer advertisments) to find nodes; the communicator might be transmitted by SOAP while buf transmission uses MPICH This brings Web Services to MPI but also MPI to web services 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 16
Collab. WS: Collaboration as a Web Service n n n This includes Audio-Video conferencing, Instant Messengers, shared applications and tools There are basic standards H 323 and SIP There are peer to peer collaboration approaches like JXTA from Sun There are various commercial products – Groove Networks effectively defines some shared application interfaces Collab. WS defines a superset of collaboration properties and subservices in XML/WSDL Then you provide several bindings of collabws. wsdl to new transports and old ones (binary H 323) 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 17
H 323 SIP JXTA 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 18
SIP H 323 JXTA SIP H 323 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" JXTA 19
A Typical SIP Message n n REGISTER sip: registrar. biloxi. com Via: SIP/2. 0/UDP 10. 4. 1. 4: 5060 To: Bob (sip: bob@biloxi. com) From: Bob (sip: bob@biloxi. com); tag=456248 Call-ID: 843817637684230@phone 21. boxesbybob. com CSeq: 1826 REGISTER Contact: (sip: bob@10. 4. 1. 4) Expires: 7200 Contact-Length: 0 Initially build a wrapper that accepts such messages and converts to …. . 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 20
Collaboration as a Web service - now 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 21
SIP as XML n n <sipasxml> <register> sip: registrar. biloxi. com</register> <via> SIP/2. 0/UDP 10. 4. 1. 4: 5060</via> <to> Bob (sip: bob@biloxi. com) </to> <from tag="456248" > Bob (sip: bob@biloxi. com) </from> <call-id> 843817637684230@phone 21. boxesbybob. com </call-id> <cseq> 1826 REGISTER </cseq> <contact> (sip: bob@10. 4. 1. 4) </contact> <expires> 7200 </expires> <contact-length> 0 </contact-length> </sipasxml> So this is either an internal format to which we map SIP messages or it is an external format used for Collaboration Web Service 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 22
Quick SIP as XML (SOAP based WS) n n <kwiksipasxmal> REGISTER sip: registrar. biloxi. com Via: SIP/2. 0/UDP 10. 4. 1. 4: 5060 To: Bob (sip: bob@biloxi. com) From: Bob (sip: bob@biloxi. com); tag=456248 Call-ID: 843817637684230@phone 21. boxesbybob. com CSeq: 1826 REGISTER Contact: (sip: bob@10. 4. 1. 4) Expires: 7200 Contact-Length: 0 </kwiksipasxml> This requires a SIP parser 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 23
Future Collaboration Web Service 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 24
Messaging/Events as a Web Service n n We can implement messaging subsystem (between WSDL resources) with either direct messages or by a queued system where you publish messages to queues and subscribe as receiver to particular queues There are many different publish/subscribe models • JMS is a cluster of central servers • JXTA is a very dynamic Peer to Peer model where pipes are queues and topics (metadata) are service advertisements n Implement JMS API with JXTA protocol – different WSDL bindings here have different fault tolerance/reliability semantics • Could use JMS as long distance “carrier” between JXTA peers • JXTA provides higher performance than JMS for nearby recipients 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 25
Event Web Service Destination Source Matching Routing (Virtual) Queue Web Service 1 WSDL Ports 9/15/2021 Filter workflow Web Service 2 WSDL Ports uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 26
Broker Network Software multicast P 2 P: Brokers are clients Broker Resource Broker Data base Broker 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 27
Narada JXTA/JMS Integration n n http: //grids. ucs. indiana. edu/ptliupages/projects/narada/ Is a network of event brokers which can reliably deliver XML specified events Using open. JMS selection module, becomes a distributed or conventional Java Message Service Linking special JXTA proxies, it can link JXTA communities Think of JXTA JMS and Narada as different bindings to event/messaging web service 9/15/2021 uri="gxos: //gridform/ggf 4/talk/fox" email="gcf@indiana. edu" 28
- Slides: 28