Grid Federation JXTA Jini etc GGF 7 Tokyo

Grid Federation JXTA Jini etc. GGF 7 Tokyo March 5 2003 PTLIU Laboratory for Community Grids Geoffrey Fox Computer Science, Informatics, Physics Indiana University, Bloomington IN 47404 http: //www. naradabrokering. org http: //grids. ucs. indiana. edu/ptliupages gcf@indiana. edu 9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 1

Users Portal such as “Jetspeed” H o s t i n g E n v i r o n m e n t AWS AWS Application/User Framework supporting development and deployment of OGSI compliant AWS (Application Web Services) Generic Application Services “Core” System Services Resource Grid Services Database 9/25/2021 AWS H o s t i n g E n v i r o n m e n t Grid Computing or Programming Environments Web Services “Core” Grid e. g. DAI compliant database Resources uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 2

Federation/Interoperability Problem? n n n Have a collection of Web Services running in Grids defined by different suppliers? Interoperability – “particular application Web Service of supplier X” can utilize “core service of supplier Y” Federation– “core service of supplier X” can be integrated with “core service of supplier Y” to provide a integration/amalgam that is also a realization of core service. Need mediation to link different Grid Islands AWS 1 Avaki Grid IBM Grid JXTA Service 2 Resource 2 AWS 2 GT 3 Grid 9/25/2021 AWS 3 Platform Service 1 Jini Grid Resource 1 uri="http: //www. naradabrokering. org" Standards Compliant Inter. Grids Federationemail="gcf@indiana. edu" Environment 3

OGSI Interface Implications n n OGSI defines the features that all Web Services must have to be called Grid Services Using Web Services is equivalent roughly (in old-style computing lingo) to agreeing on a common (object-oriented) language • It is equivalent to choosing C++ or Java or more precisely agreeing to use CORBA IDL or Java to define external interfaces (method calls) without prejudice as to implementation language n n n For example, having a “Grid Information Port” is VERY roughly analogous to agreeing that all programs have a standard input and a standard output familiar from UNIX Agreeing that “Grid Information Ports” should communicate XML records (SDE Service Data Element) is roughly equivalent to saying standard output will use Fortan namelist syntax So this is not yet a complete interoperability or federation specification 9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 4

Inter. Grids n n Develop Grid software including both the packaging of Grid Islands and federation between Grids Key underlying principles: • Support small Grids (university departments, Private Grids) with easy deployment (e. g. JXTA Jini …) • Support composability (P 2 P) and federation with itself and other Grids • Use existing (Avaki, Globus JXTA ICENI. . ) bits and pieces if possible – encourage such projects to produce modules useable in other Grid systems like Inter. Grids • Good test/benchmark/tutorial material and good support 9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 5

Features of Inter. Grids I n n n At outer levels of Grid, just need interoperable services but these services would need mediation as they access resources in a different grid • Interoperable Services have interfaces allowing them to access appropriate other services in their own grid and through mediation the analogous services in other grids At inner levels of Grid, need to federate services • Federated services produce results which need to be merged with other services of the same type when Grids are joined together In case of Jini based Grid, we keep Jini services as Jini and don’t wrap them Rather the OGSA wrapping of Jini occurs in the mediator Issues if multiple Grids overlap • Can multiple brokers manage same resources (schedulers on computers) • What happens if one shares resources between virtual organizations (Grids) 9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 6

Features of Inter. Grids II n Mediation service integrates • • n n VPN with adaptable port Firewalls and security negotiation between domains Mediation Difference between PURL/URI and URL Mediation Service is distributed i. e. is not a single proxy server Each logical message (this could be several physical messages) in system is examined – destination URI is “just” mapped to URL if internal • If external routed to “best mediator” in same way Narada. Brokering optimally routes messages • Logical messages are self contained i. e. have enough information to fully specify any transformations needed in mediator i. e. they are “all” the Schema instance 9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 7

Features of Inter. Grids III n When message arrives at mediator service, its source and destination Grids are examined • Note Grids must register themselves in mediation service and in that registration (or a later update) give specification for service and any needed transformations) • This registration involves introduction of concept of Gridtype with specification attached to a particular Gridtype • If they have same specification for this service, then message is routed on unchanged • If specification different, the mediator looks to see if any special translations available for given source or destination • If no special actions, the source mapper is used to convert message to FGSA (Federated Grid Service Architecture -- hopefully same as OGSA) and the destination inverse mapper is used to map FGSA to destination format n This can generate an error returned to calling service and logged for both Grids 9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 8

Features of Inter. Grids IV n n n This mediation is sufficient for an “interoperable” service Some services such as “search” or “look-up” are registered as federated • i. e. requests to them are multi-cast to multiple grids and results merged • Rules must be defined for federated services defining nature of result n Is more than one answer allowed n If >1 answer possible, then rules for merging results of services in each Grid must be specified Another complication is if a single service in one Grid corresponds to composition of multiple services in another Grid • One sees this for look-up which could involve different levels of meta-data in different Grids • This seems complicated but relatively clear what one has to do in composing dynamically services 9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 9

Features of Inter. Grids V n n n There are two problems mentioned earlier Shared Resource Reference Services – this must be a standard issue in federation. This occurs when federation of say look up services leads to duplicate results. The look-up may not be quite the same and one would want to remove or combine duplicate responses Shared Resource Access Services – these are services in different Grids that access and affect the same backend resource. This issue comes up even inside a single Grid • It is possible that mediator could be used to resolve this problem but an heuristic needs to be developed for this 9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 10

Federation Architecture Routing Node Resource Mediation Node Resource R Resource Grid Instance Service R Service M M Service M R Resource Grid Instance Service Resource M R M Resource Grid Instance Service Resource 9/25/2021 Resource Grid Instance M R Service Resource uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" Resource 11

Narada. Brokering Publish/Subscribe Distributed Event/Message System http: //www. naradabrokering. org “MQSeries/JMS” P/S applied to Collaboration, Grid, P 2 P(JXTA) Supports UDP, TCP/IP, Firewalls (actual transport user call) Used in other projects: Collaboration, Portal and Handheld 9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 12

Narada. Brokering n Based on a network of cooperating broker nodes • Cluster based architecture allows system to scale to arbitrary size n n Originally designed to provide uniform software multicast to support real-time collaboration linked to publish-subscribe for asynchronous systems. Now has five major core functions • Message transport (based on performance measurement) in heterogeneous multi-link fashion • General publish-subscribe including JMS & JXTA and support for RTP-based audio/video conferencing • Distributed XML data-base using P/S XPATH metaphor • Filtering for heterogeneous clients • Federation of multiple instances of Grid services as illustrated by JXTA peer-group linkage 9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 13

9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 14

9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 15

Narada. Brokering and JXTA Narada-JXTA provides JXTA guaranteed long distance delivery Narada JXTA Event Request/Response Present if targeted at Particular peer 9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 16

9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 17

9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 18

9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 19

Federation of Services n If you have a Service – Notification (as with Grid or JXTA advertisements), Search, Scheduling, Audio-Video conferencing …. • With a standard for client and server components • Then can build a “server” that services all clients n Alternatively can hierarchically consider collection of existing Server-client domains • • n IBM or Globus OGSA islands Sun Grid Engine Schedulers Polycom/Access Grid A/V sessions Enterprise Search Engines NB Hub Federation links servers for each island together • JXTA Search has XML specified federation approach – will try to include and generalize as a Narada. Brokering federation framework • Do. D High Level Architecture HLA does this for simulation 9/25/2021 uri="http: //www. naradabrokering. org" email="gcf@indiana. edu" 20