Building Grid Portlets with GTLAB Mehmet A Nacar

Building Grid Portlets with GTLAB Mehmet A. Nacar and Marlon E. Pierce Community Grids Lab Indiana University

Tera. Grid represents large and growing collection of supercomputing and data storage resources. 20 Petabytes of storage, 280 Teraflops computing power Unified accounting and allocation system

The “Grid” in Tera. Grid: View from Space The Coordinated Tera. Grid Software and Services (CTSS) represents a common software/middleware stack on Tera. Grid machines. Globus and other parts of CTSS give a common programming environment for remotely interacting with Tera. Grid resources. Services include GRAM: resource access Grid. FTP: data management My. Proxy: remote authentication infrastructure Information Services (GPIR, QBETS, MDS, etc) You Are Here

Science Portals and Gateways � Science Gateways and Web portals build on the CTSS stack. � Aggregating clients, user interfaces � Ex: VLAB portal � Tera. Grid Science Gateway program: http: //www. teragrid. org/programs/sci_gateways/ � Many Java-based gateways are based on the portlet component model. � Portlets are reusable portal parts that can be shared between development groups. � Open Grid Computing Environments project provides grid portlets encapsulating common features. � Our work to support VLAB is to build reusable libraries for building portlets. � GTLAB

Motivation � OGCE Grid portlets typically wrap each single Grid capability in a separate portlet � Grid. FTP-->Grid. FTP Portlet � Gateway portlets encapsulate sophisticated but specialized functionality. � Submitting PWSCF jobs � We � need a middle way We need a component model for portlets: reusable portlet parts � Java Server Faces (JSF) is our starting point Remove dependencies on the Servlet API. � Backing beans are just beans, so can be reused more easily outside of web and portlet applications. � � JSF also provides an extensible framework (tag libraries) � Apache JSF portlet bridge allows you to convert standalone JSF applications (development phase) into portlets (deployment phase).

Introduction to GTLAB Encapsulates clients to common Grid services as XML tag libraries and backing Java beans. Embedded by portlet developers in their portlet pages to invoke common tasks Specification of the composite action you want to occur when a user hits the submit button. Allows portal developers to concentrate on the user interface components. Tags can be arranged in directed acyclic graphs (dependency chains). These represent simple workflows. Based on extensions to Java Server Faces and the Java Co. G Kit.

GTLAB Example • Grid tags are associated with Grid services via Grid beans • Grid Beans wrap the Java COG Kit (version 4) • We show an example JSF page section below. • This allows you to develop new Grid portlets with no additional Java code. <html><body> < f: view>  <f: form> <o: submit id=”test” action=”next_page”> <o: myproxy id=”pr” hostname=”gf 1. ucs. indiana. edu” port=” 7512” lifetime=” 2” username=“mnacar” password=”***” /> <o: jobsubmit id=”task” hostname=”cobalt. ncsa. teragrid. org” provider=”GT 4” executable=”/bin/ls” stdout=”tmp/result stderr=”tmp/error” /> </o: submit> </f: form> </f: view> </body> </html>

Grid Tags Associated Grid Beans Features <submit/> Component. Builder. Bean Creating components, job handlers, submitting jobs. This is visually rendered as HTML <handler/> Monitor. Bean Handling monitoring page actions <multitask/> Multitask. Bean Constructing simple workflow <dependency/> Multitask. Bean Defining dependencies among sub jobs <myproxy/> Myproxy. Bean Retrieving myproxy credential <fileoperation/> File. Operation. Bean Providing Gridftp operations <jobsubmission/> Job. Submit. Bean Providing GRAM job submissions <filetransfer/> File. Transfer. Bean Providing Gridftp file transfer (Other JSF UI Tags) Resource. Bean Describes common properties among all tags and beans. Passing values given by standard visual JSF components.

Complex Operations �GTLAB can be used to associate multiple Grid tasks with a single <form> action click. � We call this a “multitask” �This is a form of workflow (DAG) � We build on top of Co. G workflow capabilities. � We are investigating how to abstract this to use other workflow engines. �Each multitask should be associated with a submit button or command link. � This allows many multitasks in a JSF form. � It’s useful in some cases to bind relatively different multitask with the same user input parameters.

Encoding DAGs in Portlets � Multitask provides a simple Directed Acyclic Graph (DAG) � This example demonstrates a composite Grid job using multistaged multitask � GTLAB handles lifecycle of DAG within JSF application

DAG Example JSF Page <o: submit id=”test” action=”next_page” /> <o: multitask id=”mytask” taskname=”test” persistent=”true” > <o: myproxy id=”pr” hostname=”gf 1. ucs. indiana. edu” port=” 7512” lifetime=” 2” username=“nacar” password=”***” /> <o: fileoperation id=”task. A” command=”mkdir” hostname=”cobalt. ncsa. teragrid. org” path=”/home/manacar/tmp/” /> <o: filetransfer id=”task. B” from=”gridftp: //gf 1. ucs. indiana. edu: 2811/home/manacar/input_file” to=”gridftp: //cobalt. ncsa. teragrid. org: 2811/home/manacar/input_file” /> <o: jobsubmit id=”task. C” hostname=”cobalt. ncsa. teragrid. org” provider=”GT 4” executable=”/bin/execute” stdin=”tmp/input_file” stdout=”tmp/result” stderr=”tmp/error” /> <o: filetransfer id=”task. D” from=”gridftp: //cobalt. ncsa. teragrid. org: 2811/home/manacar/tmp/result” to=” gridftp: //gf 1. ucs. indiana. edu: 2811/home/manacar/result” /> <o: dependency id=”dep 1” task=”task. B” depends. On=”task. A” /> <o: dependency id=”dep 2” task=”task. C” depends. On=”task. B” /> <o: dependency id=”dep 3” task=”task. D” depends. On=”task. C” /> </o: multitask> </o: submit> This encodes the DAG on the previous page.

A Simple Use Case � You (the portal developer) have a code that you want to run on the Tera. Grid through your portal. � You will need to develop these parts: � User interface layout in JSF/JSP so user can provide input information and select a host to run on. � The JSF/JSP file will also include the GTLa. B tags on the following slide. � Some Java code to construct an input file. � You will NOT have to reimplement all of the Grid code that describes your action methods.

JSF Page with Grid Tags <o: submit id="submit" value="Submit" binding="#{builder. body}" action="#{builder. junk. Action}" > <o: multitask id="multi" persistent="true" taskname="#{resource. taskname}"> <o: myproxy id="mypr" hostname="gf 1" lifetime="2" password="manacar" port="7512" username="manacar"/> <o: jobsubmit id="make" arguments="/home/manacar/disloc-work" executable="/bin/mkdir" hostname="gf 1. ucs. indiana. edu" provider="GT 2" stdout="/home/manacar/tmp/out-make"/> <o: jobsubmit id="disloc” arguments="/home/gateway/GEMCodes/Disloc/input. txt /home/manacar/disloc-work/disloc. out" executable="/…/disloc" hostname="gf 1. ucs. indiana. edu" provider="GT 2" stdout="/home/manacar/disloc-work/out-disloc"/> <o: dependency id="dep" depends. On="make" task="disloc"/> </o: multitask> </o: submit> Note the specific values would typically come from the user’s form inputs through the Resource Bean.

Integration with JSF Form Elements � Developers embed Grid tags snippet into JSF page � These components are non-visual and are not displayed in HTML. � Resource bean provides bridging with form inputs and GTLAB framework. � <h: output. Text value="Taskname: "/> <h: input. Text value="#{resource. taskname}" /> <o: multitask id="multi" persistent="true" taskname="#{resource. taskname}" /> � Dynamic values to Grid tag attributes are provided by Resource bean.

Tracking and Managing Jobs � GTLAB manages lifecycles of jobs and monitor their status. � Grid operations are usually batch processes � We provide callback mechanism to follow up the jobs � GTLAB creates callback handlers for jobs and persistently them via object serialization. � Can be later de-serialized. � GTLAB stores handlers manages the job events such as stopping, canceling or resuming the running jobs. � GTLAB provides archive for job metadata and allows managing the archive � Handler tag helps to organize user’s job repository � <o: handler id=”delete” action="#{monitor. delete}" > <f: param id="task" name="taskname“ value="#{task}"/> </o: handler>

Managing Metadata Repositories Job metadata is stored in metadata repositories Metadata includes: submission parameters and files, execution host and information, output parameters, files and their location Job data is stored on the specified servers. As a result data files are transferred by using Grid. FTP Metadata of the data files such as URL location or Grid. FTP locations are saved in the metadata repository Portal users see the job metadata that has links to the exact location of the data Metadata repository is built by using WS-Context repository Users see job metadata with the time stamped hierarchical repository labels /vlab/user. A/session. C/08 -08 -07/12: 22/job. ID

Summary and Conclusions �We briefly described the Tera. Grid, Grid middleware, and Science Gateways. �Motivation for our work: portlet-based gateways need to be composed out of reusable parts. �Our solution: GTLAB tag libraries and backing beans �XML tag libraries backed by Java beans that encapsulate common (Globus) Grid tasks. �Extend JSF framework � GTLAB tags can be arranged into DAGs to express composite tasks

More Information � GTLAB version 1. 0 Beta release available at � http: //grids. ucs. indiana. edu/users/manacar/GTLAB-website � Contact Marlon: mpierce@cs. indiana. edu � Contact Mehmet: mnacar@indiana. edu

Relation to Web 2. 0 � Web 2. 0 can be divided into several major categories, including primarily � � � GTLa. B and RCI � � GTLa. B does not directly address this either way RCI can be achieved through JSF implementations (which generate the necessary Java. Script). Or you can mix and match, using YUI or Scriptaculous Javascript libraries with your JSF. GTLa. B and REST � � � Rich client interfaces (RCI) such as AJAX REST-style Web services (typically transmitting RSS or Atom XML). Conceivably we could also build REST clients for consuming and manipulating RSS feeds. (c. f. Yahoo Pipes) This would require a new workflow engine implementation. For a more thorough survey, see � � � http: //grids. ucs. indiana. edu/ptliupages/presentations/CTSpart. IMay 2107. ppt http: //grids. ucs. indiana. edu/ptliupages/presentations/Web 20 Tutorial_ CTS. ppt http: //grids. ucs. indiana. edu/ptliupages/publications/CIWeb 20 Chapter. doc

Related Work � Grid Portlets 1. 3 of Grid. Sphere Now they are trying to decouple with Grid. Sphere. It’s called Vine (Portlet Vine) as separate project � Grid Portlets 1. 3 provide API and UI tags to build Grid portlets � � RSF (Reasonable Server Faces) Derived from JSF, but it separates HTML pages and backing beans � RSF provides non-visual components unlike JSF � Beans can be contained by Spring like containers. � � Lifecycles � OGCE of beans managed by Spring portlets Packages Velocity, JSP and JSF portlets � Provides portlet package for several Grid applications such as Globus, Condor, SRB and GPIR �