Leading the pervasive adoption of grid computing for

Leading the pervasive adoption of grid computing for research and industry JSDL Stephen Mc. Gough With thanks to: Ali Anjomshoaa, Fred Brisard, Michel Drescher, Donal K. Fellows, William Lee, An Ly, Darren Pulsipher, Andreas Savva, Chris Smith 10 May 2006 © 2005 Global Grid Forum The information contained herein is subject to change without notice

Describing a Job Submission: JSDL • Job Submission Description Language (JSDL) − A language for describing the requirements of jobs for submission − Declarative description • A JSDL document describes the job requirements − − Job identification information Application (e. g. , executable, arguments) Required resources (e. g. , CPUs, memory) Input/output files Job JSDL IT Infrastructure 2

JSDL Document Usage JSDL Here And Here Super Scheduler, or Broker, or … A Grid Information Service JSDL Here Existing DRM WS Clients WS Gateway Job Manager Local resource (e. g. , Supercomputer) OGSA BES system Local Information Service And Here 3

JSDL Introduction • JSDL stands for Job Submission Description Language − A language for describing the requirements of computational jobs for submission to Grids and other systems. • A JSDL document describes the job requirements − What to do, not how to do it • JSDL does not define a submission interface or what the results of a submission look like − Or how resources are selected, or … • The JSDL-WG is now considering its next steps. • JSDL 1. 0 is published as GFD-R-P. 56 − Includes description of JSDL elements and XML Schema − Available at http: //www. ggf. org/gf/docs/? final 4

Why do we need JSDL? • There are many languages out there for submitting jobs to the Grid – so why do we need JSDL? − This is the problem. We didn’t have a standard for job submission − Each system uses its own language − Hence interoperability is difficult (if not impossible) • We set out to develop one language for all to use − A standard language that we can use between systems − Or for submitting to any system • This is especially important for Workflow 5

JSDL Document (1) • A JSDL document is an XML document • It may contain − − • Out − − Generic (job) identification information Application description Resource requirements (main focus is computational jobs) Description of required data files of scope, for JSDL version 1. 0 Scheduling Workflow Security … 6

JSDL Document (2) • A JSDL document is a template … − It can be submitted multiple times; and therefore can be used to create multiple job instances • … so JSDL 1. 0 does not define attributes to describe the state of a running job − No start time, end time, submission status, or even Job. ID • A JSDL document can be composed with other languages (open content model) − For example to express scheduling, security, etc, requirements in more detail − Within other documents - Workflows 7

JSDL: Conceptual relation with other standards Workflow Job JLM Job JSDL JLM JSDL … RRL JPLJLM RRL … JPL … RRL SDL WS-A … JPL SDL WS-A … Job JSDL JLM RRL … JPL SDL WS-A … SDL – Scheduling Description Language RRL - Resource Requirements Language JPL – Job Policy Language JLM – Job Lifetime Management WS-A – WS-Agreement 8

JSDL Document Life Cycle • A JSDL document may be − Abstract • Only the minimum information necessary − For example, application name and input files • Runnable at sites that understand this level of description − Refined • More detail provided − Target site, number of CPUs, which data source • May be refined several times − Tied to a specific site/system BES • Incarnated (Unicore speak); or • Grounded (Globus speak) • This model is supported/allowed but not required by JSDL 9

A few words on JSDL and BES • JSDL is a language − No submission interface defined (on purpose) − JSDL is independent of submission interfaces • BES is defining a Web Service interface which consumes JSDL documents − This is not the only use of JSDL − Though we do like it JSDL BES Container 10

JSDL Document Structure Overview <Job. Definition> <Job. Description> <Job. Identification. . . />? <Application. . . />? <Resources. . . />? <Data. Staging. . . />* </Job. Description> </Job. Definition> • Note: − − None ? * + [1. . 1] [0. . n] [1. . n] 11

Data Staging Requirement (1) • Previous statements included: − “A JSDL document describes the job requirements • What to do, not how to do it*” − “Workflow is out of scope. ” • But … data staging is a common requirement for any meaningful job submission − Especially for batch job submission − No standard to describe such data movements − Don’t want to do a ‘workflow’ language. 12

Data Staging Requirement (2) • Our solution − Assume simple model: Stage-in – Execute – Stage-Out − Files required for execution • Files are staged-in before the job can start executing − Files to preserve • Files are staged-out after the job finishes execution • More complex approaches can be used − But this is outside JSDL − You don’t need to use the JSDL Data Staging Stage-In Execute Stage-Out 13

JSDL & Workflow • We hope that JSDL can be used as core elements within a workflow language − Such as BPEL − Etc… Action 1 JSDL Action 3 JSDL Action 4 JSDL Action 2 JSDL 14

Example Case: Grid. SAM & Sedna JSDL Job Submission 15

JSDL Adoption • The following projects have presented at GGF JSDL sessions and are known to have implementations of some version of JSDL; not necessarily 1. 0. − − − − • • Business Grid Programming Environment (GPE) Grid. SAM HPC-Europa Market for Computational Services NAREGI Uni. Grids The following groups also said they are or will be implementing JSDL: − − DEISA Grid. Bus Project (see OGSA Roadmap, section 8) grid. Matrix (Cadence) (presentation) Nordugrid Also within GGF a number of groups either use directly or have a strong interest or connection with JSDL: − BES-WG, CDDLM-WG, DRMAA-WG, GRAAP-WG, OGSA-WG, RSS-WG • An up-to-date version of this list is on Gridforge: https: //forge. gridforum. org/projects/jsdl-wg/document/JSDL-Adoption/en/ 16

JSDL Mappings • ARC (Nordu. Grid) • JOb Scheduling Hierarchically (JOSH) • Condor • e. NANOS • LSF • Fork • Sun Grid Engine • Globus 2 • Unicore • GRIA provider • <Your mapping here> • Grid Resource Management System (GRMS) 17

Questions 18

GGF Full Copyright Notice Copyright (C) Global Grid Forum (2006). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the GGF or other organizations, except as needed for the purpose of developing Grid Recommendations in which case the procedures for copyrights defined in the GGF Document process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the GGF or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE GLOBAL GRID FORUM DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. " 19