Enabling Grids for Escienc E Grid Services for

Enabling Grids for E-scienc. E Grid Services for Resource Reservation and Allocation Tiziana Ferrari Istituto Nazionale di Fisica Nucleare, INFN CNAF Terena Networking Conference, Poznan, June 6 2005 www. eu-egee. org INFSO-RI-508833

Outline Enabling Grids for E-scienc. E • Use cases • The Grid Resource Allocation and Agreement Protocol • The EGEE g. Lite Agreement Service: – Architecture – Components • Workload Management System instrumentation • Implementation: the space reservation Agreement Service prototype • Future work INFSO-RI-508833

Grid Computing (2) Enabling Grids for E-scienc. E “A computational Grid is a hardware and software infrastructure that provides dependable, consistent, pervasive and inexpensive access to high-end computational capabilities” Ian Foster, Carl Kesselman • Purpose of Grid Computing: flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources From “The Anatomy of the Grid: Enabling Scalable Virtual Organizations” INFSO-RI-508833

Use Cases Enabling Grids for E-scienc. E • NETWORK: Data replication: to optimize performance of a data transfer session (that otherwise would compete with other streams and would be subject to variable throughput) to support file transfer with deadline (to synchronize job execution with input file transfer) • COMPUTING: to reserve computing resources (e. g. worker nodes, CPU cycles) in presence of a large number of other competing jobs • STORAGE: to guarantee that a sufficient amount of space is present in a Storage Element to save the output data the Storage Element is close to the Computing Element on which computing resources have been reserved INFSO-RI-508833

Enabling Grids for E-scienc. E • Use cases • The Grid Resource Allocation and Agreement Protocol • The g. Lite Agreement Service: – Architecture – Components • Workload Management System instrumentation • Implementation: the space reservation Agreement Service prototype • Future work INFSO-RI-508833

Grid Resource Allocation and Agreement Protocol – GGF GRAAP 2/2 Enabling Grids for E-scienc. E • WS-Agreement defines a language and a protocol for – Advertising the capabilities of providers – Checking compliance to pre-defined templates – Creating agreements based on creational offers • Agreement Layer: – Provides a web service based interface – represents and monitors agreements with respect to provisioning of services implemented in the service layer • Service Layer (out of the GRAAP scope): – is an resource-specific layer of a provided service – The interface to this layer is domain-specific: may or may not be exposed as a web service interface INFSO-RI-508833

Grid Resource Allocation and Agreement Protocol – GGF GRAAP 1/2 Enabling Grids for E-scienc. E WS-Agreement specification Conceptual Layered Service Model [2] g. Lite Agreement Service allocation and reservation service provider INFSO-RI-508833

Agreement: XML structure Enabling Grids for E-scienc. E Name: Agreement Name Context Terms Service Description Terms Guarantee Terms identificator Context: participants’ names, lifetime, links to other agreements related to this (co-allocation) Terms: Service Description Terms: - provide information needed to instantiate and identify a service to which this agreement pertains - describes the functionality that will be delivered under an agreement Guaranteed Terms: specify the service level that the parties are agreeing to Terms are usage-domain independent. INFSO-RI-508833

Enabling Grids for E-scienc. E • Use cases • The Grid Resource Allocation and Agreement Protocol • The g. Lite Agreement Service: – Architecture – Components • Workload Management System instrumentation • Implementation: the space reservation Agreement Service prototype • Future work INFSO-RI-508833

The g. Lite Agreement Service 2/2 Enabling Grids for E-scienc. E Agreement Initiators Initiator Agreement Offer Storage/Computing/ Network Agreement Service Reservation and Allocation Service Providers INFSO-RI-508833

The g. Lite Agreement Service 1/2 Enabling Grids for E-scienc. E • Agreement offer – the “description” of the requested reservation • Agreement Initiator – WMS Reservation Manager (reservation manager) – Data Scheduler – User • Agreement Service – Checks offer compliance to one of theinternal templates – Can interact with several types of resource reservation service providers (network – JRA 4 bandwidth advance reservation, storage – Sto. RM, computing) • Resource reservation Service Provider Enforces the reservation on a specific resource instance • --> Full architecture described in [3] INFSO-RI-508833

Agreement Service deployment Enabling Grids for E-scienc. E Workload Manager System Agreement Offer Reservation Manager (simple and co-allocation) Agreement Offer_1 Resource INFSO-RI-508833 Agreement Offer_n-1 Agreement Service_1 Service Provider_1 Logging & Bookkeeping Agreement Service_n-1 Monitor_1 Reservation usage Sensor_1 Service Provider_1 Resource Agreement Service_n Monitor_1 Reservation usage Sensor_1

Component 1. Service Provider Enabling Grids for E-scienc. E • One Service Provider instance per “resource” instance (e. g. CE and SE). • manages resource availability information over time through a local “slot table” • enforces resource reservation through resource-specific interfaces/for example: – Network: the set of agents responsible of configuring switches/routers on a given path – Storage: srm. Reserve. Space / srm. Release. Space • Monitoring of resource usage INFSO-RI-508833

Example: the JRA 4 network Service Provider Enabling Grids for E-scienc. E “The Network Element should be seen as a set of services rather than a single component in the architecture. It provides user-oriented network services to the Higher Layer Middleware by abstracting the physical network services. Specifically it will allow reservation of network bandwidth and the definition of the desidered network Qo. S. This may involve mapping the higher level service requests into network specific service requests. ” [1] Network Agreement Service Network Service Provider INFSO-RI-508833

Component 2. Agreement Service Enabling Grids for E-scienc. E • Functionality: – Interacts with one or more service providers – Translates high-level service description terms (from initiator) to low level service-specific terms – Advertises the Service Provider capabilities through agreement templates. The template is an XML document that describes the contract skeleton. – Handles the agreement negotiation (agreement offer attributes from the initiator are tuned during the negotiation phase) – Provides information about: status of the agreement negotiation process and attributes of a specific agreement instance • Benefits: 1. It hides the complexity of the service providers’ interfaces from the agreement initiator (the client). 2. It exposes a single operation which can be used independently of the nature of the agreement type (create. Agreement). 3. Scalability: new Service Providers can be easily integrated by advertising the corresponding templates (clients can be notified when new templates are added). 4. No need of a-priori knowledge about services available to users. INFSO-RI-508833

Component 3. Agreement Initiator: the WMS Reservation Manager Enabling Grids for E-scienc. E • Reservation Manager: – a proposed component of the Workload Management System (WMS) [4] which handles: Simple agreements (to start with) Compound agreements (next in development roadmap) • Why integrated in the WMS? – Agreement offer submission: similar to job submission – Agreement status: information from Logging and Bookkeeping – Agreement Service discovery: a list of Agreement Services is tried (more powerful than submission to a single Agreement Service) based on matchmaking library takes advantage of the WMS internal cache – Periodic re-submission in case of failure INFSO-RI-508833

Enabling Grids for E-scienc. E • Use cases • The Grid Resource Allocation and Agreement Protocol • The g. Lite Agreement Service: – Architecture – Components • Workload Management System instrumentation • Implementation: the space reservation Agreement Service prototype • Future work INFSO-RI-508833

WMS instrumentation Enabling Grids for E-scienc. E Reserv. Helper User Interface Agreement Offer Task queue Matc h Make r Cache (ISM) Web Service interface INFSO-RI-508833 Agr. offer + Resource list + Agr. Service EPR list Reservation Manager Cache update r Logging and Bookkeeping Reserv. Cleaner

WMS Agreement Service Discovery Enabling Grids for E-scienc. E • Problem: given an agreement offer including 1. Resorce requirements 2. Resource preferences (optional) 3. Service functionality find Agreement Services that support service functionality (3) and can (indirectly) do reservation on resource instances satisfying (1) and (2) • • Matchmaking finds a list of resources: – on which user is authorised to do reservation and – satisfying resource requirements (1), preferences (2), and supporting (3) find Agreement Services: – authorized to talk to the resources in the list identified above – Supporting the corresponding allocation and reservation service provider interfaces INFSO-RI-508833

Enabling Grids for E-scienc. E • Use cases • The Grid Resource Allocation and Agreement Protocol • The g. Lite Agreement Service: – Architecture – Components • Workload Management System instrumentation • Implementation: the storage space reservation Agreement Service prototype • Future work INFSO-RI-508833

Implementation 2/2 Enabling Grids for E-scienc. E • Agreement service: – based on WS-Agreement XML Schema Definitions (GGF, GRAAP WG) –. . . but operations exposed are different (no Web Services Resource Framework - WSRF) – g. SOAP 2. 7. 0 – agreement service tested with Sto. RM (SRM v 2. 1) INFSO-RI-508833

Enabling Grids for E-scienc. E • Use cases • The Grid Resource Allocation and Agreement Protocol • The g. Lite Agreement Service: – Architecture – Components • Extension of the Workload Management System • Implementation: the storage space reservation Agreement Service prototype • Future work INFSO-RI-508833

Future work Enabling Grids for E-scienc. E • Extension of the prototype's functionality set – – – Improvement of error handling Authorization Study of deployment model Implementation of new operations Full integration with WMS Agreement monitoring through LB • Performance testing • Support of more Service Providers: – Network – computing INFSO-RI-508833

References Enabling Grids for E-scienc. E • [1] Specification of interfaces for bandwidth reservation service, EGEE deliverable DJRA 4. 1, May 2005 (http: //edms. cern. ch/document/501154) • [2] The Grid Resource Allocation and Agreement Protocol Working Group; Global Grid Forum (https: //forge. gridforum. org/projects/graap-wg). • [3] g. Lite Allocation and Reservation Architecture, EGEE JRA 1 technical report (http: //edms. cern. ch/document/508055) • [4] EGEE Middleware Architecture; EGEE Deliverable JRA 1. 1(http: //edms. cern. ch/document/476451) INFSO-RI-508833