EUGRID Work Program Massimo Sgaravatto INFN Padova Cristina

EU-GRID Work Program Massimo Sgaravatto – INFN Padova Cristina Vistoli – INFN Cnaf as INFN members of the EU-GRID technical team

R&D Program n Work Packages (preliminary list) n Computing Fabric Management (T. Smith) Mass Storage Management (O. Barring, J. F. Baud) Wide Area Data Management (J. Jaen, B. Panzer, W. n Wide Area Workload Management (M. Sgaravatto, C. n n n Hoschek) Vistoli) Wide Area Application Monitoring (E. Futo) Application Development Testbeds (? ? ? ) Presented and discussed during the Grid Workshop (7 Mar. )

Computing Fabric Management n n “The management of the hardware and basic software required for the local computing fabric – the processors, disks, tape drives and robots, the equipment providing the local area and storage networks, operating system installation and maintenance, file systems, SANs, gateways to external networking facilities, monitoring consoles, power distribution, etc …” No preliminary draft document presented

Mass Storage Management n n “The software required to manage the mass storage system –secondary and tertiary storage space management, migration of data between tertiary and secondary storage, provision of classes of service to schedule access to the same data from processes with different priorities” Preliminary draft document presented covering HEP needs and state of the art (no details on work needed, work plan, …)

Wide Area Data Management n n n “Universal name space, efficient data transfer between sites, synchronisation of remote copies, wide-area data access/caching, interfacing to mass storage management systems” Preliminary draft document (prepared by 8 persons) presented 3 Phases n n 1 year/phase 4 -5 persons/year (preliminary assumption)

Wide Area Data Management - Phase 1 Design and prototype the basic infrastructure (lower level services) n Security n Data Access n Data Location n Data Migration n Data Replication n Metadata Publishing and Management

Wide Area Data Management – Phase 2 Leverage the lower level services delivered by Phase 1, and build higher level services on top of them (not sufficiently well defined) n Maintain sets of logically related files n Query optimisation and estimation n Access Pattern Management n Transactions n Transparent Access

Wide Area Data Management - Phase 3 n Not yet worked out in detail n n Prototype data grid enabled HEP analysis environment Provide tools for location and DB independence and seamless data integration

Wide Area Workload Management n n “Scheduling of work submitted by users Scheduling support taking account of relative priority levels, data location, automated parallelisation of work APIs and GUIs which provide users access to the distributed computing facility” Preliminary draft document presented

Wide Area Work. Load Management n n n Job description and resource specification Job “decomposition” Scheduling n Definition of scheduling policies in order to find the best match between job requirements and available resources

Wide Area Workload Management n Co-Allocation & Reservation n Resource Management n n Co-allocation necessary because many applications need simultaneously multiple resources Advance reservation necessary to guarantee endto-end Qo. S Implementing scheduling policies Services n n n Bookkeping Accounting Logging

Wide Area Application Monitoring n n “The instrumentation of applications, and tools to analyze dynamically the performance of an application in a GRID environment; automatic recognition of application-level performance and resource problems; automatic re-configuration of the application’s resources and rescheduling of the application components” Presentation on Application Monitoring problem and status (no details on work needed, work plan, …)

Wide Area Application Monitoring n n Integrate performance measurement with realtime performance visualization and adaptive control of application behavior and resource policies Migration from static resource optimization based on a posteriori measurements to closed-loop adaptive control using realtime data streams

Application Development n n “The seamless integration of existing enduser applications with GRID middleware and the development of new, grid-aware applications is an important component of any testbed and pivotal to the success of the project” Not charged to the technical group: addressed to physicists