Enabling Grids for Escienc E AstroWise and EGEE

Enabling Grids for E-scienc. E Astro-Wise and EGEE Fokke Dijkstra Donald Smits Center for Information Technology University of Groningen, NL In collaboration with Omega. CEN, Kapteyn institute, University of Groningen, NL 23 September - EGEE conference Istanbul www. eu-egee. org EGEE-III INFSO-RI-222667 EGEE and g. Lite are registered trademarks

Introduction Enabling Grids for E-scienc. E EGEE-III INFSO-RI-222667

Astro-Wise setup Enabling Grids for E-scienc. E EGEE-III INFSO-RI-222667

TARGET diagram Enabling Grids for E-scienc. E Extreme data lineage! EGEE-III INFSO-RI-222667

Enabling Grids for E-scienc. E EGEE-III INFSO-RI-222667 Astro-WISE: Services

Lofar in short Enabling Grids for E-scienc. E • Large distributed radio telescope • In NL 2008: initial roll out 20 stations – Full scale aperture synthesis array, extends 100 km • Fibre network, Software Correlator • Run as a broad common-user observatory – Making the transition now from “project” to “observatory” • Extended with European partners • Up to 4 Petabytes/year to archive EGEE-III INFSO-RI-222667

Lo. Wise Enabling Grids for E-scienc. E Code base Blue Gene New Tier 1 location Jülich - Germany Amsterdam CIT / Tape CIT – Groningen Meta data and Files Nearline storage & processing Global FS view Astro-WISE SAS Archive C=IT/O=INFN VOMS /L=CNAF pseudo. Oracle /CN=Pinco Pallacert DB /CN=proxy Export by file Tier 0 Astro-WISE view Communities - Groningen - Amsterdam - Leiden - Nijmegen - Jülich EGEE-III INFSO-RI-222667 C=IT/O=INFN VOMS /L=CNAF pseudo/CN=Pinco Pallacert /CN=proxy Lofar Communities Grid user EGEE Tier 1 Euro VO Tier 2

Astro-Wise and Grid Enabling Grids for E-scienc. E • User authentication – VOMS proxies – Proxy server – Preferably client installation § Python only! § Web portal (should work) – Astro-Wise system will obtain delegated proxy – Extend to Oracle database • Job submission – Manage collection of tasks • Storage – Support SRM EGEE-III INFSO-RI-222667

Astro-Wise job submission Enabling Grids for E-scienc. E ts c e nn k tas • g ts DPU s VOMS server task Proxy server Linux cluster nn ec UI tials e d e r c a nti s co e Lit s ls EGEE-III INFSO-RI-222667 ects Astro-Wise Database task creden Astro-Wise user conn VOMS attribute s co

Problems encountered Enabling Grids for E-scienc. E • Preferrably no g. Lite-UI installation for all users • Currently only myproxy-init really necessary – Python implementation needed § OMII-UK (NDG) seems to have one, earlier tests unsuccessful – Other option java webstart version § CCLRC version works EGEE-III INFSO-RI-222667

Job submission Enabling Grids for E-scienc. E • Image processing inherently parallel – e. g. Processing per single CCD image – One step requires global communication § Communication through database • MPI job type used for processing – No real MPI program used – Independent processes started using ssh Advantages Disadvantages • Reduces the amount of data • MPI support limited that is shipped around • Shared directory necessary (currently) • Single job, easier management • Depends on password-less ssh between WNs • No problem with varying system speeds EGEE-III INFSO-RI-222667

Future plans Enabling Grids for E-scienc. E • Investigate current state of python myproxy implementation as used by NDG • Use of SRM for Lofar storage (Sto. RM on global file system) • Further integration of GSI with Astro-Wise and Oracle • Allow Astro-Wise to store data through SRM EGEE-III INFSO-RI-222667

Final remarks Enabling Grids for E-scienc. E • Grid job submission relatively easy • Credential management stumble block for users – Obtaining certificate – Managing (proxy) certificates – Renewing certificates • Lack of portable tools – Cannot ship current g. Lite UI to everyone – UI portals may be a solution • Advantages of Grid – Standards! § Job submission § Data management § Security EGEE-III INFSO-RI-222667

Acknowledgments Enabling Grids for E-scienc. E • • Kor Begeman (implementation in Astro-Wise) Willem Jan Vriend (implementation in Astro-Wise) Danny Boxhoorn Andrey Belikov All from: EGEE-III INFSO-RI-222667 Omega. CEN Kapteyn Institute University of Groningen, NL