Enabling Grids for Escienc E DPM status and

Enabling Grids for E-scienc. E DPM status and plans Jean-Philippe Baud JRA 1 All-hands meeting 23 March 2006

Outline Enabling Grids for E-scienc. E • • • Introduction Software status Virtual Ids and VOMS integration Access Control Lists Storage Resource Managers User and site requirements Priority list Roll-out plan Support Summary LFC/DPM status and plans

Introduction Enabling Grids for E-scienc. E • DPM is the Light Weight Disk Pool Manager (possible solution for Tier 2 s, replacing Classic SEs) – – – Easy to install and configure Low effort for ongoing maintenance Easy to add/remove resources (pools, filesystems) Secure (GSI + VOMS) Interfaces: SRM v 1, SRM v 2, RFIO and grid. FTP • DPM is currently used at 27 sites by 32 VOs – GRIF, INFN, ITEP, JINR, NIKHEF, SCOTGRID, Montreal … LFC/DPM status and plans

Software status (1) Enabling Grids for E-scienc. E • Version 1. 4. 5 released in LCG 2. 7. 0 – – – automatic reconnection to database when connection lost Support for disk servers spread over multiple domains Support for TURL syntax in RFIO (rfio: //server: port/dpm/domain/) Script for migration from Classic SE to DPM A few bug fixes (srm. Ls and bug affecting transfer performance between DPM and d. Cache) LFC/DPM status and plans

Software status (2) Enabling Grids for E-scienc. E • Version 1. 4. 6 (1 st February 2006) – Simpler configuration files – Better protection against bad parameters • Version 1. 5. 4 (27 th February 2006) – Support for Virtual Ids and VOMS in DPM (Jiri Kosina) § Access to DPM files is done using Virtual Ids § Access to non-DPM files (logs) is done using standard grid mapfiles § Backward compatibility: old clients can contact new servers and new clients can contact old servers LFC/DPM status and plans

Virtual Ids and VOMS integration Enabling Grids for E-scienc. E • DNs are mapped to virtual UIDs: the virtual uid is created on the fly the first time the system receives a request for this DN (no pool account) • VOMS roles are mapped to virtual GIDs • A given user may have one DN and several roles, so a given user may be mapped to one UID and several GIDs • Currently only the primary role is used in DPM • Support for normal proxies and VOMS proxies • Administrative tools available to update the DB mapping tables: – To create VO groups in advance – To keep same uid when DN changes – To get same uid for a DN and a Kerberos principal LFC/DPM status and plans

Access Control Lists Enabling Grids for E-scienc. E • DPM support Posix ACLs based on Virtual Ids – Access Control Lists on files and directories – Default Access Control Lists on directories: they are inherited by the sub-directories and files under the directory • Examples – dpns-mkdir /dpm/cern. ch/home/dteam/jpb – dpns-setacl -m d: u: : 7, d: g: : 7, d: o: 5 /dpm/cern. ch/home/dteam/jpb – dpns-getacl /dpm/cern. ch/home/dteam/jpb # file: /dpm/cern. ch/home/dteam/jpb # owner: /C=CH/O=CERN/OU=GRID/CN=Jean-Philippe Baud 7183 # group: dteam user: : rwx group: : r-x #effective: r-x other: : r-x default: user: : rwx default: group: : rwx default: other: : r-x LFC/DPM status and plans

Storage Resource Managers Enabling Grids for E-scienc. E • DPM provides srmv 1 and srmv 2 functions – SRM v 1: § ping, get. Protocols, get, put, get. File. Meta. Data, get. Request. Status, set. File. Status – SRM v 2 (http: //sdm. lbl. gov/srm-wg/srm. v 2. 1. 1. modified. wsdl): § Space Management functions: partial implementation § Permission functions: Set. Permission ok, Check. Permission impossible to implement with wsdl 2. 1. 1 § Directory functions: recursive option not supported § Transfer functions: Suspend. Request, Resume. Request not implemented – Abort. Files (v 2) being tested – srm. Copy method being implemented (v 1 and v 2) – Compatibility tests between CASTOR and DPM SRM v 2 run by Jiri Mencak LFC/DPM status and plans

User requirements (1) Enabling Grids for E-scienc. E • From “Summary of Open Issues reported by LHC experiments” (Flavia Donno) – – – – Address library conflicts: RFIO, gsoap, globus, openssl Improvement of POOL-LFC interface Checking file integrity/validity after copy (filesize/checksum) Perl/Python interface to GFAL/lcg_util/DPM Read-only mirrors of LFC catalogues Efficient strategy for selecting file catalogue instance More attributes on replica entries Efficient strategy for choosing best file replica LFC/DPM status and plans

User requirements (2) Enabling Grids for E-scienc. E • From “Summary of Open Issues reported by LHC experiments” (Flavia Donno) – – – Support for srm. Copy and srmcp in DPM Better implementation of bulk deletion of file replicas Reliable registration service Global space reservation relative path (VO HOME) Support for disk quota management LFC/DPM status and plans

Site requirements Enabling Grids for E-scienc. E • • CLI for dpns_listreplicax My. SQL backups DPM pool hierarchy (+ default pool) Drain of a DPM pool or filesystem (automatic) replication inside a DPM pool Limit number of streams per disk server Support for RFIO port range SRM test suite LFC/DPM status and plans

Other features Enabling Grids for E-scienc. E • Integration with Information System – Report of DPM pool capacity and free space § Development done by UK (Graeme) § Development lead by Patrick Fuhrmann (common solution for d. Cache and DPM) § Paul Millar (Glasgow) proposes also his contribution • Integration with Fabric Monitoring – Lemon sensors ? • Support for ROOTD/XROOTD protocols in DPM • Use of grid. FTP 2 with DPM (Yaodong Cheng’s prototype) • Streaming mode (SRM v 3) • implementation of a migrator/recaller to either recall/migrate files automatically between Tier 1 and Tier 2 or interface to a tape/DVD backend LFC/DPM status and plans

Priority list Enabling Grids for E-scienc. E • https: //twiki. cern. ch/twiki/bin/view/LCG/Summary. Open. I ssues. TF • https: //twiki. cern. ch/twiki/bin/view/LCG/Data. Manageme nt. Work. Plan • List of priorities set at the workshop in Mumbai: – – – SRM v 2 compatibility tests between CASTOR/d. Cache/DPM Perl/Python interfaces Drain of filesystems srm. Copy and support for srmcp in DPM Global Space Reservation in DPM LFC/DPM status and plans

Current developments Enabling Grids for E-scienc. E • Better compatibility between the different SRM v 2 implementations (ongoing). See https: //srm. fnal. gov/twiki/bin/view/Srm. Project/Srm. V 2 Status. Codes • File replication inside DPM (being tested) • Drain of filesystems (being tested) • Support for RFIO port range (done) • Port to Solaris on Intel platforms (Eamonn Kenny) • Use of Kerberos 5 instead of GSI (LIP) LFC/DPM status and plans

Roll-out plan Enabling Grids for E-scienc. E • Release 1. 4. 5 distributed in LCG 2. 7. 0 – Need to run migration script for both LFC and DPM • Recommend sites upgrade directly to 1. 5. 4 – Need again to run a migration script for DPM – Release 1. 5. 4 is part of g. Lite 3. 0 (March/April 2006) • The clients are backward compatible with LCG 2. 6. 0 except for ACLs • The migration scripts are automatically run by Yaim • We would like to provide bug fixes only for 1. 5. 4 to avoid branching • Installation and upgrade procedures are on the Wiki page LFC/DPM status and plans

Support Enabling Grids for E-scienc. E • First level support – support@ggus. org • Second level support – hep-service-dpm@cern. ch – hep-service-lfc@cern. ch • https: //twiki. cern. ch/twiki/bin/view/LCG/Data. Manageme nt. Documentation – Admin guides – Troubleshooting LFC/DPM status and plans

Summary Enabling Grids for E-scienc. E • Software is stable but • Many features requested by experiments with critical or high priority LFC/DPM status and plans