SRM 2 2 Shaun de Witt s de

Introduction Design n Dependencies n Set-up n Requirements n Support n Upcoming n

Design n Database Centric – Heavy re-use of CASTOR core components n Thread library

Design n Secure – Uses GSI authentication n Cgsi plugin for gsoap 2. 7

Server Daemon SRM Database CASTOR Clients Design

Data Flows n n Two types of request in SRM spec Synchronous – Client

Data. Flows Two Examples n Directory Function (sync) n – e. g. srm. Mkdir,

Server CASTOR Nameserver Clients Directory Functions

Server Daemon SRM Database CASTOR Clients PUT operation (Prepare)

Server Daemon SRM Database CASTOR Clients PUT operation (Status)

Server Daemon SRM Database CASTOR Clients PUT operation (Done)

Dependencies n Since everything done using RPMs should be no trouble working then out

Dependencies n Server and Daemon in separate rpms – allows ease of deployment on

Set-Up n Two configuration files – srm 2. conf n general configuration – srm

SRM 2. conf n NOTIFYHOST – hostname where srm. Daemon is running n NOTIFYPORT

SRM 2. conf n MAXLSRETURNS/MAXLSCOUNT – Max # returns from srm. Ls, and max

SRM 2. conf n Stager mapping – SRM <VO> <STAGE_HOST> – default: n SRM

SRM 2. conf vo=alice vo=cms c 2 alice c 2 cms SRM vo=lhcb no

srm 2_storagemap. conf n Maps SRM retention policy/access latency to space token – hence

srm 2_storagemap. conf ALICE DEFAULT TAPE 1 DISK 1 TAPE 1 DISK 0 TAPE

srm 2_storagemap. conf n NOTE – Disk 0 => Garbage Collection – Disk 1

Oracle Setup n Must initially set up service classes in CASTOR – Service class

Oracle Setup n Storage. Area table: – – – – – space. Token VARCHAR

Setup ALICE example n Service Classes: n – alimdc (Tape 1 Disk 0) –

srm 2. conf #ALICE TAPE 1 DISK 1 TAPE 1 DISK 0 TAPE 0

Requirements n Suitable database and table space – Current at RAL and CERN SRM

Differences n Space Reservation – Spaces are static n No user defined spaces –

Space reservation n Permitted user issues srm. Reserve. Space – Spacetoken assigned UUID, and

Problems n Deployment problems – Related to cgsi plugin for gsoap n Changes in

Support n Supported by Giuseppe Lo Presti (CERN) and myself (RAL) – – Details

Upcoming n Change. Space. For. Files – Could be done wholly through SRM but

Slides: 42

Download presentation

SRM 2. 2 Shaun de Witt (s. de. witt@rl. ac. uk)

Introduction Design n Dependencies n Set-up n Requirements n Support n Upcoming n

Design n Database Centric – Heavy re-use of CASTOR core components n Thread library in daemon n DLF for both – Request Flow depends on request type n not all functions hit SRM database… n …but all of them hit the CASTOR database

Design n Secure – Uses GSI authentication n Cgsi plugin for gsoap 2. 7 – VOMS ready n n whatever that means Server and Daemon multithreaded – configurable # threads – Throttled number of heavyweight threads n srm. Ls and possibly srm. Change. Space. For. Files

Server Daemon SRM Database CASTOR Clients Design

Data Flows n n Two types of request in SRM spec Synchronous – Client waits for response – Typically does not go through SRM database of daemon – Server talks directly to nameserver and/or stager n Asynchronous – Request stored in SRM database. – Associated with srm. Status. Of. XXX call

Data. Flows Two Examples n Directory Function (sync) n – e. g. srm. Mkdir, srm. Rmdir n PUT function (async) – srm. Prepare. To. Put – srm. Status. Of. Put. Request – srm. Put. Done

Clients Directory Functions Server

Server CASTOR Nameserver Clients Directory Functions

Server Daemon SRM Database CASTOR Clients PUT operation (Prepare)

Server Daemon SRM Database CASTOR Clients PUT operation (Status)

Server Daemon SRM Database CASTOR Clients PUT operation (Done)

Dependencies n Since everything done using RPMs should be no trouble working then out CASTOR: n CGSI-GSOAP plug-in: n Globus Toolkit n VOMS n Oracle n – 2. 1. 1 or later – 2. 7 -1. 1. 14 or later – vdt_globus_essentials-VDT 1. 2. 2 rh 9_LCG-2 – glite-security-voms-api-c-1. 6. 16 – as per castor stager

Dependencies n Server and Daemon in separate rpms – allows ease of deployment on separate nodes n Both have same dependencies – srm. Copy is daemon side operation that needs security libraries

Set-Up n Two configuration files – srm 2. conf n general configuration – srm 2_storagemap. conf n n Maps storage areas (space tokens) to service class Oracle – – – Script to create tables Need to set up tnsnames, ORASTAGERCONFIG Need to pre-populate storage. Area table

SRM 2. conf n NOTIFYHOST – hostname where srm. Daemon is running n NOTIFYPORT – port numbers which daemon is listening on n one for GET/PUT one for COPY PINTIME – Garbage Collection Weighting – Not implemented by default in CASTOR

SRM 2. conf n MAXLSRETURNS/MAXLSCOUNT – Max # returns from srm. Ls, and max allowable offset n prevents n too heavy load on srm SERVERTHREADS/LSTHREADS – SERVERTHREADS is max # of threads in srm. Server – LSTHREADS is max # of Ls requests n avoids accidental DNS attacks

SRM 2. conf n Stager mapping – SRM <VO> <STAGE_HOST> – default: n SRM STAGE_HOST <blah> n For single castor instance sites

SRM 2. conf vo=alice vo=cms c 2 alice c 2 cms SRM vo=lhcb no vo info c 2 lhcb c 2 public

srm 2_storagemap. conf n Maps SRM retention policy/access latency to space token – hence to service class n Format – <VO> TAPENDISKM <service class>

srm 2_storagemap. conf ALICE DEFAULT TAPE 1 DISK 1 TAPE 1 DISK 0 TAPE 0 DISK 1 default alimdc wan default dteam

srm 2_storagemap. conf n NOTE – Disk 0 => Garbage Collection – Disk 1 => No Garbage Collection – Tape 1 => Migration – Tape 0 => No Migration n All should have default specified – Request with no space token or retention. Policy info.

Oracle Setup n Must initially set up service classes in CASTOR – Service class name reused n Populate Storage. Area Table – by hand – VO production manager run srm. Reserve. Space, then update entry showing data available n preferred since gets association between storage area and VO for free

Oracle Setup n Storage. Area table: – – – – – space. Token VARCHAR 2(2048) token. Description VARCHAR 2(2048) storage. Lifetime INTEGER svc. Class VARCHAR 2(2048) storage. Type NUMBER id INTEGER PRIMARY KEY srm. User INTEGER parent INTEGER storage. Status INTEGER

Setup ALICE example n Service Classes: n – alimdc (Tape 1 Disk 0) – recovery (Tape 0 Disk 1) – wan (Tape 1 Disk 0) – default (Tape 1 Disk 0) – spare (Tape 0 Disk 0)

srm 2. conf #ALICE TAPE 1 DISK 1 TAPE 1 DISK 0 TAPE 0 DISK 1 default not supported alimdc recovery default

Requirements n Suitable database and table space – Current at RAL and CERN SRM uses same database instance as stager – Unclear of load, may need its own oracle instance n Suitable port open – 8443 is normally used

Differences n Space Reservation – Spaces are static n No user defined spaces – Only suitable VOMS role allowed to reserve space n Pinning – Pinning is advisory – Pinning represents GC weight

Space reservation n Permitted user issues srm. Reserve. Space – Spacetoken assigned UUID, and set to space requested n – – – UUID assigned initially since this help Tier-1 sites Description can be meaningful Alert sent to DLF Manual update of space table once disks and pools are ready.

Problems n Deployment problems – Related to cgsi plugin for gsoap n Changes in Standard – Things missed and inconsistencies n External pressure – Last three months – optimistically 25% of time

Support n Supported by Giuseppe Lo Presti (CERN) and myself (RAL) – – Details to be worked out Informal e-mails should get reasonable response n but need to integrate into tracking s/w at CERN or RAL? – srm mailing list might be resurrected for support n n On-site support for deployment at other sites to be discussed Client tools being tested now.

Upcoming n Change. Space. For. Files – Could be done wholly through SRM but easier internally to CASTOR n Update. Space – Needs reimplementing similar to Reserve. Space n Lots and lots of bug fixes – Finally getting lots of testers! n Per VO threading – To try to stop 1 VO killing everyone n 2 weeks intensive effort coming up (external influences permitting)