Catching Up With XRoot D Up To 4

Catching Up With XRoot. D Up To 4. 4 XRoot. D Workshop Tokyo Stanford University/SLAC November 8, 2016 Andrew Hanushevsky, SLAC http: //xrootd. org

R 4. 0 Highlights in 2014 IPv 6 support Public Private Networking HTTP Plug-in Remote Debugging New Client Many other smaller improvements November 8, 2016 XRoot. D Workshop Tokyo 2

Since R 4. 0 There were three feature releases n R 4. 1 (24 -Nov-14) n XRoot. D Workshop @ UCSD (27 -Jan-16) R 4. 2 (29 -Apr-15) n R 4. 3 (25 -Feb-16) n R 4. 4 (21 -Jul-16) n On the cusp R 4. 5 (xx-Nov-16) n Will be covered Thursday November 8, 2016 XRoot. D Workshop Tokyo 3

New In R 4. 1 (24 -Nov-14) Multiple version support Subordinate clusters Multiple checksum support TCP keepalive support Object ID support n Useful only in 4. 2 Many other smaller features November 8, 2016 XRoot. D Workshop Tokyo 4

Multiple Version Support Allows a no reinstall back out n Done with plug-in shared library versioning n lib. Xrdxxx-n. so n The n is the version number n Currently, it’s 4 n Run-time search for compatible library n Caution when specifying a plug-in library n Do not include the dash and version number! Do not include n November 8, 2016 No need to change configuration file XRoot. D Workshop Tokyo 5

Subordinate Clusters Accept a discreet cluster as a logical unit all. subcluster [of] a. slac. stanford. edu Redirector SC Manager a. slac. stanford. edu Data Server b. slac. stanford. edu Data Server Use cases: - Increase DFS bandwidth - Manage non-shared resources - Separate disparate resources November 8, 2016 XRoot. D Workshop Tokyo Ceph. FS, GPFS, Lustre, etc 6

Multiple Checksum Support Checksum directive extended xrootd. chksum [chkcgi] [max num] digest [path [args]] digest: algorithim [digest] Client can specify checksum type wanted n CGI element: cks. type=algorithm Different checksum values now recorded n No need to recompute value every time November 8, 2016 XRoot. D Workshop Tokyo 7

TCP Keepalive Support OS keep alive mechanism now the default Used on client and server sides n xrd. network directive extended n xrd. network [kaparms idle[, itvl[, cnt]] [[no]keepalive] idle itvl cnt November 8, 2016 The time the connection needs to remain idle before TCP starts sending keepalive probes. The idle value may be optionally suffixed by m for minutes, s for seconds (the default), or h for hours. The time between individual keepalive probes. The itvl value may be optionally suffixed by m for minutes, s for seconds (the default), or h for hours. The maximum number of keepalive probes TCP should send before dropping the connection. XRoot. D Workshop Tokyo 8

Object ID Support Non-slash paths can now be exported all. export {path | *[? ]} [[no]lock] [oss_options] * Allow arbitrary object identifiers (i. e. names that do not start with a slash). *? Allow arbitrary object identifiers (i. e. names that do not start with a slash). Inspect the names for CGI information and, if present, separate it from the object identifier (i. e. characters before the question mark) before passing the object name and CGI information to the file system plugin. Added for Ceph support n Introduced in 4. 2 November 8, 2016 XRoot. D Workshop Tokyo 9

New In R 4. 2 (29 -Apr-15) Ceph block store plug-in I/O Throttling plug-in Blacklisting or whitelisting of sites Object store extended attributes Full Python bindings Dynamic remote debugging control Many other smaller features November 8, 2016 XRoot. D Workshop Tokyo 10

Ceph block store plug-in Implemented as on OSS plug-in n Works at storage system level n Allows access via Castor, EOS, HTTP, XRoot. D Pros Full benefit of the distributed object store n No file size limit n Embedded striping and parallel I/O n Cons n No directory listing November 8, 2016 XRoot. D Workshop Tokyo 11

Enabling & Using Ceph Enabling Ceph plug-in ofs. osslib lib. Xrd. Ceph. so ofs. xattrlib lib. Xrd. Ceph. Xattr. so all. export *? Usage example xrdcp myfileroot: //myserver/myfile [1000 MB/1000 MB][100 n%][======][17. 24 MB/s] #rados ls | grep myfile. 000000001 # xrdcp root: //myserver/myfile 2 [1000 MB/1000 MB][100 n%][======][16. 38 MB/s] November 8, 2016 XRoot. D Workshop Tokyo 12

Ceph Caveats You must have Ceph installed Along with working Ceph configuration n Plug-in uses lib. Rados. so to access Ceph n RAL is currently deploying this n Also used at CERN for Castor For more information n https: //indico. cern. ch/event/330212/contributions/1718786/att achments/642384/883834/Ceph. Plugin. For. Xroot. pdf November 8, 2016 XRoot. D Workshop Tokyo 13

I/O Throttling Plug-in Allows site to limit I/O access to disk n Useful for throttling external access n Used in federated environments via proxy server n Configured via xrootd. fslib directive n xrootd. fslib throttle default n And specific throttling directives n throttle [concurrency concur] [data rate] n throttle. trace options n November 8, 2016 options: [all] [off|none] [bandwidth] [ioload] [debug] XRoot. D Workshop Tokyo 14

Black or White-Listing sites Disallow entry to a cmsd based federation n Controlled via directives n cms. {black|white}list [check sec] [listfilepath] n And control file listfilepath n hostspec | [hpfx]*[hsfx] [redirect target] n n hostspec: fulldnsname | [ipv 6 address] | ipv 4 address target: fulldnsname[+]: port [target] Disallows (blacklist) or allows (whitelist) login n Optionally redirects login elsewhere n November 8, 2016 XRoot. D Workshop Tokyo 15

Object Store Extended Attributes Using extended attributes in object stores n Controlled via plug-in directive n ofs. xattrlib {osslib | path} [parms] n Specifically added to support Ceph attributes n Can be used with any backend storage service n November 8, 2016 However an xattr plug-in needs to be provided XRoot. D Workshop Tokyo 16

New In R 4. 3 Part I (25 -Feb-16) Mostly performance oriented changes n Multiple event loops in client n pss. setop Parallel. Evt. Loop num (default 3) n Fast path external DB stat() lookups n oss. statlib [-2] [non 2 n] [preopen] path [parms] n November 8, 2016 Added to improve DPM support XRoot. D Workshop Tokyo 17

New In R 4. 3 Part 2 (25 -Feb-16) More performance oriented changes n Allow client to pass reason for a retry n New CGI element (triedrc=reason) on open URL n reason: enoent | ioerr | fserr | srverr | resel n Used by client to reduce recovery time n Jumbo I/O buffers n xrd. buffers [maxbsz bsz] memsz [rint] n n n November 8, 2016 New undocumented option maxbsz allows jumbo buffers Has performance impacts when greater than 2 MB Added specifically for improved Ceph performance XRoot. D Workshop Tokyo 18

New In R 4. 4 (21 -July-16) Full metalink support Public cloud networking support Object ID authorization Logging plug-in support Access control test tool Numerous performance improvements November 8, 2016 XRoot. D Workshop Tokyo 19

Meta-Link Files XML file that describes one or more data files available for access n Meta-link file identified by dot suffix n metalink (v 3) or meta 4 (v 4 & incompatible w/ v 3) Conjured up by the http community n Provides multiple sources for downloads n ALICE and ATLAS want to use it for direct access n November 8, 2016 Provides places where to open and read the file XRoot. D Workshop Tokyo 20

Sample Metalink V 4 File <? xml version="1. 0" encoding="UTF-8"? > <metalink xmlns="urn: ietf: params: xml: ns: metalink"> <published>2009 -05 -15 T 12: 23 Z</published> <file name="example. ext"> <size>1447</size> <identity>Example</identity> <version>1. 0</version> <language>en</language> <glfn name="/atlas/rucio/example. txt"></glfn> (xrootd specific foreign tag) <description> A description of the example file for download. </description> <hash type="sha-256">3 d 6 fece 8033 d 146 d 8611 eab 4 f 032 df 738 c 8 c 1283620 fd 02 a 1 f 2 bfec 6 e 27 d 590 d</hash> <url location="de" priority="1">xroot: //xrd. example. org/example. ext</url> <url location="de" priority=“ 3">xroot: //xrd. slac. stanford. edu/example. ext</url> <url location="fr" priority="1">http: //example. com/example. ext</url> <metaurl mediatype="torrent" priority="2">http: //example. com/example. ext. torrent</metaurl> </file> </metalink> November 8, 2016 XRoot. D Workshop Tokyo 21

XRoot. D Metalink Support I When client sees a metalink file as a source xrdcp /path/file. meta 4 target n xrdcp root: //location//file. meta 4 target n ROOT: TFile. Open(root: //location//file. meta 4); n n location: host[: port] or word localfile Reads it from local disk or remote server Parses the file to extract information Uses result to construct a virtual redirector November 8, 2016 XRoot. D Workshop Tokyo 22

XRoot. D Metalink Support II Virtual redirector drives future access All “root: //” and “xroot: //” sources extracted n Placed in the correct priority order n Tried as needed to perform data access n n Just like when using a real redirector All relevant tags are supported hash, size n url (including type, preference and priority) n November 8, 2016 XRoot. D Workshop Tokyo 23

XRoot. D Specific Foreign Tag Special foreign tag for federation support n <glfn name=“global-logical-name"></glfn> When encountered and envar has been set XRD_REDIRECTOR envar or n XRD_Redirector root. rc variable n Additional file entry added at the end n xroot: //XRD_REDIRECTOR/global-logical-name November 8, 2016 XRoot. D Workshop Tokyo 24

Public Cloud Support Cloud host names registered with fixed NAT public address Public DNS All nodes in cloud only have a private IP address NAT Box Fixed 1 -to-1 Public-Private DNS Cloud host names registered with private address xrd. network [no]rpipa n Do or do not resolve private IP addresses n Resolution required in this case so need “rpipa” n November 8, 2016 Default is norpipa XRoot. D Workshop Tokyo 25

Object ID Authorization Can specify object IDs in auth. DB file idtype id { {path | objectid} privs | tname } [ • • • ] ] idtype: g | h | n | t | u privs: plets | -plets | plets-plets: { a | d | i | k | l | n | r | w } [ plets ] Object ID’s need not start with a slash The escape symbol is used for object ID’s n Addresses long standing issue for Ceph n November 8, 2016 XRoot. D Workshop Tokyo 26

Logging Plug-in Support Sites can provide a logging plug-in n Allows messages to be sent elsewhere n Messages can be custom-formatted as well Requested via –l command line option -l [=]fn | - | @lib[, bsz=sz][, cse={0|1|2}][, logfn=[=]fn] Plug-in exists in shared library lib n See Xrd. Sys. Log. PI. hh on how to write one n This may impact performance! November 8, 2016 XRoot. D Workshop Tokyo 27

Access Control Test Tool Distributed as source and must be compiled Syntax: testaccess [-c cfn] [d] [-t] [user host op path] Where: -c sets the config file name (default is. /acc. cf) -d turns on debugging. -t turns on tracing. user is the requesting username host is the user's host name op is the requested operation and is one of: cr - create mv - rename st - status lk - lock rd - read wr - write ls - readdir rm - remove ? - display privs November 8, 2016 XRoot. D Workshop Tokyo 28

On The Way! Release 4. 5 n Almost done Release 4. 6 n Already planned Details on Thursday! November 8, 2016 XRoot. D Workshop Tokyo 29

That’s All! Questions? November 8, 2016 XRoot. D Workshop Tokyo 30