CTA CERN Tape Archive Future of archival storage

CTA: CERN Tape Archive Future of archival storage at CERN Vladimir Bahyl, Germán Cancio, Eric Cano, Michael Davis, Julien Leduc, Cristina Moraru, Steven Murray Tuesday 12 Sep 2017 pre-GDB Storage 1

Overview What is EOS+CTA? • Making the most out of tape • Status and Timeline • Tuesday 12 Sep 2017 pre-GDB Storage 2

What is EOS+CTA? • EOS plus CTA is a “drop in” replacement for CASTOR Future deployments with EOS plus CTA Current deployments with CASTOR EOS es Experiment s CASTOR July 2017 Files Tape libraries Fi Files Fi le Fi Files Experiment s le l Fi le s EOS + CTA CERN SKA visit Files Tape libraries 3

EOS+CTA architecture CTA is: • Natural evolution of CASTOR • A tape backend for EOS • A preemptive tape drive between disk and tape • CTA front-end Tape files appear in EOS namespace as replicas. EOS workflow engine glues EOS to CTA metadata EOS disk server Tuesday 12 Sep 2017 Tape library Scheduling information and tape file locations Xrootd based recalls (not SRM) Tape file catalogue, archive / retrieve queues, tape drive statuses, archive routes and mount policies Archive and retrieve requests rts A clean separation Archive and retrieve requests po • EOS namespace and redirector Central CTA instance Re scheduler One EOS instance per experiment Files CTA CTA tape server tapeserver tape (taped) pre-GDB Storage Tape drive Files 4

Making the most out of tape • Single step mount scheduling for tape+drive • • Mount preemption • • • Avoids choosing tapes that no drive will serve when writing => improves write mount latency Replace handcrafted/scripted drive dedications Allow full utilization of drives all the time Background repack/verify user will yield to user mounts Many possible combinations (priority based) Recommended access order • • Optimizes read order to minimize seek time 4 x – 5 x speedups for a typical O(100 s) files recall mount Backport to CASTOR coming soon … but enterprise drives only Tuesday 12 Sep 2017 pre-GDB Storage 5

Currently achieved Preliminary EOS-CTA integration • Archive and retrieve files to/from EOS • • Request queueing Decentralized single step scheduling • Administrative interface • • Drive control Tape management File routing to tape Recommended access order (RAO) prototype • All drive features from CASTOR • Tuesday 12 Sep 2017 pre-GDB Storage 6

Currently achieved Preliminary EOS-CTA integration • Archive and retrieve files to/from EOS • [root@tpsrv 600 ~]# cta drive ls library drive host desired request status since vid tapepool files MBytes MB/s session age IBM 1 JB I 1 JB 0402 tpsrv 033 Up Free 4722 31 IBM 1 JB I 1 JB 0404 ~]# tpsrv 034 Free 4466 2 [root@localhost cta drive ls Up [root@tpsrv 600 ~]# cta tapepool ls -h IBM 1 JB I 1 JB 0405 tpsrv 068 Up Free 4386 32 [root@p 06253947 b 39467 eos ls -y /eos/dev/test/gcancio/201709011300 library drive host desired ~]# request status since vid tapepool files MBytes MB/s session age name I 1 JB 0406 # partial tapes c. user m. host IBM 1 JB Up Archive - Transfer Free c. host 4705 -c. time - 0. 04 33 [root@localhost ~]# cta encrypt showqueues -h […] tpsrv 072 VLSTK 10 VDSTK 11 tpsrv 01 Up 15 V 01003 ctasystest 519 - 0. 53 0 supply_IBM 1 JB 1 false vlado tpsrv 600. cern. ch Mon Jun 12 11: 10: 40 2017 vlado tpsrv 600. cern. ch Mon IBM 1 JB I 1 JB 0407 tpsrv 071 Up Free 4611 30 type tapepool vid files queued 124 oldest age priority age cur. d 1: : t 0 -rw-r--r-1 gcancio c 3 Down 1 - Sep 1 13: 00 testfile-1005 VLSTK 10 VDSTK 12 tpsrv 02 Down queued -MBytes -min - max drives -cur. mounts 3 supply_tpsrv 601 1 false vlado tpsrv 600. cern. ch Wed Aug 30 16: 14: 58 2017 vlado tpsrv 600. cern. ch Wed IBM 1 JB I 1 JB 0701 tpsrv 073 Up Free 4589 112 Archive ctasystest 194 0. 20 42 1 1 d 1: : t 0 -rw-r--r-1 gcancio c 3 1 Sep 1 13: 01 testfile-10050 supply_tpsrv 602 1 false vlado tpsrv 600. cern. ch Wed Aug 30 16: 14: 58 2017 vlado tpsrv 600. cern. ch Wed IBM 1 JB I 1 JB 0702 tpsrv 037 Up Free 4642 2 d 1: : t 0 -rw-r--r-1 gcancio c 3 1 Sep 1 13: 16 testfile-100500 supply_tpsrv 603 1 false vlado tpsrv 600. cern. ch Wed Aug 30 16: 14: 58 2017 vlado tpsrv 600. cern. ch Wed IBM 1 JB I 1 JB 0703 tpsrv 038 Up Free 4587 3 d 1: : t 0 -rw-r--r-1 gcancio c 3 1 Sep 1 13: 16 testfile-100501 supply_tpsrv 604 1 false vlado tpsrv 600. cern. ch Wed Aug 30 16: 14: 58 2017 vlado tpsrv 600. cern. ch Wed IBM 1 JB I 1 JB 0705 tpsrv 065 Up Free 4681 43 d 1: : t 0 -rw-r--r-1 gcancio c 3 1 Sep 1 13: 16 testfile-100502 supply_tpsrv 605 1 false vlado tpsrv 600. cern. ch Wed Aug 30 16: 14: 58 2017 vlado tpsrv 600. cern. ch Wed IBM 1 JB I 1 JB 0706 tpsrv 035 Up Free 4810 20 d 0: : t 1 -rw-r--r-1 gcancio c 3 1 Sep 1 13: 16 testfile-100503 supply_tpsrv 606 1 false. Up 1 vlado Wed 30 -16: 14: 58 2017 - vlado Wed IBM 1 JB I 1 JB 1003 tpsrv 045 - tpsrv 600. cern. ch 4736 - Aug - tpsrv 600. cern. ch 27 d 1: : t 0 -rw-r--r-gcancio c 3 Free 1 Sep 1 13: 16 - testfile-100504 supply_tpsrv 607 1 false vlado tpsrv 600. cern. ch Wed Aug 30 16: 14: 58 2017 vlado tpsrv 600. cern. ch Wed IBM 1 JB I 1 JB 1004 tpsrv 046 Up Free 4582 22 d 1: : t 0 -rw-r--r-1 gcancio c 3 1 Sep 1 13: 16 testfile-100505 supply_tpsrv 608 1 false vlado tpsrv 600. cern. ch Wed Aug 30 16: 14: 59 2017 vlado tpsrv 600. cern. ch Wed IBM 1 JB I 1 JB 1005 tpsrv 036 Up Free 4641 10 d 0: : t 1 -rw-r--r-1 gcancio c 3 1 Sep 1 13: 16 testfile-100506 supply_tpsrv 609 1 false. Up 1 vlado Wed 30 -16: 14: 59 2017 - vlado Wed tpsrv 601 L 601 D 1 tpsrv 601 - tpsrv 600. cern. ch 4484 - Aug - tpsrv 600. cern. ch 17 d 0: : t 1 -rw-r--r-gcancio c 3 Free 1 Sep 1 13: 16 - testfile-100507 supply_tpsrv 610 1 false vlado tpsrv 600. cern. ch Wed Aug 30 16: 14: 59 2017 vlado tpsrv 600. cern. ch Wed tpsrv 601 L 601 D 2 tpsrv 601 Up Free 4573 12 […] test_gcancio 1 false. Up gcancio - tpsrv 600. cern. ch Tue tpsrv 601 L 601 D 3 tpsrv 601 Free 4609 - Jun 13 -13: 47: 31 - 2017 -gcancio - tpsrv 600. cern. ch 1 test_jleduc 1 false. Up jleduc - tpsrv 600. cern. ch Wed tpsrv 601 L 601 D 4 tpsrv 601 Free 4446 - Jul 19 -15: 54: 01 - 2017 - jleduc - tpsrv 600. cern. ch 75 test_steve 1 false. Up vlado - tpsrv 600. cern. ch Mon tpsrv 601 L 601 D 5 tpsrv 601 Free 4440 - Jun 12 -11: 09: 30 - 2017 - vlado - tpsrv 600. cern. ch 19 test_unit_513 1 false. Up vlado - tpsrv 035. cern. ch Fri tpsrv 601 L 601 D 6 tpsrv 601 Free 4455 - May 26 -16: 01: 40 - 2017 - vlado - tpsrv 035. cern. ch 99 test_vlado 1 false. Up vlado - tpsrv 035. cern. ch Mon tpsrv 601 L 601 D 7 tpsrv 601 Free 4472 - May 29 -16: 38: 07 - 2017 - vlado - tpsrv 035. cern. ch 61 tpsrv 601 L 601 D 8 tpsrv 601 Up Free 4558 5 tpsrv 601 L 601 D 9 tpsrv 601 Up Free 4520 28 tpsrv 602 L 602 D 1 tpsrv 602 Up Free 4474 1 tpsrv 602 L 602 D 2 tpsrv 602 Up Free 4474 5 tpsrv 602 L 602 D 3 tpsrv 602 Up Free 4363 20 tpsrv 602 L 602 D 4 tpsrv 602 Up Free 4478 24 • Request queueing Decentralized single step scheduling • Administrative interface • • Drive control Tape management File routing to tape Recommended access order (RAO) prototype • All drive features from CASTOR • Tuesday 12 Sep 2017 pre-GDB Storage 7 f. J A A A A A J J J M M

Upcoming features • • • Streamlining and extension of EOS-CTA interface (from scripts to RPC) Object store performance optimization Repack & verification Mount preemption Library support rework (currently using the one of CASTOR) EOS-CTA catalogue synchronizations • Reconciliation & tape based disaster recover Tuesday 12 Sep 2017 pre-GDB Storage 8

Timeline Q 4 2017 Additional and redundant backups of AFS/NFS and LEP data Q 2 2018 Ready for small experiments Volunteer experiments to test are more than welcome! Tuesday 12 Sep 2017 pre-GDB Storage Q 4 2018 Ready for LHC experiments CTA and CASTOR use the same tape format. Only metadata will need to be migrated. No files will need to be copied between tapes. 9