Repack 2 Felix Ehm CERN IT Felix Ehm

Repack 2 Felix Ehm CERN / IT Felix Ehm Castor Face 2 Face – November 2006

Content What is Repack? The old Repack Design and Features of the new Repack 2 Current Scenario at CERN Tests and Performance Felix Ehm Castor Face 2 Face – November 2006

What is Repack “Repack is a synonym for a copy process which moves all of a tape’s data onto temporary disk storage and then rewrite it to another tape. ” The reasons Prevent data loss by reaching tape’s mechanical lifetime • Mounts • Reported read errors by RTCPD Stack data on high-density tapes (reduce number of tapes) • e. g. 2 x STK 9940 B (200 GB)-> 1 IBM 3592 J 1 A (500 GB) Move data to more durable media Release tapes for reuse Optimise tape space usage Felix Ehm Castor Face 2 Face – November 2006

What is Repack Optimization of tape space usage ‘Resurrection’ of space, which is not marked as valid in the Name. Server • e. g. user deletes file(s) Name. Server Table VID FSEQ Filename Tape 1 1 File 1 Tape 1 2 File 2 Tape 1 3 File 3 Tape 1 4 File 4 Tape 1 5 File 5 deleted valid data Tape 1 1 2 3 4 5 invalid data Repack 2 Tape 2 4 free space BOT Felix Ehm EOT Castor Face 2 Face – November 2006

The old Repack Mismatch to new CASTOR 2 architecture Only available for CASTOR 1 Files were staged and written in sets of segments (depend on stage_util_max_stcp_per_request) up to several times of mounting for migration for one tape to repack tapes were explicitly mounted for writing files No defragmentation of files (around 22. 000 !) CASTOR 1 migration policy > 1 mount for one file No maintenance Stateful, process takes up to 6 -8 hours for 200 G (9940 B tape) Felix Ehm Castor Face 2 Face – November 2006

Design and Features of the new Repack 2 Design I Client-Server Architecture Using existing functionality from Castor II • • Common recall / migration procedure Easy to maintain (you know CASTOR 2, you know Repack ) Parallel writing / reading from tape Uses Stager API for interface Stateless components • Process states is kept in DB (Oracle/My. SQL) Components are multi-threaded Multi-Stager ability Independent from Stager machine Felix Ehm Castor Face 2 Face – November 2006

Design and Features of the new Repack 2 Design II Tape Server Send Request Repack. Client VMGR validates tape Repack. Server repack files Stager triggers Repack. Client Get files from tape Name. Server Recaller Migrator Update location of file Data transfer Diskserver Felix Ehm Castor Face 2 Face – November 2006

Design and Features of the new Repack 2 Features I Adopted to CASTOR 2 No direct intervention with mount/umount of tapes Recall / migration of files is up to Stager Defragmentation of files Usage of DLF for logging Repack has no limit in repacking tapes Monitored repack process ====================================== vid cuuid total staging migrating status ------------------------------------------------------------------L 30069 44 e 08287 -0000 -1000 -b 044 -8861613 b 0000 1395 0 STAGING L 30214 44 e 08291 -0000 -1000 -9 bce-dbb 2 d 4100000 0 30 1568 MIGRATING P 01538 44 fc 5286 -0000 -1000 -88 f 8 -cde 07 a 2800000 2120 0 0 START ====================================== Felix Ehm Castor Face 2 Face – November 2006

Design and Features of the new Repack 2 Features II Repack targets are Service. Classes • Usage of existing assigned recall/migration policies • Files from source tape are repacked to tapes in the assigned Tape. Pool of the specified Service. Class Atlas. Repack Stager repack –V 1: 2: 3 –o Atlas –S Atlas. Stager 1 2 3 Repack Atlas Service. Class atlas. Pool. B Tape. Pool Data transfer Felix Ehm Castor Face 2 Face – November 2006

Design and Features of the new Repack 2 Features II Multi Stager ability • Can be used for load balancing of repack processes C 2 Test Repack. Server C 2 Public C 2 Alice Backup of old location of file is stored in Repack DB • Possibility of recovering of old tapes Runs unattended • “Fire and forget” Felix Ehm Castor Face 2 Face – November 2006

Current Scenario at CERN Around 22. 000 tapes to repack STK 9940 B have to be replaced by IBM and Storage. Tek solution (IBM 3592, STKT 10000) Estimated time to repack tapes 160 days for 5 PB of data 20 x 9940 drives dedicated for process Shared Instance for Repack with Users (C 2 Public) Felix Ehm Castor Face 2 Face – November 2006

Test and Performance Successful test with different kinds of scenarios Different tape types Configuring target Service. Classes • One tape / many tapes in target Tape. Pool Deployment next week Performance depends on Diskservers Bottleneck for data throughput Diskserver deals with too many copy processes : timeout • Connection is dropped from the tapeserver • Another mount is needed Felix Ehm Castor Face 2 Face – November 2006

Questions ? Felix. Ehm@cern. ch Felix Ehm Castor Face 2 Face – November 2006