z EDC User Experience XV EPV User Group

z. EDC User Experience XV EPV User Group - Rome 25 october 2017 Roberto Gioi Responsabile Capacity e Ottimizzazione

Contents • • • Compression before z. EDC The ‘New’ z. EDC compression z. EDC: what it is and how it works Operational Environment zlib z. BNA tool z. EDC in MPS Results from Test enviroment Results from Production enviroment Monitoring Operational problems 2

‘Old style’ Compression - CSRCESRV Software Compression • Run Length Encoding: replaces runs of character with a smaller number of bytes indicating character type and number. The algorithm exploits the presence of repeated characters in a sequence of data. • Compressed data consists of a combination of symbols representing repeating character strings and original characters that are not repeated. • Blocks of data containing many repeating characters can effectively benefit from this algorithm: datasets with fixed length fields containing many blank fields, assembler programs sources. • For other data formats this algorithm may not be effective if it does not even cause the data volume to increase. 3

‘Old style’ Compression - CSRCMPSC Hardware Assisted Compression (Compression co-processor Dictionary-based (DBB)) • Replaces common character strings with a shorter identifier of the string using the CMPSC hardware instruction. • Algorithm relies on a 'Symbol table', otherwise known as a 'dictionary'. • The data encoded by the algorithm contains symbols that match dictionary entries. The CSRCMPSC macro uses this algorithm to perform compression. • Compared to the CSRCESRV macro, this service uses a larger amount of storage but is more effective if you know in advance the type of data to be compressed. • Particularly useful with small data blocks such as VSAM or DB 2 rows. SW and HW compression designed to have a low cost of decompression: ‘compress the data once, but decompress many times’ 4

z. EDC: what it is and how it works Main features • z. EDC is a feature created to perform HW data compression/decompression. . • Designed to execute high performance compression and low latency: it reduces CPU consumption, optimizes compression, improves disk storage and optimizes data exchange between platforms (uses an algorithm available on most platforms, NOT only z/system ). • z. EDC compression replaces characters with pointers to identical strings earlier in the block: the larger the block of data being compressed, the more effective the compression. • It consists of the z/OS z. EDC software feature (z/OS V 2. 1 and later) and z. EDC Express adapter, available starting from of z. EC 12 (GA 2) and later. • Connected via Peripheral Component Interconnect Express (PCIe), standard for expansion cards. • Each z. EDC Express shareable across 15 LPARs • Up to 8 devices per CEC (2 per drawer) 5

z. EDC: what it is and how it works • The first card must be activated by IPL (PARMLIB IFAPRDxx), subsequent ones can be added dynamically (check last PTF level for z. EDC). • z. EDC Express bandwidth (throughput) is up to 1 GB / sec. • CMPSC is up to 300 MB/sec and zlib is between 50 and 100 MB/sec. Benefits: • Faster data backups using less backup media • Data transfer latency reduction • CPU time reduction and improved throughput (compared to other compression methods) • Disk space savings • Useful with many types of large files and exploiters • z. EDC APIs also available for Java code running on z. IIPs • Applicable for "active" data instead of only inactive (archival) data • Compressed data can be exchanged across platforms (Linux, z/OS, Mac OS X, i. OS. . . ) via zlib. 6

z. EDC: what it is and how it works • The z. EDC card works on a simple round robin polling mechanism, continuously checking each LPAR it is connected to see if they have any work for it. • As the card utilization increases, the interval between when it polls each LPAR gets larger, potentially resulting in requests queueing in the z/OS images. • Whether this happens or not can be checked in the RMF PCIE report (“Request Queue Time”). • Unlike z. IIPs, there is no ‘crossover’ to general purpose CPs: if the card is overloaded, users of its services will simply observe longer queue times. • In such a case you can allow longer queue times OR reduce the number of z. EDC exploiters OR add more z. EDC cards. 7

z. EDC: what it is and how it works z. EDC is available through the FPZ 4* family of z/OS authorized system services, and can be exploited at different levels: • Data Owner request: The data owner might compress the data. As an example, SMF can use z. EDC to compress records sent to SMF logstreams, with the System Logger being completely unaware. • Access method request: Selected DFSMS access methods can use z. EDC. The user program doesn’t need any changes. Data is compressed/decompressed by the access method as it is written/read to/from the disk. • Mixed: DFSMSdss is able to use z. EDC compressed DFSMSdss files can in turn be managed by BSAM/ QSAM as z. EDC compressed. In this case data is z. EDC compressed twice. • z. EDC is available for both Linux on z and z/OS. z/VM only provides guest support. • z. EDC compressed data can be exchanged across all platform (CMPSC only works in z/OS ). 8

Setup and Operation • z. EDC Compression for new data sets can be requested in the same way the existing types of compression (generic and tailored compression) are requested. • It can be selected at either or both the data set level or system level. - Data class level COMPACTION option values: T (Tailored), G (Generic), ZR ( "z. EDC Required"), ZP ("z. EDC Preferred") - System level (i. e. when COMPACTION(Y) specified in data class) – the following values will be available on the COMPRESS parameter in IGDSMSxx member of SYS 1. PARMLIB (ignored for VSAM KSDSs): • COMPRESS (GENERIC) refers to DBB-based compression. • COMPRESS (TAILORED) the system attempts to derive and use a compression dictionary • COMPRESS (ZEDC_R) refers to "z. EDC Required". • COMPRESS (ZEDC_P) refers to "z. EDC Preferred". Data class takes precedence over system level. The default continues to be GENERIC. . 9

Setup and Operation • COMPRESS (GENERIC) - DBB-based. With DBB-based compression (the original form of compression used with both sequential and VSAM KSDS compressed format data sets), the system selects a set of dictionary building blocks (DBBs), found in SYS 1. DBBLIB, which best reflects the initial data written to the data set. The system can later reconstruct the dictionary by using the information in the dictionary token stored in the catalog. • COMPRESS (TAILORED) With tailored compression, the system attempts to derive a compression dictionary tailored specifically to the initial data written to the data set. Once derived, the compression dictionary is stored in system blocks which are imbedded within the data set itself. An OPEN for input reconstructs the dictionary by reading in the system blocks containing the dictionary. This form of compression IS NOT SUPPORTED for VSAM KSDSs. 10

Setup and Operation • COMPRESS (ZEDC_R) refers to "z. EDC Required" compression. When creating a z. EDC-compressed format data set, ZEDC_R indicates that the system should fail the allocation request if the z. EDC function is not supported by the system (regardless if the z. EDC feature is installed), or if the minimum allocation amount requirement is not met (5 MB or 8 MB Primary if no Secondary), allocation fails with IGD 17168 I message. • COMPRESS (ZEDC_P) refers to "z. EDC Preferred" compression. When creating a z. EDC-compressed format data set, ZEDC_P indicates that the system should not fail the allocation request, but rather create either a tailored compressed data set if the z. EDC function is not supported by the system (regardless if the z. EDC feature is installed), or a non-compressed extended format data set if the minimum allocation amount requirement is not met (5 MB or 8 MB Primary if no Secondary). 11

Cross platform - zlib • The zlib data compression library provides in-memory compression and decompression functions, including integrity checks of the uncompressed data. • The z/OS zlib compatible C library provides a set of wrapper functions that use z. EDC compression when appropriate and when z. EDC is not appropriate, software-based compression services are used. (USS - z/OS v 2. 1) • A modified version of the zlib compression library is used by z. EDC. • If the system requirements for z. EDC have been met: - For a deflate stream, the parameters specified on deflate. Init 2() are supported by z. EDC. - For an inflate stream, all the parameters specified on inflate. Init 2() are supported. Because there are overhead costs when communicating with the hardware, ONLY on the first call to deflate or inflate a data stream, the provided input is checked to ensure that it is sufficiently large enough to make it worthwhile to use z. EDC (IQPPRMxx, DEFMINREQSIZE, INFMINREQSIZE 4 K). If the data stream is large enough, z. EDC is used. If the data stream is small, it might cost more to compress the data stream with z. EDC so software-based compression services are used. Once z. EDC is used as the compression mechanism, you cannot change the compression method to software-based compression services. At the same time, if software-based compression services are used as the compression mechanism (for example, after the first call to inflate or deflate the data stream is completed), you cannot change the compression method to z. EDC. 12

z. BNA tool • IBM z. BNA is a no-charge, as-is tool that can be used to analyze a single batch window of userdefined length. It reads System Management Facilities (SMF)and from version 1. 3 it provides a means of estimating the number of jobs and BSAM/QSAM data sets that might be eligible for compression using the z. EDC Express, and helps determine the number of features • z. BNA reports available (z. EDC - related): • Top z. EDC Compression Candidate BSAM/QSAM DASD Data Sets Report that includes: - COMP - Already compressed using System z hardware compression – z. EDC Eligible. - EF - Extended Format data set but not compressed - z. EDC Eligible. - PS - Physical sequential not extended format to be converted to Extended Format - z. EDC - Using the z. EDC Express feature for compression • Estimated number of z. EDC Cards Required by Hour for BSAM/QSAM compression • Estimated CPU savings when using the z. EDC feature for BSAM/QSAM. NOTE: The CPU savings estimate offered by z. BNA analysis only considers DFSMS data sets that are using Hardware Data Compression (Generic or Tailored). 13

z. EDC Product Usage Overview Roadmap (from IBM) ü MPS t. b. d. 14

z. EDC in MPS • Installation task set up • Implementation waves in Test domain 23/01/2017 01/02/2017 Initial setup and tests: - z. BNA simulation. - HW-SW installation (HCD definitions, IFAPRDxx, z. EDC feature in PARMLIB IPL). - Check and assessment: CPU, Storage, I/O, elapsed time savings. Implementation calendar in TEST: - HSM - Image Copy DB 2 - Archive Logs DB 2 - Sequential DS - new Dataclass definition DCCMP 59 - Content Manager Ondemand 15

z. EDC in MPS • Production deployment - LPAR SYRE - Sistemi ‘storici’ 08/03/2017 (HSM) - LPAR OS 0 A - Aziende di Gruppo 21/03/2017 (HSM, Ondemand) - LPAR DBM 2 - SAP 21/03/2017 (HSM, Backup DB 2, z. EDC dumps) - LPAR OS 0 U - Sistema unico 22/03/2017 (HSM, Ondemand - part) - LPAR PROD - Batch MPS 22/03/2017 (HSM, Ondemand, Batch DFSMSdss) - LPAR CICS - TP MPS 27/03/2017 (Standby – waiting for 2 add. z. EDC feat. ) • Next steps - HSM LPAR CICS - DB 2: Image Copy, Archive Log, Unload - Logstream SMF - Sequential DS (from z. BNA analysis) - MQ: Archive Log, Channel compression (via zlib) 16

Early observations in TEST - HSM Main results from TEST environment ü First step: z. EDC and HSM • Different time windows : 18: 00 -> Backup 00: 00 -> Migration • Weekend: Card utilization doubled as consequence of the weekly migration activities 17

Early observations in TEST - HSM 18

TEST environment - Image Copy 19

TEST environment - DUMP DFSMSdss 20

TEST - HSM CPU 21

HSM – PROD LPAR CPU 22

HSM - DBM 2 LPAR CPU 23

Batch window elapsed time 24

Ondemand OS 0 A LPAR 25

SPIN-OFF activity: elapsed time ed p p ro ed m i n t ur o i t ho ra Ope 9 to 1 from DCCMP 59 parameter in all batch jobs used for copying archives before and after spinoff activity Same schema introduced in all batch procedures related to Dev. Ops operations 26

Unload processing 27

AXA batch window elapsed time 28

Monitoring z. EDC cards 29

Monitoring z. EDC cards - detail 30

Monitoring z. EDC cards - detail 31

Operational problems • Recycle Heavy weekend activity (from Saturday afternoon to Sunday morning). Performs Tape Analysis in order to verify tape status against set threshold: Ex. consolidates all tapes with free space greater than 40%. With z. EDC, elapsed times literally rocketed: activity on Monday morning is still running and requires manual stop / start by operator Under IBM analysis: addressed to HSM (fails in calculating %) Still no fix available Workaround: decreased percent criteria less tapes eligible for recycle • Long elapsed time for AXA dump operations Before night batch an extensive DFSMSdss dump of all data is perfomed. The subsequent restore shows poor and unsatisfactory response times. Should be fixed by PTF HIPER OA 52037 • Data mover malfunction for some dump types When performing a DASD to DASD move via DFSMSdss for certain z. EDC compressed dump the process fails. Under investigation by IBM (low priority) 32

HIPER OA 52037 - Detail 33

Operational problems 34

THANK YOU roberto. gioi@mps. it