UK CMG 2008 H W Barry Merrill Ph

UK CMG 2008 H. W. "Barry" Merrill, Ph. D Merrill Consultants Dallas, TEXAS www. mxg. com Tuesday May 20, 2008 Remote from Emerald Island, NC

A. MXG Support for SAS Version 9. 2 1. Compatible, No ERRORs, New Warnings 2. B. MXG and WPS 1. Current Status of MXG and WPS 2. Run Time Comparison 3. Revisions to SAS Clone Newsletter Article 4. Summary 5. QA Steps successfully compiled and run C. CPU Parked Time Metric D. z. IIP and z. AAP Metrics E. Daily CPU cost of Performance Measurement

A. Support for SAS Version 9. 2: COMPATIBLE, no ERRORS, new WARNings All MXG Versions execute WITHOUT error with SAS Version V 9. 2 libraries can be read/written by SAS V 8. 2 or V 9. 1. 3, & vice versa. SAS V 9. 2 Phase I Foundation Level z/OS + ASCII was tested. These old MXG Versions WILL print a new SAS V 9. 2 WARNING, that sets CC=4 (condition/return code), but that warning is harmless (to MXG code), and all MXG output SAS datasets are correct, even with that warning and return code. MXG Version 26. 02 eliminates this SAS V 9. 2 WARNING internally, by enabling new OPTION VARLENCHK=NOWARN to suppress the creation of both the warning and the condition code. So the ONLY exposure with older MXG Versions under V 9. 2 Is only on z/OS, and ONLY if condition code tests are used in your MXG job streams.

This new-in-SAS V 9. 2 "MULTIPLE LENGTHS OF A VARIABLE“ Warning message surfaced in MXG delivered code in two cases: -The shortening of the LENGTH of a numeric variable, but only if when the LENGTH statement precedes SET/MERGE/UPDATE. This occurs in VMXGSUM where the fixed-length-8 variables output by PROC MEANS were reduced to 4 -bytes, prior to option KEEPLEN. The VMXGSUM utility is used in all MXG summarization, like ASUMxxxx and TRNDxxxx, many ANALxxxx members, and in summarizing RMFINTRV and CICINTRV programs in BUILDPDB. It is pervasive in MXG. This case was eliminated in MXG 26. 02 by relocating its LENGTH statement to after the SET. - The JOIN of multiple datasets (SET MON. JOBS TUE. JOBS. . . ) where a variable has different lengths in different datasets, and the first dataset’s variable has shorter length. This also occurs in VMXGSUM, when multiple input datasets are combined, like TRENDing, where TREND had short LENGTHs, but the "NEWTREND" internally has fixed, longer LENGTHS. MXG 26. 02 adds KEEPLEN option to PROC MEANS to eliminate.

MXG Version 26. 02 code eliminates the SAS V 9. 2 WARNING internally, but also enables OPTION VARLENCHK=NOWARN to suppress BOTH the warning and the condition code, for all your SAS programs, as long as you use the MXGSASV 9/CONFIGV 9 MXG initialization. Without VARLENCHK=NOWARN, EVEN with MXG 26. 02, WARNING can OCCUR: - If you have tailoring members in "USERID. SOURCLIB" from old MXG versions, that need the same code revisions. - In any user-written SAS programs, but this could actually be a valid warning that a variable was unintentionally truncated. or, at any time in the future, the WARNING can still occur: - When an MXG Version that changed a variable LENGTH is installed, subsequent WEEKLY or MONTHLY jobs create the WARNING because some PDB's have the old length and some have the new length, when those multiple datasets are joined. Previous to V 9. 2, length could be changed with no WARNING. Between MXG 24. 24 and 25. 25 1206 variables were changed. Hot Fix VARLENCHK for Phase I will be included in Phase II. Changes 26. 060 and 26. 065 have all the V 9. 2 details.

B. MXG and WPS 1. Current status of MXG testing under WPS. a. What has been tested successfully: b. MXG QA compile-only (dummy INFILEs) TYPExxxx and TYPSxxxx members This exercised the MXG DATA-step Code for compile errors, and created DOCVER to compare the contents of WPS-built datasets (variables, LENGTHs, LABELs, FORMATs). Steps 1 thru 36 of the QAJOBXX were tested. Default BUILDPDB with 480 MB SMF File. Tailored BUILDPDB with IMACEXCL with 480 MB SMF File. TYPETPMX with small file - verified 72 position FORMATs worked. TYPENTSM with test file - verified open-system-style MXG code. b. WPS is often updated daily, and as errors were encountered in MXG tests on z/OS and Windows/XP, a new release was generated which did correct each error that could be fixed short-term. The current release is World Programming System 2. 3 of May 4.

c. MXG Version 25. 11 is required for testing under WPS; its updates eliminate the need for user modifications to MXG Software, and new WPS-specific members (CONFIGW 2, MXGWPSV 2, JCLINSTW, AUTOEXEW) were created. d. Facilities used by MXG Software not yet in WPS for z/OS: INFILE CCHHR option - needed only for TYPEEREP (EREP, SYS 1. LOGREQ) VIEWS - for I/O performance, used in VMXGSUM, which is invoked over 60 time in QABUILDPDB, and also in all ASUMxxxx and TRNDxxxx members. Eliminates a full pass of the input data. PROC CONTENTS - minor, does not report High Block Used size. e. Facilities used by MXG Software not yet in WPS for z/OS and Windows: INFILE with ftp access method - performance slower, more disk space SAS/GRAPH, SAS/STAT, SAS/ETS, etc. Currently WPS supports the Base DATA Step Support and a few PROCs, including PRINT, MEANS, CONTENTS GPLOT, GCHART, with more planned. f. Output differences - minor PROC MEANS printed output - some values printed with one digit more resolution by WPS. - no error in output values spot checked - prevents automated output comparisons

g. Untested - most due to complexity or time to set up multiple inputs: DATA Libraries on TAPE WEEKBLDT, MNTHBLD special TAPE format to DISM, Mod, etc. TESTTRND, TESTANAL - requires extensive test data ANALxxxx - requires test data ASUMxxxx - except ASUMJOBS, ASUM 70 PR were tested. UTILxxxx - specialized utilities for MXG Tech Support Internal SORT - on z/OS, internal SAS SORT may be required when BY list variables don't fit in first 4096 bytes. Have not confirmed in WPS. NFS Files - not tested. Broken VBS files - not tested.

h. Migration issues - see WPS Migration Instructions: On z/OS, copy all archived SAS Data Libraries on DASD (including HSM migrated) to tape (Sequential) format with the SAS System first. WPS can read SAS Data Libraries in SEQUENTIAL format on tape or DASD. WPS cannot read DASD format SAS Data Libraries i. Customer reports: Several MXG customers have been running their BUILDPDB on z/OS. Early tests had to make source-level-changes to a few MXG members. Most had relatively simple BUILDPDBs with minimal tailoring.

j. MXG Support Position for execution under WPS product: MXG 25. 11+ and WPS 2. 2 or later are required. Your current MXG Software License Agreement states: Merrill provide continuous product support for MXG Software: When error conditions are the results of errors in MXG code, they will be corrected; error is defined: SAS® execution of MXG code produces either a return code or an ABEND. If you encounter an execution error testing MXG under WPS: You should report the error to WPS Technical Support for initial investigation. If WPS support believes the error is an MXG problem, they can contact MXG, or they can refer you to contact support@mxg. com. If the error can be replicated under the SAS System, it will be corrected per our license terms. For data errors, (INPUT STATEMENT EXCEEDED RECORD etc) Contact MXG Support by sending the FULL job log, and maybe - to ftp the raw data file that caused the error - or to ftp your USERID. SOURCLIB (MXG tailoring) libraries

2. A. Run time comparisons of SAS and WPS - NOVEMBER 2007: BUILDPDB, ASUMJOBS, ASUM 70 PR with 448 MB SMF File z/OS Comparison CPU TCB minutes Elapsed minutes ------ Compressed --------Work Size PDB Size CICSTRAN Size SAS: 3. 88 5. 20 209 MB 55 MB 59 MB WPS: 10. 67 18. 50 233 MB 59 MB 67 MB Windows/XP Comparison SAS: 2. 18 Note 1 65 MB 63 MB WPS: 2. 71 Note 1 78 MB 70 MB Note 1: Neither WPS nor SAS provide a way to track maximum work space on ASCII, except for free space monitoring with DIR

2. B. Comparison of SAS V 9. 2 and WPS 2. 3 on z/OS. May 2008: BUILDPDB and ASUMs with 448 Mega. Byte input SMF file; z/OS Step Totals: SAS V 9. 2 WPS 2. 3 RATIO Total CPU time mm: ss 03: 45 sec 225 mm: ss 08: 30 sec 510 2. 26 Total Elapsed time 08: 00 480 16: 58 1018 2. 10 Total EXT Memory Total SYS Memory Total Memory 104 11 115 Mega. Bytes 188 Mega. Bytes 504 Mega. Bytes 692 Mega. Bytes SMF Data Step Elapsed time SMF Read Rate mm: ss 01: 12 sec 72 6. 22 MB per sec mm: ss 03: 27 sec 207 2. 16 MB per sec 2. 85

3. Revisions to SAS Clones article in MXG Technical Newsletter FIFTY: WPS is no longer vapour-ware. The company had bent over backwards to provide corrections. When originally written in JAVA several years ago, WPS performed so poorly, with run times at least ten times worse than the current release, that JAVA is no longer used in the product. So the offload of your SAS CPU time to a z. AAP won’t happen. Items listed in sub-paragraphs i, iii. , and iv. have all been addressed to my satisfaction, with the exception of the items that are listed above in this note. Pricing has been significantly reduced from those original IBM prices. As an example, in late 2007, an MXG site in the USA was quoted an IBM price for a 21 Value Unit system (about 1000 MIPS) of $42, 000 first year and $8, 400 for renewals. WPS can be licensed through IBM or through World Programming; their home page is at http: //www. teamwpc. co. uk

4. Summary: Most of MXG has compiled successfully under WPS on both z/OS and Windows/XP. BUILDPDB has compiled and processed SMF data on both z/OS and Windows/XP. Not all of MXG has been tested with data. WPS is still in development; January’s 2. 2 had five errors. This week’s new WPS 2. 3 claims to have corrected them. WPS on z/OS requires thrice the CPU and Elapsed Run Time of SAS. It is claimed that WPS 2. 3 has improved z/OS performance. WPS on Windows/XP run times are similar to SAS run times. Disk Space required on both platforms are similar. So, it is safe now to test MXG under WPS.

5. QA Steps successfully compiled and executed with dummy input. STEP STEP STEP STEP 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. CREATE FORMAT LIBRARY RUN TESSNT RUN TESSIBM 1 RUN TESSIBM 2 RUN TESSIBM 3 RUN TESSUSER RUN TESSUSR 1 RUN TESSOTHR RUN TYPSCMHM RUN BUILDPD 3+ASUMS RUN BUILDPDB+ASUMS RUN TYPERMFV RUN TYPECMFV+TYPEMVCI RUN TESSHSM RUN TESSFACO

STEP STEP STEP STEP STEP 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. RUN TESSVM RUN TESSCRAY RUN TESSHPCS RUN TESSUNIA RUN TESSUNIK RUN TESSQAPM RUN TESSQACS RUN TESSTUX RUN TESSPW RUN ASUMPRTR RUN TYPEIMS 7 RUN TESSIMSD RUN COPYSTEP RUN TESSTRND - partially completed, not WPS fault. RUN TESSMNTH RERUN TYPE-S FOR NON PROC SORT FOR DATASET LABEL RUN CROSSREF RUN DOCVER - output DOCVER file is identical.

This data for LCPUADDR=5 shows a CP engine that was parked for 103 seconds of that 5 minute interval. RMF subtracts the SMF 70 PAT parked duration from the SMF 70 ONT online duration to calculate the Percent MVS Busy value. In this interval, ORIGWAIT was zero for this engine, as MVS never entered the wait state on that engine, so RMF calculates the MVS busy percent as: PCTMVSBY= 100*(SMF 70 ONT-ORIGWAIT-SMF 70 PAT)/(SMF 70 ONT-SMF 70 PAT); PCTMVSBY= 100*( 299 -0 -103) /(299 -103) = 100% The IBM calculation of the PCTCPUBY, the LPAR CPU busy percent, is NOT altered by parked time; PCTCPUBY=32%, calculated as PCTCPUBY= 100*(LCPUPDTM/SMF 70 ONT); = 100 * (96 / 299 ); The "PATWAITM", the time when the CP engine is "not parked", is the time when this CP engine could/should have been parked, but was still online and not-dispatched, because the algorithm to park a CPU only executes occasionally. It is not created in TYPE 70 PR.

D. z. IIP and z. AAP measurements when they are faster than CPU engines. When specialty engines are faster than the speed of your CPs, there is a normalization factor to convert the recorded seconds to their NORMALIZED (EQUIVALENT) time, as if they had executed on your CPs. In all MXG workload datasets, TYPE 72 GO and RMFINTRV, (and TYPE 30), all time variables for z. IIPs and z. AAPS are NORMALIZED seconds, and all of the service units are segregated by engine type. However, the IBM RMF reports present these data quite differently. This system has the normalization factor, R 723 NFFS=569/256=2. 222, that is, one second of z. IIP is equal to 2. 222 seconds of CP time. ================================ MXG Dataset TYPE 72 GO dataset values: ================================ SERVICE 3, 932, 091 CPUUNITS 1, 793, 920 ZIPUNITS 2, 137, 167 CPUTCBTM 178. 92 CPUZIPTM 213. 16

RMF WORKLOAD REPORT: Under "SERVICE TIMES", the RMF "CPU" value of 392. 9 seconds is the total of the real CPU time on CP engines, plus the NORMALIZED CPU time on the z. IIP and z. AAP engines; it is NOT the CPU "TCB" time. ( 392. 9 = 178. 92 + 213. 16 "RMF CPU" = CPUTCBTM + CPUZIPTM ) But also under "SERVICE TIMES", the RMF "IIP" (z. IIP) value of 96. 1 seconds is the UN-NORMALIZED, raw, seconds on the z. IIP engine. And the RMF "AAP" value for z. AAPs is also the UN-NORMALIZED value. And under "SERVICE", the RMF "CPU" value of 3931 K is the TOTAL SERVICE units from CPs, z. IIPs, and z. AAPs. REPORT BY: POLICY=OWL WORKLOAD=CSSDDF TRANSACTIONS ---SERVICE TIMES AVG 0. 23 IOC 0 CPU 392. 9 MPL 0. 23 CPU 3931 K SRB 0. 0 ENDED 51 MSO 0 RCT 0. 0 END/S 0. 01 SRB 0 IIT 0. 0 #SWAPS 0 TOT 3931 K HST 0. 0 EXCTD 0 /SEC 1092 AAP N/A AVG ENC 0. 23 IIP 96. 1 ---APPL CP AAPCP IIPCP %--4. 98 0. 00 0. 07 AAP IIP N/A 2. 67

While the workload datasets have normalized CPU time, in all of the "hardware" datasets, TYPE 70 PR, ASUM 70 PR etc. , the CPU times for the z. IIP and z. AAP engines are the raw seconds of CPU Dispatch Time on those engines, and is NOT normalized. As a result, then, the total ZIPACTTM recorded in TYPE 70 for the above system for the day was 10, 887 seconds, but the total CPUZIPTM in TYPE 72 GO for the day was 23, 079 seconds. Those 10, 887 raw hardware seconds would be 24, 190 normalized seconds so the z. IIP capture ratio at this site is 23079/24190 = 95. 4%.

E. The CPU cost of performance monitoring and capacity planning. One MXG user reports they currently write 500 GB of SMF data per day or an average rate of 6 Mega. Bytes per second across all platforms. They dump SMF multiple times each day, and build multiple "PDB's" throughout the day, and run many ad hoc analysis reports as well. They have SMF, RMF, OMEGAMON, and NETVIEW monitors consuming CPU. The daily total CPUTM for each of their workloads were: OMEGAMON MXG JOBS RMF III RMF I SMF DUMPS MONITORS SMF ASID 28: 56: 37 19: 05: 01 12: 20: 05 6: 29: 11 4: 12: 30 2: 17: 10 0: 29: 16 TOTAL CPUTM 73: 30: 50 = 2% of 3744 HOURS with 156 CPs

Thus this sites total daily cost of 74 CPU hours is an average use of 3 CP engines all day long, but with 156 CP Engines, that is ONLY 2% of the installed CP engine capacity, for the entire CPU cost of performance monitoring, data collection, building PDBs, archiving, and all MXG daily reporting and ad hoc analysis. to read 518 GB of SMF data 838 K TMC records 902 K DASD DSN records (DCOLLECT) 27 K DASD VOL records (DCOLLECT) The following graph shows the hourly CPU consumption of these workloads.