Save MSUs and Reduce RunTimes for Analytics Paul

  • Slides: 26
Download presentation
Save MSU’s and Reduce Run-Times for Analytics Paul Massengill Systems Engineer Copyright © 2018

Save MSU’s and Reduce Run-Times for Analytics Paul Massengill Systems Engineer Copyright © 2018 Luminex Software, Inc. All Rights Reserved. CMG Canada

Agenda Business Goals Why Explore this Solution? Discovery Process Review of Business Findings Solution

Agenda Business Goals Why Explore this Solution? Discovery Process Review of Business Findings Solution Results Next Steps to More Savings! 2 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Business Goals Reduce 4 Hr. Rolling Average Reduce mainframe licensing costs Reduce ongoing code

Business Goals Reduce 4 Hr. Rolling Average Reduce mainframe licensing costs Reduce ongoing code maintenance Modernize Analytics: Introduce new programming tools such as R and Python Inventory and optimize MXG PDB builds and reporting Quicker Analytics: Get Answers Faster Add in Additional DB 2 records for Analysis 3 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Why Consider Off-Host SAS Processing? Remove data center processing that is not revenue generating,

Why Consider Off-Host SAS Processing? Remove data center processing that is not revenue generating, such as MXG, off the mainframe SAS language processing is often ranked in the top “heavy hitters” list of programs Prepare for Tailored Fit Pricing Reduce the baseline Manage growth over time Opportunity to: Reduce Analytics Run Time Modernize SAS processes, integrate with R and Python Streamline existing SAS processing for efficiency Fulfill latent demand for additional/more timely SAS analytics & workloads 4 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

The Solution Process 5 • DISCOVERY Identification of SAS estate and associated assets •

The Solution Process 5 • DISCOVERY Identification of SAS estate and associated assets • ANALYSIS Analysis, interpretation, presentation of results with recommendations for change • OPTIMIZATI ON Improve operations efficiencies and areas to modernize applications • MIGRATION Migration of SAS language-based workloads to MDI Platform • DEVELOPME NT Uncover alternate solutions using existing technologies Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

SAS Program Questions Approximately how many SAS programs do you have? Approximately how many

SAS Program Questions Approximately how many SAS programs do you have? Approximately how many SAS users? Do any of your SAS programs update live DB 2 or VSAM databases? What types of output are you producing? CSV, PDF, etc. Are You Afraid of Change? 6 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

SMF Analysis Summary Reporting Period: May–June Future MSU Reduction Targets 6 LPARs SAS #2

SMF Analysis Summary Reporting Period: May–June Future MSU Reduction Targets 6 LPARs SAS #2 on the Heavy Hitter’s list 28, 000+ SAS executions 399 unique job names FTP Activity 631, 000 file transfers via FTP over Port 21 233, 000 Client FTPs 398, 000 Server FTPs Some GDGs Majority Application data . COPYLIB. DATALIB. PDS . BATCH . PROCLIB. JCL. SOURCE. ISPSLIB. DCLGEN. CARDS SAS is a registered trademark of SAS Institute 7 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Overall Programs Rankings SAS is #2 8 Copyright © 2018 Luminex Software, Inc. All

Overall Programs Rankings SAS is #2 8 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Top 9 Intervals Program Rankings Customer Stated SAS and MXG used 18% of the

Top 9 Intervals Program Rankings Customer Stated SAS and MXG used 18% of the processor Every Day from 8 am to 6 pm Processing previous day’s SMF records. 9 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Potential MSU Reductions Overlay 10 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Potential MSU Reductions Overlay 10 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Potential MSU Savings 11 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Potential MSU Savings 11 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Top 20 SAS Programs by CPU Time 12 Copyright © 2018 Luminex Software, Inc.

Top 20 SAS Programs by CPU Time 12 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Problem Identified Now What Can We Do About It ? ? 13 Copyright ©

Problem Identified Now What Can We Do About It ? ? 13 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

MDI: SLP Solution Goals INVOICE Move all “heavy-hitter” SAS/MXG applications off the mainframe $

MDI: SLP Solution Goals INVOICE Move all “heavy-hitter” SAS/MXG applications off the mainframe $ Reduce MLC and SAS Institute software licensing costs SAS license can be reduced on the mainframe SAS can be licensed on the mainframe for remaining workloads Free up expensive DASD storage by moving SMF and MXG PDB data to commodity storage Efficient FICON data transfer without mainframe TCP/IP overhead Maintain mainframe-centric job control and security 14 Copyright © 2018 Luminex Software, Inc. All Rights Reserved. TCP/IP

MDI: SLP Component Layers FICON 1 or 10 Gb. E LUMXPROC Reports SSH Commands

MDI: SLP Component Layers FICON 1 or 10 Gb. E LUMXPROC Reports SSH Commands • SAS Server • Storage for SMF Data • Storage for PDB Data SAS Compiler MXG Code Client Layer Compute Layer Source of data Job control and scheduling Send SAS programs and data down to MDI Platform via LUMXPROC and JCL MDI Platform 15 Storage Layer SAS Server Data movement & communications Linux/ASCII licensing MDI Code Dispatcher SAS compiler MDI: SLP product MXG code Copyright © 2018 Luminex Software, Inc. All Rights Reserved. SMF data PDB data

Luminex Migration Services worklo ads assets Identified all MXG related assets, workloads, users, data

Luminex Migration Services worklo ads assets Identified all MXG related assets, workloads, users, data libraries, code libraries and SAS products Identified wasted resources that can be recovered through re-engineering or modernization S A S usersdata code Prioritized jobs for migration and identified jobs that can migrate immediately Assisted in designing migration plans with minimal impact Provided constant monitoring during migration to ensure all workloads have been migrated 16 Copyright © 2018 Luminex Software, Inc. All Rights Reserved. s lib rie ra

What Were The Results? $$$$ $$ $ $ $$ 17 worklo ads s t

What Were The Results? $$$$ $$ $ $ $$ 17 worklo ads s t e s as e u l a V usersdata code Processing reduced to 26 -28 minutes from 9 hours Moved MXG processing completely to MDI: SLP (18% Processor Usage Reduction) Added in DB 2 detailed records Moved SMF storage, MXG PDBs and workspace to commodity storage that was already on the floor Removed many custom jobs to maintain trend PDBs 4 Hr Rolling Average Slightly Reduced (Latent Demand) Copyright © 2018 Luminex Software, Inc. All Rights Reserved. ies l ar r b i

From Virtual Tape… Controller Source Target • VIRTUAL TAPE Compression Encryption Replication Push Button

From Virtual Tape… Controller Source Target • VIRTUAL TAPE Compression Encryption Replication Push Button Disaster Recovery Cloud Connectivity FICON 18 Internal SAS, Fibre Channel, 1 or 10 Gb. E Copyright © 2018 Luminex Software, Inc. All Rights Reserved. • Create / Process • Protect • Transfer / Share • Analyze / Report

From Virtual Tape… to Virtually Anywhere Controller Source Compression • VIRTUAL TAPE • DATA

From Virtual Tape… to Virtually Anywhere Controller Source Compression • VIRTUAL TAPE • DATA INTEGRATION Target Encryption Replication Push Button Disaster Recovery Cloud Connectivity FICON Internal SAS, Fibre Channel, 1 or 10 Gb. E FICON 1 or 10 Gb. E • Create / Process • Protect • Transfer / Share • Analyze / Report Data Movement Communications Data Integration Control Software Translation / Conversion Listener Hadoop Source / Target 19 Controller Copyright © 2018 Luminex Software, Inc. All Rights Reserved. NFS FTP/SFTP Source / Target Analytics Kafka

MDI is a Data Transfer & Co-Processing Platform Mainframe FICON MVT or Dedicated MDI

MDI is a Data Transfer & Co-Processing Platform Mainframe FICON MVT or Dedicated MDI Platform Data Sharing Targets/Sources MDI Big. Data Transfer • web. HDFS, S 3 • FICON • 1 or 10 Gb. E MDI XPDS • NFS MDI Secure. Transfer Secure High speed Efficient, redundant I/O channels 20 Profile-based architecture for extending processing & interface capabilities High speed, scalable transfer rates SAF integration & protocol-based encryption Bi-directional movement and communication for multi -platform workflows and co-processing Data translation / conversion Copyright © 2018 Luminex Software, Inc. All Rights Reserved. • SFTP MDI SLP • Analytics MDI z. Konnect • Kafka

Q&A 21 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Q&A 21 Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Paul Massengill Systems Engineer & Analytics Specialist Luminex Software pmassengill@luminex. com 22 www. linkedin.

Paul Massengill Systems Engineer & Analytics Specialist Luminex Software pmassengill@luminex. com 22 www. linkedin. com/in/paul-massengill-analytics Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

From Virtual Tape… to Virtually Anywhere Controller Source Compression VIRTUAL TAPE DATA INTEGRATION Target

From Virtual Tape… to Virtually Anywhere Controller Source Compression VIRTUAL TAPE DATA INTEGRATION Target Encryption Replication Push Button Disaster Recovery Cloud Connectivity Tape Migration & Conversion FICON Internal SAS, Fibre Channel, 1 or 10 Gb. E FICON 1 or 10 Gb. E Data Movement Communications Data Integration Control Software Translation / Conversion Listener Source / Target 23 Controller Copyright © 2018 Luminex Software, Inc. All Rights Reserved. Source / Target

MDI is a Data Transfer & Co-Processing Platform Mainframe FICON MDI Platform Data Sharing

MDI is a Data Transfer & Co-Processing Platform Mainframe FICON MDI Platform Data Sharing Targets/Sources MDI Big. Data Transfer web. HDFS FICON 10 Gb. E MDI XPDS NFS MDI Secure. Transfer Secure High speed Efficient, redundant I/O channels Profile-based architecture for extending processing & interface capabilities High speed, scalable transfer rates SAF integration & protocol-based encryption Bi-directional movement and communication for multi -platform workflows and co-processing 24 Including data translation from EBCDIC to ASCII and between character sets Copyright © 2018 Luminex Software, Inc. All Rights Reserved. SFTP MDI SLP SAS, MXG … and more

MDI Secure. Transfer Benefits Faster Data Transfer Rate The FICON connection between the mainframe

MDI Secure. Transfer Benefits Faster Data Transfer Rate The FICON connection between the mainframe and the MDI platform copies large data files off the host faster and more securely than TCP/IP based protocols Customer Example: Move 10 GB of Data Off the Mainframe 45 minutes Mainframe FTPS Luminex MDI: ST 2 minutes 0 5 10 15 20 25 Transfer Time in Minutes Large files are no longer an issue 25 Copyright © 2018 Luminex Software, Inc. All Rights Reserved. 30 35 40 45

Thank You Copyright © 2018 Luminex Software, Inc. All Rights Reserved.

Thank You Copyright © 2018 Luminex Software, Inc. All Rights Reserved.