Mainframe Batch Window Challenges Thinking Outside the Box
Mainframe Batch Window Challenges – Thinking Outside the Box Data. Kinetics
What Does Data. Kinetics Do? Our comprehensive, world-renowned suite of solutions enables Fortune 500 companies to: • • • Process over a billion mission-critical transactions every day Accelerate application processing by up to 98% Seamlessly integrate data on mainframe and distributed systems Enable big data and analytics in multi-platform environments Enable increased control and flexibility in sub-capacity pricing and R 4 H soft capping environments © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Who Does Data. Kinetics Work With? • Some of the world’s largest companies are our clients: • 3 of the top 5 global banks • 3 of the top 5 global credit card companies • 3 of the top 5 US property & casualty insurance companies • 2 of the top 5 US health insurance companies • 20% of the Fortune 50 © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Solutions that solve mainframe batch window challenges Solving Mainframe batch window challenges in new ways • • • Most have challenges with their batch window. A race against time that you sometimes lose. Most “inside-the-box” solutions are either extremely costly, risky, or both. What if there were less conventional - and less risky solutions? Some “outside-the-box' solutions are low-risk, budget-friendly, and will definitely make a difference for you. • Let’s attack this problem from 3 angles, from the application level, the system level, and from the database level. • All low risk, reasonable cost and very effective. © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Solutions that solve mainframe batch window challenges Agenda • • • About mainframe batch processing Batch processing challenges Solutions (inside the box) Solutions (outside the box) Application level: In-memory Technology Database level: Multi-platform Replication System level: IT business intelligence Summary Q & A © 2017 Data. Kinetics Ltd. All rights reserved. Confident
About mainframe batch processing Batch processing is used for some of the most critical business operations: • Credit card settlement • Generating payroll • Daily consolidation of business transactions such as billing, interest calculations, generating reports of data processed that day • Processing files with large amounts of data from business partners • Printing or sending account statements to all customers of a bank • Database maintenance: • Bulk database updates • ETL • And much more © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Mainframe batch processing attributes • • • Process large volumes, or batches, of instructions or files sequentially A large volume of work requires repetitive actions or repetitive logic Often run within a given time frame, or batch window Jobs often queued, with the outputs of one job providing the inputs for the next job Typically includes reading, writing and sorting sequential files or tables Runs automatically once a batch job begins, it continues until it is done or until an error occurs Manual intervention or inputs not required Often have repeated reads of static data May create temporary files as part of the transaction processing © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Mainframe batch processing challenges • • • Demands for 24/7 OLTP constantly increasing Web and now Mobile transaction loads constantly increasing Batch windows shrinking More batch processing to complete Often cannot complete batch processing within the batch window – can cause SLA default penalties • System resource usage (CPU, MSU) associated with batch constantly increasing • Monthly costs rising indefinitely • System upgrades required every few years to keep up © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside-the-Box solutions for batch • Techniques that can help optimize enterprise batch processing at the system, application and database levels: • • • Mainframe high-performance in-memory technology Multi-platform replication IT business intelligence Automated Soft Capping Automated SQL bottleneck detection • By themselves, each one can make an impact on your batch performance and cost • Combined, they can make a serious impact on your batch performance and cost - one that you need to be aware of. . . © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside-the-Box solutions for batch Hardware Software Ongoing monitoring Mainframe high-performance in-memory solution No Yes No None – Medium * Low - Medium / High Multi-platform enterprise replication No Yes No No Medium / High IT business intelligence No Yes** No Medium Low / High Automated soft capping No Yes** No Low Medium / High Automated SQL bottleneck detection No Yes** No Low / High Solution to batch window challenges Code changes Complexity Cost / ROI © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Mainframe In-memory technology • Why high-performance in-memory technology? • • • Accelerates access to your reference data Accelerates your batch applications (even your legacy OTS packaged batch apps) Improves transaction throughput capacity Reduces mainframe resource usage at the batch application level Reduces MSU and CPU costs …. All while leveraging existing IT assets © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: High-Performance In-memory technology • How mainframe high-performance in-memory technology works • Uses a much shorter code path to access data • Top path is a typical DBMS code path (typ. 10, 000 to 100, 000 machine cycles) • Bottom path is the high-performance in-memory code path (typ. 400 machine cycles) © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Mainframe In-memory technology • How Mainframe in-memory technology works: • Highly accessed reference data identified using DB 2 stats, SMF 64 records, Strobe, etc. • Reference data copied into in-memory tables • Apps access it from there • Access all other data in the normal manner • No changes to the DBMS • No changes to batch apps (or minimal changes) © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Mainframe In-memory technology • The Benefit: • Time needed for batch apps to access RO data is reduced significantly • Overall batch run time is also sharply reduced • Overall resource usage is reduced • Overall operational cost is reduced © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Mainframe In-memory technology The Results: • Reduced batch application CPU usage • In certain environments, this technology can reduce monthly mainframe operations costs by 4 -8% • In these cases, it can also lower your system upgrade frequency from every 4 years to every 5 years (capex deferment) © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Multi-platform Data Replication • Why Multi-platform Data Replication? • Helps reduce resource usage for multi-platform batch applications that use ETL • Reduces elapsed time for these implementations as well • Also reduces monthly costs for these implementations • Improved batch performance & cost at the database level © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Multi-platform Data Replication Multi-platform Replication in batch processing scenarios: • Some organizations elect to run some mainframe batch processing on other platforms • Many of these operations have large investments in ETL infrastructure • Can be difficult to manage, and can consume personnel time depending upon the data transfer technology • Consumes CPU and MSU on the mainframe side, and is a regular and very heavy resource consumer on the enterprise network © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Smart Data Replication How Multi-platform Replication works: • ETL optimization / replacement • Your current ETL systems can be replaced/improved on by a multi-platform replication solution that leverages a highly efficient Changed Data Capture (CDC) • After an initial databases replication event, subsequent updates are fast, near-real time, and consume few platform and network resources. • Source and target databases are virtually identical at all times • Very low risk and low maintenance © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Multi-platform Data Replication The Benefits: • Removes an expensive, complex and highly resourceintensive process from the IT infrastructure • Removes the ETL bottleneck from all processes affected by it • Far less resource usage on mainframe systems and on the enterprise network • Replicated databases are in sync at all times © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Multi-platform Data Replication The Results: • Reduced batch application CPU usage • In environments that use ETL for batch processing offloading, this technology can reduce data transfer costs by 75% • In these cases, it can also lower your system upgrade frequency from every 5 years to every 6 years (capex deferment) © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: System level: IT business intelligence Why IT business intelligence? • Cost reduction/avoidance and transparency • Valuable information for decision makers on all levels • New information based on system information combines with customer information about organization, applications, finance • Better planning and forecasting, predictability • Control costs and impact of batch processing © 2017 Data. Kinetics Ltd. All rights reserved. Confident
IT business intelligence: Architecture Data collection Data ingestion & storage Analysis & Dashboarding © 2017 Data. Kinetics Ltd. All rights reserved. Confident
IT business intelligence: Business Mapping Technical Dimensions: • • Server Transaction Batch Job DB 2 plan or package Process SQL Server Databases Disk etc. Mapping Business Dimensions • • • Business Area (e. g. Capital Markets) Business System (e. g. Netbank) Owner (Who pays for the resource usage? ) Environment (e. g. test/prod) etc. © 2017 Data. Kinetics Ltd. All rights reserved. Confident
IT business intelligence: The usual z/OS Suspects • Identify Peak, Production and Batch analysis and optimization • Identify WLC optimization and SLO for cutting software costs • Synchronize business and workload priorities (WLM) • Exploit utility processors, mobile workload and software and hardware features • Identify and track most resource-consuming (expensive) applications, jobs, transactions and users to initiate dialog for implementing change / behaviour adjustments • Reduce time spent on reporting, capacity management and performance management • Follow up with IT outsourcers, and validate invoices © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: System level: IT business intelligence The Benefit: • Control the impact of your batch processing • Cost reduction/avoidance and transparency into the entire IT infrastructure – both mainframe AND midrange distributed systems • Dev. Ops: Hold departments accountable for performance & capacity impact of new development, application changes, etc. • Outsourcing: Make sure you have the information to hold your outsourcer accountable for their billing © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: System level: IT business intelligence The Results: • General • 10 -15% savings in MSU usage, using raw IT data • More, when implementing business dimensions • Batch • Controls costs and impact of batch processing © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: System level: Automated Soft Capping • Why Automated Soft Capping? • • Improves batch throughput capacity leveraging existing assets Reduces mainframe resource usage Reduces batch MSU and CPU costs Improves batch performance and reduces costs for batch processing at the systems level • Allows you to handle new workloads • Opens new opportunities in how you run your operations © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Automated Soft Capping • What Automated Soft Capping does: • Optimizes the performance of your system while controlling WLC • Automatically determines LPAR capacity that best fits performance objectives while avoiding capping constraints • Available MSUs can be shared between LPARs • Provides capacity on demand to ensure high-priority activity is not capped • Uses metrics collected for each LPAR at regular intervals: • MSU, R 4 HA, capacity, capping, etc. • Analyzes the collected metrics for each CEC and LPAR involved: • Can protect specific LPARs from participating • Can dictate activities by timeframe © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Automated Soft Capping How it works– Minimize Billing Contribution. When IMSU > R 4 HA DC just greater than R 4 HA When IMSU < R 4 HA DC just greater than IMSU DC MSUs taken from pool (or capping enforced) DC MSUs release to pool © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: System level: Automated Soft Capping • The Benefit: • Using soft capping automation lowers the trend line for environments experiencing growth in resource usage • Associated costs (monthly bills) are similarly controlled © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Database level: Automatic Bottleneck Detection Why Auto Bottleneck Detection? • • Helps reduce batch resource usage Also helps reduce the monthly costs associated with DB 2 batch processing Identifies and helps to eliminate DB 2 SQL bottlenecks affecting batch applications More effective than monitoring (often incapable of identifying bottlenecks that have existed all along) • Can identify legacy bottlenecks in production batch environments • Can also help to eliminate bottlenecks before they make it into production environments - they can be identified and eliminated in the development, test and staging environments © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Automatic Bottleneck Detection How it works: • SQL code rules are set from a database of rules (Custom rules are possible as well) • All SQL calls are verified against the rules database • Unacceptable code is rejected • Applies to Development / Test / Production environments • Quality of SQL is ensured (Also guarantees the quality of outsourced development) © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Automatic Bottleneck Detection The Benefit: • Enforced DB 2 SQL quality • Fewer costly coding problems make it to the field • No more quality concerns with your outsourcing plans © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Outside the Box: Database level: Automatic Bottleneck Detection The Results: • Reduced application CPU usage system-wide • A typical implementation can expect a 15 -20% reduction of MSU usage annually. • In many cases, the savings can be much more. • In these cases, it can also lower your system upgrade frequency from every 5 years to every 6 years (capex deferment) © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Summary Solving Mainframe batch window challenges • Most “inside-the-box” batch solutions can be costly, risky or marginally effective. • These “outside-the-box' solutions are effective, low-risk, budget-friendly: • Mainframe high-performance in-memory technology • Multi-platform replication • IT business intelligence • Attack this problem from 3 angles, from the application, database and system levels. • All low risk, reasonable cost and very effective. © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Q&A © 2017 Data. Kinetics Ltd. All rights reserved. Confident
Thank you for your time. Data. Kinetics +1. 613. 523. 5500 info@dkl. com
- Slides: 37