Optim Virtual Users Group Test Data Management Co

  • Slides: 27
Download presentation
Optim Virtual Users Group Test Data Management Co. E January 22, 2020 Blue Cross

Optim Virtual Users Group Test Data Management Co. E January 22, 2020 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

Topics for Discussion 1. TDM Enterprise Shared Services 2. Co. E Journey and Capability

Topics for Discussion 1. TDM Enterprise Shared Services 2. Co. E Journey and Capability Model 3. Data Governance - NPDM 4. Questions 2 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

TDM Services Catalog Test Data Support: Full-Program On multi-year, multi-track/wave programs ETDM provides comprehensive

TDM Services Catalog Test Data Support: Full-Program On multi-year, multi-track/wave programs ETDM provides comprehensive data services: • A data office is set up for the duration of the program • Data Governance/Architecture Support • Analytical Services • Program Lead that ensures complete coverage • Sophisticated dashboard reporting • All required documentation produced • Tooling: TDM Accelerators, Test. M, Oracle & DW. Often additional capabilities constructed to support program data requirements. Test Data Support: Analytical • Targeted DW querying to identify a sufficient data pool for testing. • Production Synchronization – Comparison / Gap analysis between Prod and MO. • Tooling: TDM Accelerators, Oracle & DW. Post Go-Live and Production Support • Production Support Data • Command Center Support • Resources (contract staffing) • Tooling: HP Quality Center, IBM Rational Development Support • Unit Test Data Support Services • Resources (contract staffing) • Tooling: HP Quality Center, IBM Rational Test Data Support: One-Off Portfolio or M & E • Data Governance/Architecture Support • Analytical Services • Resources (contract staffing) • All required documentation produced • Tooling: TDM Accelerators, Test. M, Oracle & DW. Test Data Support: Annual Enrollment • Membership Services • Resources (contract staffing) • Tooling: TDM Accelerators, Test. M, Oracle & DW. 3 Test Data Support: Regression • Analytical Services • Resources (contract staffing) • Sophisticated dashboard reporting • All required documentation produced • Tooling: TDM Accelerators, Test. M, Oracle & DW. Test Data Provisioning • Membership, Providers, Claims, other X 12 s, etc. • Tooling: TDM CORE Data Architecture Support • Map existing data flows • Construct program/project data flow models • Test Harness Design Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT) Data Governance Support • NPDM Operations • Data De-Identification/Re-Identification • Security Exception Support • Tooling: IBM Info. Sphere Optim & Information Analyzer: TDM ESS 11/1/2019

TDM Core Maturity Model Timeline Leverage Tools to Scale Leverage Process Automation Capabilities Maturity

TDM Core Maturity Model Timeline Leverage Tools to Scale Leverage Process Automation Capabilities Maturity 2014 -2016 2017 -2018 2019 ü Edifecs Test. M ü Info. Sphere Optim ü Oracle & SQL DBs ü Custom Claims Scripting ü Point Solutions for Projects & Departments 4 ü Members Edge Load Tool ü Data Request System ü BMI Re-formatter ü Data Privacy Analysis ü Membership Valuator ü Data Governance ü Plus >20 other Accelerators ü Data Refresh / Search & Reserve Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT) TDM Self-Service 2020 ü On-Line Search & Reserve ü On-Line Masking Requests ü Data Services to support: ü Automation ü Dev. Sec. Ops ü Data Virtualization

How is the TDM Center of Excellence Defined? 5 Blue Cross Blue Shield of

How is the TDM Center of Excellence Defined? 5 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

TDM COE Capabilities Dashboard 6 Blue Cross Blue Shield of Massachusetts | Quality Assurance

TDM COE Capabilities Dashboard 6 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

TDM CORE Capabilities Maturity 2 3 4 1 7 Blue Cross Blue Shield of

TDM CORE Capabilities Maturity 2 3 4 1 7 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

TDM CORE Capabilities Maturity 1 8 Blue Cross Blue Shield of Massachusetts | Quality

TDM CORE Capabilities Maturity 1 8 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

Search & Reserve Capability: Target State including TDM Portal Release TDM Portal will have:

Search & Reserve Capability: Target State including TDM Portal Release TDM Portal will have: An enterprise wide Test Data Self Service Tool that supports Search & Reserve Membership / Provider / Claims / Product Architecture/Approach • Continuous periodic refresh of data from ODS UAT data store to Test Data Mart • Data Mart source schema has the exact replica of ODS tables and is periodically refreshed by copying only the test data created by the TDM team from ODS UAT • De-normalized tables are created in the Data Mart target schema based on the Source DB schema • Search and Reserve portal functions use the Data Mart Target DB schema to retrieve data for the end users • Pre-defined set of search criteria are defined in the search page and whenever the user clicks any particular search criteria, SQL statements are generated dynamically which produce the test data as per the search condition • Search/Reserve metrics to be stored in the Portal Repository 9 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

Search & Reserve Capability: Service Layer Daily Data Refresh Request Placed Includes Attributes: 100

Search & Reserve Capability: Service Layer Daily Data Refresh Request Placed Includes Attributes: 100 members (from any Group), Female 18 -26 YO, with HMO Family Plan 10 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT) Result Set

TDM CORE Capabilities Maturity 2 11 Blue Cross Blue Shield of Massachusetts | Quality

TDM CORE Capabilities Maturity 2 11 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

UDF (in-Place) Masking Overview: • The IBM® Info. Sphere® Optim™ Data Privacy solution provides

UDF (in-Place) Masking Overview: • The IBM® Info. Sphere® Optim™ Data Privacy solution provides comprehensive capabilities to mask and govern sensitive data effectively across applications, reports and databases in production and nonproduction environments. Data masking is the process of systematically transforming confidential data elements such as personal health information (PHI) and personally identifying information (PII) into realistic but fictionalized values. User Defined Functions (UDFs) is available with IBM Optim Masking on Demand (MOD) software which is part of the IBM Info. Sphere Optim Data Privacy solution UDF Masking is supported for limited database platforms including Oracle, DB 2 LUW, DB 2 ZOS, Teradata, SQL Server, Netezza. • • • Use Case: • Applicable to Data Masking for very large data sources due to the elimination of data movement/processing between the data source and the IBM optim server. All transformations happen within the database server Enables Dynamic data masking or On-Demand data masking feature through the UDF masking for the supported database platforms Supports integration of data masking functionality within the application code • • Benefits: • Improved performance benefits over the traditional optim masking methodology for very large databases due to less movement of data outside of the database server. Reduced storage costs over the traditional approach due to the elimination of extract and convert files Ease of implementation for real time masking via incremental data pushes from production databases to non-prod environments Customization made possible in application functionality • • • 12 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

UDF Masking Approach UDF Masking using ODPP Typical UDF masking activities • Install and

UDF Masking Approach UDF Masking using ODPP Typical UDF masking activities • Install and configure Optim MOD software on the Application Programs NPDM Masking Scripts String Mask UDF Oracle Server AIX Date Mask UDF Number Mask UDF DB 2 Server Timestamp Mask Netezza Server • • UDF Functions ODPP Libraries Oracle Software DB 2 Software Netezza Software Linux Windows Solaris HP/UX z/OS ODPP is an acronym for Optim Data Privacy Provider ODPP provides a set of built-in providers for performing data masking/privatization and these services are provided in DLL/library format on a wide variety of platforms Optim provides a set of UDF masking functions for each supported data type that enable the masking process within the database 13 • • Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT) • target database server as a one time activity Register the UDFs on the target database Custom code UDF calls for invoking masking functions against the sensitive data stored in the database Create intermediate staging tables to reduce the turn around time (TAT) for masking of sensitive tables Perform data validation against the staging table and the table that contains sensitive data Complete the data masking process Examples: select Optim. Mask. Str('972 -861 -6708', 'pro=aff, MTD=REP, flddef 1=(name=c 1, dt=char)') from sysibm. sysdummy 1; The Optim. Mask. Str call for phone number 972 -861 -6708 produces a masked value of 797 -996 -5900 with the above UDF call

De-Identification: Standard versus UDF Approach Standard Approach Temp Table Data Source E R XT

De-Identification: Standard versus UDF Approach Standard Approach Temp Table Data Source E R XT T AC /nas/npdm Sensitive Table UPDATE OR INSERT TR Data Source AN S FO R M Temp Table /nas/npdm 14 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT) Source Table UDF UDF

TDM CORE Capabilities Maturity 3 15 Blue Cross Blue Shield of Massachusetts | Quality

TDM CORE Capabilities Maturity 3 15 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

Big Data Hadoop Masking Leveraging Hive Data at Rest Steps followed • • •

Big Data Hadoop Masking Leveraging Hive Data at Rest Steps followed • • • Performance Metrics Install and Configure IBM Optim MOD software locally on the server Extract data locally on the HDFS Staging server by connecting to hive The hive table namely hyb_publish_cns. enrol_member_demo_201 90723 is extracted into a pipe delimited file Mask the extracted file using IBM Optim MOD Java API The masked file is then loaded back into the hive table using the Truncate and Load approach File Name File Type No of Records File Size Extract Convert Load Comments Member Enrollment Delimited (pipe) 1, 000 254 MB 00: 01: 47 00: 01 00: 22 Total Columns Available/Masked: 25/3 First Name, Last Name, Zip Code Member Enrollment Delimited (pipe) 10, 000 2. 3 GB 00: 03: 07 00: 09: 17 00: 54 Total Columns Masked: 25/3 First Name, Last Name, Zip Code Member Enrollment Delimited (pipe) 34, 249, 617 7. 4 GB 00: 08: 48 00: 31: 27 00: 01: 40 Total Columns Masked: 25/3 First Name, Last Name, Zip Code 16 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

Big Data Hadoop Masking Leveraging Hive Data in Motion 17 Blue Cross Blue Shield

Big Data Hadoop Masking Leveraging Hive Data in Motion 17 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

TDM CORE Capabilities Maturity 4 18 Blue Cross Blue Shield of Massachusetts | Quality

TDM CORE Capabilities Maturity 4 18 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

4 Introducing… The TDM Portal 19 Blue Cross Blue Shield of Massachusetts | Quality

4 Introducing… The TDM Portal 19 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

4 TDM Portal Release Plan • Central Way to Access TDM Services • Accessible

4 TDM Portal Release Plan • Central Way to Access TDM Services • Accessible by everyone • Customer-focused, creates transparency • Next evolution for TDM of Process Automation Release Plan: • Release 1: Test Data Requests 11/2019 - Delivered • Release 2: Masking Requests 2/2020 – In Process • Release 3: Live Heatmap 4/2020 – Design Completed • Release 4: Self Service Data & Masking Requests – 6/2020 20 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

Holistic Approach to Data Protection 21 Blue Cross Blue Shield of Massachusetts | Quality

Holistic Approach to Data Protection 21 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

Tech_177 Non-Production Data Management (NPDM) Our Mission: Inform Sr. Leadership on the Data Compliance

Tech_177 Non-Production Data Management (NPDM) Our Mission: Inform Sr. Leadership on the Data Compliance in Non-Prod. Clean as much as is reasonable and begin Governance monitoring of all covered Non-Prod environments. Project Structure: Wave-Based Rollout – First Cleanup, then Governance and data support 22 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

NPDM Technology Requirement from ET Architecture: Tooling for De-identification needs to work with all

NPDM Technology Requirement from ET Architecture: Tooling for De-identification needs to work with all current and future data management types and platforms. 23 • Two Regions Built: Dev & Test • Access is Provided thru Security Groups using Service IDs whose Passwords are in PAM • Capability for working with all data types in the BCBSMA ecosystem: Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

NPDM: Building Non-Production Governance The first metric to determine our level of compliance is

NPDM: Building Non-Production Governance The first metric to determine our level of compliance is the Denominator: – what is the total number of Non-Production regions that we are governing? For NPDM, the TOTAL NUMBER OF REGIONS in our full scope (Waves 1 -6) = 155 As of 12/31/2019, 111 of the 155 regions are in Governance = 72% Data Governance Metrics Current Regions Governed: 111 Current STATUS for all Regions: Compliant Non- Compliant 24 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT) 111 0

NPDM Dashboard NPDM Data Governance Cleanup In Progress In Governance 12/31/2019 25 Blue Cross

NPDM Dashboard NPDM Data Governance Cleanup In Progress In Governance 12/31/2019 25 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

TDM Dashboard Business Portfolio Project TDM Contracts in 2019: 28 Ad-Hoc Test Data Project

TDM Dashboard Business Portfolio Project TDM Contracts in 2019: 28 Ad-Hoc Test Data Project Requests in 2019: 17 26 Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

Questions? Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)

Questions? Blue Cross Blue Shield of Massachusetts | Quality Assurance & Testing (QAT)