Azure Data Warehouse Solutions Raul Firu YOUR SUCESS

Azure Data Warehouse Solutions Raul Firu

YOUR SUCESS, OUR PASSION! – Driving Success as an experience makes us unique! Drivers CTO Office Technology Strategy 2020 SSMP Golden Rules Seite 2

To tomorrow’s Composable Enterprise? Evolvable by Design Interoperability Built for Change Open to Partners Customer Centric Rapid Development Fast Time to Market Internationalisation Localization 11/5/2020 Acquisitions at Scale Operate at Scale Efficiency at Scale http: //www. adamalthus. com/blog/2013/04/04/the-composable-enterprise/ 3 Seite 3

YOUR SUCESS, OUR PASSION! – Driving Success as an experience makes us unique! Today Seite 4

Manny data sources integrated SAP-ERP CAS (delta) Pegasus (delta) DO-Kontakte u. a. Beschwerden Support Kunden, Ansprechpartner Aufträge, Faktura Produkte, etc. VM (full) CRM* Oracle 11 AKA-Seminar Information MIWH Daten Over BI Applicati ons Chordiant *Teile aus SAPCRM direct Sugar* ecom NL-Bezieher, Lizenznehmer, Servicedaten Atlantik Portal-registrierer Whitepaper Downloader, etc. Econda *Teile aus Sugar CRM OSPIdesk tracking Online-produktund Support Tracking online Verhalten bedirect TM-Live Business-info, Hierarchien Push-Kontakte Add on Call-Data Salesforce Historische SP Aufträge ACD-Kontaktdaten Details Lead/Opp – weitere Aktivitäten Maileon / Inxmail E-Mail Kontakte, Bounces, etc. Wired Minds Lead-TM Index, Korrekturen, Neue Firmen/ASP Mafo Identifikation Firmen …. ad hoc bei Analysen ohne Registrierung Seite 5

Aktueller Beladungsprozess 23: 00 7: 00 SAP BW ungenutzt Reports SAP BW FI/CO Datenstand 23. 06. 2015 CRM 6: 00 Tagesgeschäft BI –Daten verfügbar 3: 00 ETL für MIWH Start d. Beladung MIWH 24. 06. um 19: 30 DB-Link MIWH (Oracle 11 g, Partitioning, OWB) FI/CO Datenstand SAP-Daten 23. 06. 2015 (23: 00) Verfügbar MIWH Delta 1 Tag 25. 06. 2015 (6: 00) BI –Daten von Ende 23. im MIWH verfügbar am 25. Berechnungen, Datenaufbereitung, , Anreicherung, Chordiant-Logiken, etc. SAP-BW Reports ERP 19: 30 JDBC Atlantic (Postgres DB) e. Com (Postgres DB) FTP TMDienstleister i. d. R. Tägliche Beladung des Vortags Flatfile Upload, API in Arbeit Cast Iron Sales, Service & Marketing (Salesforce) Seite 6

YOUR SUCESS, OUR PASSION! – Driving Success as an experience makes us unique! Vison Picture Seite 7

Target Architecture – API Driven Data Warehouse Integration Iaa. S Platform Sales Cloud API Mngmt. Adaption µServices Processed Data Processing Marketing Tool Row Data Analytics Tool Etc… Data Warehouse Existing Data Sources New Data Sources Data Connectors Seite 8

YOUR SUCESS, OUR PASSION! – Driving Success as an experience makes us unique! Building it all up Seite 9

Schema on Read vs Schema on Write – we need both Schema on Write Schema on Read - DB schemas - Query data in native format - Traditional RDBMS - Send data in native format (implicit ETL) - Easy to query - Implicit schema - OLTP use cases - More resilient interfaces - Late binding schemas are good for discoveries Good for known unknowns Good for unknowns Seite 10

Why cloud? Cost efficiency Scalability No up-front invest in HW Self service A lot of out of the box tools (vendor dependent) However… Virtual HW can influence performance Seite 11

YOUR SUCESS, OUR PASSION! – Driving Success as an experience makes us unique! Azure data tool belt Seite 12

YOUR SUCESS, OUR PASSION! – Driving Success as an experience makes us unique! Azure data tool belt Seite 13

YOUR SUCESS, OUR PASSION! – Driving Success as an experience makes us unique! A deeper look Seite 14

Azure Blob Storage Used as data landing zone HDFS compliant Very cost effective 10 T=2. 5 k/year Is easily connected SQL stores Basic Authorization management Easy API Supports multiple Hadoop clusters Seite 15

Azure SQL Data Warehouse Self service SQL DB optimized for Warehousing T-SQL support, Stored Procedures, etc Poly. Base – query language used for connecting blob storage, HDFS with DW Elastic cost model build around storing data with flexible compute Seite 16

Different flavors of Hadoop Use Azure as Iaa. S + bring your own Hadoop stack Hortonworks or Cloudera as Managed Service HDInsight Azure Data Lake Services Seite 17

Azure ML / Data Marketplace Visual Machine Learning solution Focused on the less tech-savvy analyst Woks with SQL, HDFS and Blob Storage sources Start form scratch or from marketplace templates Deploy any experiment as o. Data service Gets less attention at the moment… Seite 18

Azure ML / Data Marketplace Visual Machine Learning solution Focused on the less tech-savvy analyst Woks with SQL, HDSF and Blob Storage sources Start form scratch or from marketplace templates Deploy any experiment as o. Data service Gets less attention at the moment… Seite 19

(HDInsight) Spark Fast growing OSS Analytics platform In memory support Scales linearly Supports various languages for working with data – each with it’s own strength Has access to ML libraries and it is used more and more as an ML engine Stream/real-time analytics support Seite 20

Real-time with Event Hub Massively scalable Evening framework Integrates with SQL and HDFS stores Good. NET SDK Great for IOT or social use cases Has an out of the box API but probably needs a service in front Seite 21

Data Factory Data focused ETL Service Connects to all types of data stores Can be used to transform data inside a data lake The “advantage” over OSS solutions like Pig/Tez is that is it visual It’s not a full Middleware – lacks the service endpoints Seite 22

Own plumbing Adaption Services in docker containers Web APIs with WCF, Cloud Services, Azure App service/API Apps VPN or Express Route Etc… Seite 23