Business Analytics Data Monetization Other Big Data Adventures
Business Analytics Data Monetization & Other Big Data Adventures A Telecommunications Use Case (Story) Dirk Jungnickel, March 14 th, 2017 du 2017. Proprietary and Confidential 1
Agenda Business Analytics 1. A Short History of Analytics (in du) 2. The Big Data Journey 3. Challenges and Learnings 4. Outlook du 2017. Proprietary and Confidential 2
A Short History of Analytics (in du) Du: 2007 – 2013 Business Analytics • Small-medium integrated teleco operator in the UAE • Mobile market entry in 03/2007 Ø Quickly reaching 50% mobile market share • Fixed-line market: Ø Several 100 k subs, triple-play services Ø Market recently opened up for competition du went through a tremendously successful initial growth phase of 5 -6 years, during which, quite understandably, Analytics was not a priority. du 2017. Proprietary and Confidential 3
A Short History of Analytics (in du) Du: 2013 – Now Business Analytics • Then things became more challenging: Ø Saturated market (200% mobile penetration) Ø Competition significantly fiercer. . . Ø Decline of ARPUs due to OTT players • Resulting strategic changes: Ø Analytics recognized as key strategic enabler to improve core telco business Ø New ‘adjacent services’ from Io. T to Smart City… Ø. . . many of which are part of the ‘data economy’, i. e. enabled by Analytics For the past 3 -4 years, Analytics was increasingly recognized as a key enabler for strategic differentiation and new revenue streams. du 2017. Proprietary and Confidential 4
A Short History of Analytics (in du) Business Analytics Where to Start: Small or Big Data? Initial Big Data Strategy Data & Info. Scope First fix the basics before worrying about Big Data External (Big) Data Other Internal (Big) Data Customer/Product Reference & Transaction Data c rip s De e tic tiv Di ag s no e tiv ive Pr ict ed ip cr Analytics Type es Pr For the first 1. 5 years, focus was entirely on fixing the basics and putting standard analytics capabilities in place around (small & relational) transactions telco data. du 2017. Proprietary and Confidential 5
The Big Data Journey Business Analytics Why Big Data? • Only a minority of telcos maximize business benefits from analytics programmes • These are the ones that apply Analytics systematically across the business… • . . . beyond Marketing and Sales into the Network domain Business value from Analytics in telcos is maximized by applying Analytics beyond Marketing and Sales into the Network domain. Telco Network data is Big Data! du 2017. Proprietary and Confidential 6
The Big Data Journey Business Analytics The Telco Big Data Conundrum For decades now, telecommunications has used and operated two interwoven Io. T networks: The Telco Production Factory Io. T (aka Telco Network) The Subscriber Phone Io. T • Millions of subscribers carrying connected Things (aka phones)… • Thousands of pieces of connected equipment Things (aka Base Stations, TRXs, Core NW elements, …)… • …continuously exchanging Big Data/Information with each other • . . . continuously logging and storing Big Data/Information of what they are doing and what is going on Telco Operators had Io. T Networks & Big Data long before the rise of Google or Facebook. We should have invented Hadoop & Spark! But then, we should have invented Facebook and Whats. App too… du 2017. Proprietary and Confidential 7
Agenda Business Analytics 1. A Short History of Analytics (in du) 2. The Big Data Journey 3. Challenges and Learnings 4. Outlook du 2017. Proprietary and Confidential 8
The Big Data Journey Additional Motivation (In Case Still Required) du 2017. Proprietary and Confidential Business Analytics 9
The Big Data Journey The Risks of Attempting to Ride a Flying Unicorn Business Analytics “Only 15% of businesses reported deploying their big data project to production, effectively unchanged from last year (14%). ” Source: October 4, 2016 Gartner Press Release There are profound differences in complexity and cost between Hadoop pilots, POCs or lab projects and a true Enterprise Big Data production setup. du 2017. Proprietary and Confidential 10
The Big Data Journey How Did We Approach The Unavoidable Big Data Project? Business Analytics Start by setting up a Hadoop Cluster (tempting)…. . . or hold our horses and follow a use-case driven approach? 1 Big Data Use Case Roadmap (priority/benefit vs. complexity) with associated Data Sources 2 3 Selection of Tools & Technologies as well as implementation partner Big Data project: first set of E 2 E business use cases as deliverables In retrospect, one of the key success factors was to identify feasible and maximum benefit business use cases, and thereby secure strong and broad company sponsorship for such a large project. du 2017. Proprietary and Confidential 11
The Big Data Journey 1 Business Analytics Selected Business Use Cases Ø Based on Mobile NW Probe Data 1. Mobile Network Quality of Service Analytics (40+ Qo. S KPIs) Ø Approx. 2 -3 TB raw data per day Ø Processing initially in batchmode… 2. Geo-Location Subscriber Analytics Ø. . . to be followed by streaming analytics du 2017. Proprietary and Confidential 12
The Big Data Journey 1 Where is the Value: 1. Network Qo. S Analytics Ø Descriptive: What’s going on, down to cell and subscriber-level… Ø Diagnostic: Understand why Business Analytics Applications Ø Value-based Network Optimization & Planning… Ø. . . resulting in improved Customer Experience Ø Predictive: o Location-based Qo. S degradation prediction o Cell-level Traffic forecasting Ø Prescriptive: Add customer & cell value dimension…. . . and optimize actions for highestvalue / lowest-cost impact Ø Significant Cap. Ex & Op. Ex reduction o Preventive actions o Small cell technologies du 2017. Proprietary and Confidential 13
The Big Data Journey 1 Business Analytics Where is the Value: 2. Geo-Location Analytics Applications Ø Descriptive: : Ø Data Monetization: Information & insight products o Subs-level location history o Footfall, catchment, dwell-time, etc. analysis for various customer profiles & micro-segments… o Retail industry: E. g. footfall analysis o Financial services: E. g. international credit card fraud, etc. o. . . including full population extrapolation o Road & Transport: E. g. road and city planning o Advertisement: E. g. Customer-profile based advertisement locations Ø Prescriptive: Various optimization capabilities (higher value) o Optimal location (e. g. retail or advertisement) o Traffic planning optimization o Etc. Ø Geo-Marketing (du): o E. g. optimal retail or outdoor advertisement location, etc. Ø Smart Dubai Support: o du 2017. Proprietary and Confidential Subscriber-level geolocation feed 14
The Big Data Journey 1 Business Analytics Internal Benefits vs. Data Monetization Benefits More than 100 Mio US$ annually • Recent Mc. Kinsey study: Cap. Ex reductions of >5% revenues are achievable Internal Benefits Data Monetization • For a company like du, a reduction by 3% translates into Cap. Ex savings of US$ 100 Mio yearly • Total annual UAE data monetization market after several few years: around US$ 50 Mio Few Mio US$ annually Time While Data Monetization might sound more sexy than NW analytics, initially it is a small byproduct of Telco Big Data initiatives. In the long run it has the potential to deliver significant revenue growth. du 2017. Proprietary and Confidential 15
The Big Data Journey 1 Data Sources: Mobile Core NW Probing Data Business Analytics Data sources comprise more than 65 different core network interfaces and amount to 2 -3 TB per day. The resulting project complexity including associated data engineering work was significant. du 2017. Proprietary and Confidential 16
The Big Data Journey 2 Partner & Tool/Technology Selection Business Analytics Implementation Partner Selection Criteria: • Specialization (!) • Architectural soundness against requirements • Capability to provide ops support post-launch • Cost Tool/Technology Decision Points: • Proprietary solution vs. Hadoop/Spark: Decided prior to partner Rf. P • Do-it-yourself vs. Hadoop Distro vs. Appliance: Decided through partner selection • Specific Hadoop components and proprietary elements: Decided through partner selection du 2017. Proprietary and Confidential On commodity HW 17
The Big Data Journey 3 Business Analytics Enterprise Data Hub Overview Data Sources Comments du Internal Use Closed Loop User Access Policy BSS • EDH is built and operated by du’s IT department Reports Enterprise Data Hub OSS E-DWH Teradata Storage Secure Landing … Polystar Data Hadoop Storage • • Access Policy Data Monetization Conference Presentations Smart Dubai Support Etc. Data Curation (incl. Big Data Tools & Data Mining) Aggreg. & Sharing Policy Security Tools Aggreg. & Sharing Policy External Use Data Reports Sharing Policy • du’s Business Analytics function is the main user of this hub • ‘Usage’ includes development of reports and dashboards… • . . . as well as data mining, predictive and prescriptive models and algorithms A key architectural decision taken in du several years ago, was to minimize movement of data, i. e. centralize management and analysis of data in an Enterprise Data Hub. du 2017. Proprietary and Confidential 18
The Big Data Journey 3 Zoom-In: Hadoop Secure Landing Area Business Analytics • Data sources: Probe e. XDRs, DWH reference data & rated CDRs • Bedrock used to ingest files into secured area and tokenize PIDs • Secure Landing Area and secured Vault only accessible by IT Security User Group du 2017. Proprietary and Confidential 19
The Big Data Journey 3 Business Analytics Zoom-In: Processing Layer • Two types of processing: – KPI Calculation (40+ Qo. S KPIs) – Geo Location Feed • Automated job flows orchestrated by Bedrock. • Metadata and workflows managed by Bedrock. • Primary Users: ETL group du 2017. Proprietary and Confidential 20
The Big Data Journey 3 Business Analytics Zoom-In: Access Layer • Two egress points – Export to DWH: detokenized – REST APIs for external parties (e. g. Smart Dubai): tokenized • REST APIs secured through SSL, HTTPS authentication methods du 2017. Proprietary and Confidential 21
Agenda Business Analytics 1. A Short History of Analytics (in du) 2. The Big Data Journey 3. Challenges and Learnings 4. Outlook du 2017. Proprietary and Confidential 22
Challenges & Learnings Business Analytics Project Key Success Factors 1 • Secure broad buy-in at the start by making this a businessfocussed and use-case driven project rather than a Big Data Technology project 2 • Security and Data/Privacy Protection 3 • Highly cross-functional exercise: Strong project mgmt for coordination and alignment between involved functions 4 • Specialized implementation partner, which allowed to focus on non-technical challenges, which are all related to internal coordination and buy-in du 2017. Proprietary and Confidential 23
Challenges & Learnings Non-Project C&Ls – What is Analytics Good For Anyways? Business Analytics • Contrary to widespread believe (in particular in the Analytics community), Corporate Decision making is usually not based on Analytics and systematic validation / falsification of hypothesis through data & facts… • …but on the 3 Ps of Corporate Decision Making: Politics, Persuasion, and Power. Point Don’t underestimate the challenges of putting analytics capabilities to business use, i. e. promoting an Analytics Driven Decision Culture. No (Big Data) technology will help you with this task. . . du 2017. Proprietary and Confidential 24
Challenges & Learnings Non-Project C&Ls – The Mystical ‘Data Scientist’ Busy in Fin Services industry, preparing the next financial crisis Nerds Math & Stats Coding Real skill challenge for this project Awesome Nerds! Nerds in Suits Rare Nerds Business Knowledge “Data Scientists” – Sexiest Job in the 21 st century (HBR). Management Consultants Business Analytics The Real Skill Challenge • Don’t get carried away and jump straight into ‘Data Science’! • (Even though this might impress your friends, colleagues and spouse) • The main challenge often is Data Engineering, not Data Science! • In our case, core NW knowledge combined with coding and ETL • Try to let a DS without NW understanding correlate raw data from the plethora of NW interfaces to identify, e. g. a dropped call… While Data Scientist job openings do exist, individuals who could fill them apparently only exist in analytics fairytales (aka whitepapers) or (supposedly) in Silicon Valley. Hence, for many companies Data Science will actually be a team job! du 2017. Proprietary and Confidential 25
Agenda Business Analytics 1. A Short History of Analytics (in du) 2. The Big Data Journey 3. Challenges and Learnings 4. Outlook du 2017. Proprietary and Confidential 26
Outlook Business Analytics Current Status and Next Steps Status: • Up and running in a true Production setup since Dec 2016, i. e. nearly 3 months of raw probe data • Finalists in global Gartner Data & Analytics Excellence Awards in the category of Best Data Management and Infrastructure Sweating the asset: • Qo. S Prediction • Various NW Quality of Service dashboards • Various Data Products in Retail and FS industry • Internal Geo-Analytics for optimized Retail Rollout New use cases (H 2/2017): • DPI data • DWH & ETL Offloading • Security use-cases du 2017. Proprietary and Confidential 27
Business Analytics Thank You ! ﺷﻜﺮﺍ Dr. Dirk Jungnickel, SVP Business Analytics, du dirk. jungnickel@du. ae, +971 -55 -9531334 du 2017. Proprietary and Confidential 28
- Slides: 28