Pentaho Analytics for Big Data SEPTEMBER 2013 1
Pentaho Analytics for Big Data SEPTEMBER, 2013 1 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555
Pentaho Mission The Future of Analytics: Big Data Exploration without Boundaries Modern, unified data integration and business analytics platform • Native integration into big data ecosystem • Embeddable, cloud-ready analytics Fast and Broad Innovation • Open source development model Critical mass achieved 2 • Over 1, 000 commercial customers • Over 10, 000 production deployments © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555
Pentaho Business Analytics Platform Our approach addresses the challenges. Data Integration Connect Visualize Report Dashboard Analyze Explore Access Integrate Cleanse Enrich Data Discovery Predictive Analytics Score Forecast 100% Java 3 Open Web-based API’s © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 Multi-Tenant Ready
Architectural Approach STREAMLINE ACCESS Information Delivery All Data Sources Traditional Relational Data ERP / CRM / Enterprise Apps (e. g. SAP, Oracle) Cloud Hadoop, No. SQL Data & Analytical Unstructured & semi-structured METADATA LAYER ‣ Proactive ‣ Operational ‣ Enterprise ANALYSIS ‣ (e. g. Salesforce, Amazon, Dell) REPORTING Direct Access ‣ INTEGRATE, CLEANSE, & ENRICH DATA VISUALIZE & Report Information In Any Style ‣ ‣ Graphical ETL Designer Enterprise Scalability Hadoop Clustering ‣ ‣ ‣ Relational OLAP Cubes In Memory Caching High Performance ‣ Ad hoc Exploration ‣ Multi-Dimensional DASHBOARDS ‣ Interactive Metrics ‣ Rich Visualizations DATA MINING ‣ Data Integration ‣ (XML, Excel, Files, etc. ) DELIVER When & Where Users Need It STANDALONE Web Mobile E-Mail Print INTEGRATION ISV & Packaged Applications Saa. S / Cloud Applications Advanced & Predictive Analytics CENTRAL ADMINISTRATION, AUDITING & MONITORING 4 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 4
The Value of Big Data for our Customers Big opportunities Drive incremental revenue • Predict customer behavior across all channels • Understand monetize customer behavior Improve operational effectiveness • Machines/sensors: predict failures, network attacks • Financial risk management: reduce fraud, increase security Reduce data warehouse cost 5 • Integrate new data sources without increased database cost • Provide online access to ‘dark data’ © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555
Traditional Big Data Analytic Solutions. 01 Data Preparation 02 Metadata Modeling 03 BI & Analytics Pain Multiple Tools 6 + Heavy Months [or more] Labor Average Time © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555
Pentaho Visual Development eliminates need for complex coding. Scheduling 7 Integration © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 Manipulation Ingestion Modeling
Visual Map. Reduce Processing VISUAL MAP REDUCE 1. MAPREDUCE INPUT – CALLING DATA 2. CALCULATE MONTH, DAY OF WEEK 3. EXTRACT 3 DIGIT AREA CODE Raw Data 4. LOOKUP GEO MASTER DATA IN HDFS Master Data Parsed Data 5. FILTER FOR WEEKEND AND US ONLY CALLS Analytic Datasets 6. CREATE “VALUE” FIELD FOR KEY-VALUE PAIR Java Programing 7. CREATE “KEY “ FIELD FOR KEY-VALUE PAIR 8. MAPREDUCE OUTPUT – KEY-VALUE PAIR 8 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555
Pentaho Visual Map. Reduce Drag&Drop then run in the cluster Parallel execution as Map. Reduce in the Hadoop cluster. As much as 15 x faster than hand-written code. 9 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555
Adaptive Big Data Layer Greater Flexibility and Insulation from Change and Risk Transparent access to and integration of big data • Insulates from changing versions, vendors, data stores • Give customers broad flexibility and choice, rapid time to value, reduced risk • Provides native integration into big data ecosystem • Broadest, deepest Big Data Support: • Hadoop: Cloudera, Hortonworks, Map. R, Intel • No. SQL: Mongo. DB, Cassandra • Specialized: Splunk Access Once, Process, Combine, Consume Anywhere 10 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555
Pentaho Map. Reduce Case Study Major Financial Institution Business Challenge • Gain competitive advantage through intraday balance reporting for commercial customers • Technical proof of concept: • Job 1: extract, clean/manipulate, load 1. 2 M records in Hadoop HDFS (10 lookups to HDFS) • Job 2: load processed results into RDBMS Java Map. Reduce 2 Java developers from major SI Staff Dev Time 2 days 13 min 45 secs Execution Time 47 secs Incorrect data due to coding errors Results 1+ month • Job 1: 5 min 43 sec • Job 2: 7 min 45 sec 11 1 ETL Pre-sales © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 (15 x faster) • Job 1: 20 secs • Job 2: 27 secs (17 x faster) Accurate Results 11
Pentaho Map. Reduce Case Study Who: Mobile Advertising Company What: Daily Job comparing incoming millions of rows to existing billions in a Hive table. Results: Native Hive Query 12 Map. Reduce 1 Consultant Staff 30 Minutes Dev Time 6 Hours 75 Hours Execution Time 17 Min © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 1 Consultant (264 x faster)
Leveraging Hadoop with Pentaho Data Management Platform – Visual Map Reduce, Orchestration, Connectivity – Fusion of all data sources & processing – Control/Manage/Optimize flow of data Hybrid – Leverages non-Hadoop infrastructure 13 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 OEM – Flexibility, Extensibility, Architected to Embed Pricing – One of top reasons customers choose us Community/Open Source Cache – Similar to Hadoop 13
Soundcloud 200 million users/month, 8% of the internet, 12 h/min Business Challenge • Quickly analyze and understand user behaviour • Capture Listens, Sounds, Users, Comments, Favorites, Shares, Reposts, Impressions, Clicks, Conversions, Suggestions, Downloads, Tags, Uploads • Listening generates >6, 000 events/sec Pentaho Benefits 14 • Easy to use tools • Avoided data silos • Simple and fast access to data with personalised reporting and time for “deep dive” analytics © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555
Data Warehouse Optimization Cost effective, fast processing Business Challenge • Gain competitive advantage through intraday balance reporting for commercial customers • Use Hadoop and relational data stores to process huge volumes 15 x faster to develop 10 x faster to execute No coding Pentaho Benefits 15 • Graphical orchestration for Hadoop, Hbase & DB 2 data integration workloads • 15 x faster to develop, 10 x faster to execute © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 Easy to find resources Integrate with existing
Complete Big Data Analytics & Visual Data Management Pentaho Big Data Fabric Data Ingestion Manipulation Integration 16 Enterprise & Ad Hoc Reporting © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 Data Discovery Visualization Predictive Analytics
Reports 17 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 17
Analyzer 18 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 18
Dashboards 19 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 19
Dashboards 20 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 20
Mobile 21 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555 21
Why Pentaho Modern architecture Purpose built data integration for analytics Big data & cloud analytics Fully battle tested 10, 000 strong developer community 80% more cost effective 22 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555
Parting Shot “Cognizant received the recognition for Cognizant Enterprise Analytical App Engine (CEAAE), among the world’s largest enterprise-class Business Intelligence platforms with the lowest per-user cost. Used by well over 150, 000 Cognizant employees globally, it offers extensive analytical capabilities, real-time data access, and predictive analytics. Its innovative add-ons custom-built on top of the technology stack have enabled comprehensive democratization of information assets across Cognizant. ” http: //news. cognizant. com/cio 100 -jun 3 -2013 23 © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555
Thank You JOIN THE CONVERSATION. YOU CAN FIND US ON: 24 blog. pentaho. com Facebook. com/Pentaho @Pentaho Business Analytics © 2013, Pentaho. All Rights Reserved. pentaho. com. Worldwide +1 (866) 660 -7555
- Slides: 24