THE BIG DATA ECOSYSTEM AND YOU 1 HP
THE BIG DATA ECOSYSTEM AND YOU! 1 HP Confidential
BIG DATA ECOSYSTEM OVERVIEW DIAGRAM: External Users Backoffice (ERP) Social Media and Consumer Sentiment Internal Users Statistics Transactional Applications Unstructured Data ETL Transactional Data (OLTP) 2 HP Confidential Business Intelligence Data Warehouse
BIG DATA ECOSYSTEM OVERVIEW DIAGRAM: External Users Backoffice (ERP) Transactional Applications ETL Transactional Data (OLTP) 3 HP Confidential Social Media and Consumer Sentiment Transactional Data – Source Systems and/or Point of Sale Internal • Vary Greatly from Company. Users to company • EX: Insurance. Statistics Policy and Claims Systems, Brokerage Unstructured Data Systems, Academic Trade Enrollment Systems, Enery Smart Grids, Hospital Patient Business Intelligenceetc, etc…. . Management Systems, Data ERP Data – Operational Management, Planning and Feedback Warehouse • EX: Supply Chain, Finance & Accounting, Inventory Management, HR, CRM, etc. . .
Big Data Decision Point! – Data Mart or EDW ETL, Replication, Data Quality Bulk & Trickle Loads ETL, Replication, Data Quality Key Drivers: • What is the Size of your Source Data? Enterprise Data Warehouse • Should Big Data sit inside or Outside of your PEDW? • Normalize or Denormalize? SQL, • ODBC, JDBC Integrated Reporting or Isolated Reporting? App. /Reporting front-end 4 HP Confidential BI or Reporting Interface
BIG DATA ECOSYSTEM OVERVIEW DIAGRAM: ETL – Extraction, External Transformation, Loading Users Social Backoffice (ERP) Or Transactional Applications Extraction, ELT – Transformation Loading, Unstructured Data ETL Transactional Data (OLTP) 5 HP Confidential Media and data from one system to Moving Consumer Sentiment and getting it right is another often Internal the most frustrating and time. Users consuming piece of the BI Statistics Ecosystem puzzle Quality - Time - $$$ (is good enough… good enough? ) Business Intelligence Data Warehouse
Big Data Decision Point! – ETL, ELT or Data Virtualization? Key Drivers: • What is the Quality of your Source Data? • How complex is the model? • Can the database handle ELT batch rework in time? • Do you have the right tools (Informatica, Composite. SW)? Can you live with the maintenance overhead, performace? • 6 HP Confidential
BIG DATA ECOSYSTEM OVERVIEW DIAGRAM: Data Warehouse(s? ) External Users Central Storage Tank for most the company’s most critical Transactional data Backoffice (ERP) Applications True success requires Executive Evangelism Transactional The ecosystem lives or dies Data (OLTP) based on the performance of your data warehouse! 7 HP Confidential Social Media and Consumer Sentiment Internal Users Often very difficult to ROI: “Are your users satisfied with your Statistics company’s BI delivery? ” - Gartner Unstructured Data ETL Business Intelligence Data Warehouse
Old School Tech (Row Store) or New School Tech (Columnar, No. SQL)? AAPL NYASE NYAASE NYASE NGGYSE NYGGGSE NYSE 143. 74 NYSE 5/05/09 Key Drivers: AAPL BBY • Can your database handle your data volumes? • Does the cost of performance tuning justify status quo? • Real-time or batch? • Commodity MPP or Big Iron? Are you monetizing your data or heating your data center? • 8 HP Confidential
BIG DATA ECOSYSTEM OVERVIEW DIAGRAM: Social Media and Consumer Sentiment Business Intelligence and Advanced Analytics External Users Backoffice (ERP) Is your corporate culture one that views Business Intelligence output as a key driver to your company’s Transactionalsuccess? Applications 9 HP Confidential Statistics Unstructured Data Can analysts use BI output to make positive, ETL effective, fact-based decisions? Transactional Data (OLTP) Internal Users Data Warehouse Business Intelligence
In-Database Analytics or Application Based Analytics? Key Drivers: • Performance – Cubes or Relational? • Are you living in a canned report or ad-hoc world? Does your business have highly advanced, specialized analytics needs? • Do your data mining users require the aid of an analytics GUI? • 10 HP Confidential
BIG DATA ECOSYSTEM OVERVIEW DIAGRAM: Unstructured Data: External Users The final piece of the Backoffice (ERP) big-data Social Media and Consumer Sentiment Social Media puzzle? Transactional there. Applications data floating Is around the you need to understand but can’t figure out how to analyze? ETL Web logs / Clickstream Call. Statistics Center Unstructured Data Consumer Sentiment Business Do you know Transactional what’s happening on your company’s network? How Data (OLTP) consumers feel about your product? If your call centers are actually helping your business? 11 HP Confidential Internal Users Lobs, Blobs, and Clobs Intelligence Data Warehouse
Hadoop / Vertica: ETL Hadoop or not to Hadoop? Computer Cluster Data data data data data data data data data data data data data data data DFS Block 1 Map DFS Block 1 Key Drivers: DFS Block 2 Map Reduce DFS Block 2 • Is it critical you understand your company’s messiest data? data data data data data Map DFS Block 3 • DFS Block 3 Do you have the skills and know-how? • Hadoop / Vertica: Advanced Analytics HDFS vs Map. Reduce Computer Cluster • Could you take action on the information if you had it? DFS Block 1 Map DFS Block 2 • Exposing Map. Reduce results through BI or EDW DFS Block 2 Map Reduce DFS Block 2 Map • 12 Are there alternatives to Hadoop? DFS Block 3 HP Confidential DFS Block 3
Big Data Trends - Data Governance: Data About Your Data • Do you really understand your data? Do your users? • How many versions of the truth are there? Does your organization have the commitment to build adhere to standards? • 13 HP Confidential
Big Data Trends - Cloud: You *MAY* put your data in the Cloud, but DON’T put your Head there! • Public cloud vs. Private Cloud vs. Saa. S – What’s best for you? • Is it secure? Is it disaster-proof? • Does is perform? • Is it cost effective? 14 HP Confidential
Big Data Trends – Mobile: If you’re not delivering Mobile BI to your user and customers, you’ve fallen behind. • Tools are making Mobile Delivery easier every day. Your customers expect it. Your internal users *should* expect • it. • Do you have a Mobile Data Security Strategy? 15 HP Confidential
Advice for Survival in the Big-Data world: • BE PROACTIVE IN TRANSFORMING YOUR BUSINESS! • STAY ON TOP OF NEW TECHNOLOGIES! • PARTNER WITH THE BUSINESS! • BE AWARE OF THE BIG DATA TECHNOLOGY YOU INTERACT WITH EVERY DAY, AND APPLY IT TO YOUR BUSINESS! 16 HP Confidential
THANK YOU FOR YOUR TIME… AND GOOD LUCK IN YOUR BIG DATA EFFORTS! 17 HP Confidential
Q&A 18 HP Confidential
- Slides: 18