Integrating Search And BI Satish Ramanan April 16

Integrating Search And BI Satish Ramanan April 16, 2011

AGENDA Context Why - Integrate Search with BI? How - do we get there? - Tool Strategy What - is in it for me ? - Outcomes & Benefits Who - should I go to? - Reference Vendors

Context Explosion of Web 2. 0 Data Extended Scope of Enterprise Information Management Structured and Unstructured data Unified Information Access Guided Analytics Growing

Search v/s BI SEARCH BI Querying unstructured data Querying structured databases Humungous volumes of data that is scattered all over Size is usually known and manageable Documents, web pages, emails, etc. Tables and columns Standard, Simple and brain-dead easy Interface – Usually a search box or an Advanced Search box Complex formats taking time and effort to build – Reports, Dashboards, Charts & Graphs Typically uses keywords as inputs Selection criteria using one or more fields as input Usually a dump of results with drill down capability Aggregate and visualize data in tabular or graphical formats with or w/o complex calculations. Drill down possible

Search with BI A system that gives u best of both worlds A user interface that can consume simple inputs but produce complex outputs Ability to access and query information contained in tables and columns in a database/warehouse as well as web pages, documents, email Ability to predict or forecast future outcomes based on such unified information Use innovation to succeed – linguistic technology, converting simple text to SQL

Why Integrate Search with BI Information Explosion within organizations (and outside!) Gigabytes of structured data lying unused Terabytes of unstructured data lying unexplored and unattended Unstructured data seen providing critical insights to business rendering it almost indispensible True Predictive Analytics depends upon ability to analyze outputs from unified information sources

Possible Approaches Categorized BI Search A Search engine indexes both metadata and data generated A list of search results based on keyword is displayed in the main body Categories / Sub Categories of results derived from metadata are displayed on the left side Results contain drilldowns to records in the source systems and reports that are created on the fly Eg. www. dice. com Web. Focus Magnify

Possible Approaches…. 2 Natural Language Processing (NLP) Search Uses Linguistic technology to decipher the meaning of keywords used Technology maps the meaning of keywords to metadata of databases or documents Users can query database using plain English (some other languages as well) that will be converted to SQL automatically by the technology Technology actually uses a knowledgebase of concepts, business rules, jargons & acronyms The generated SQL is displayed alongwith the results, which can then be converted into a table, chart, or dashboard. Easyask, Semantra

Possible Approaches……. 3 Visual Search Not really a search, but uses advanced BI tools that runs a visualization search directly against an analytic platform The source is usually an in-memory columnar database that helps returns results extremely fast Using a point and click paradigm, users can sort, filter, group, drill, and visualize the results SAP Business Objects Explorer Accelerated, Endeca Information Access

Typical Challenges Attempts to unify structured and unstructured data facing major road blocks Limited awareness Lack of appropriate tools and technology Data ownership lying with disparate groups Huge time and costs involved in efforts to build a unified information repository ROI questionable and can be easily challenged Big Bang approach coupled with Quest for the most economical solution leading to major failures

Choosing the right tool Text Analytics Treatment of Data Relationships Query flexibility Information Availability Secure Access Ability to Build/Develop Powerful Interactive Applications easily Enterprise Readiness Total Cost of Ownership

Outcomes and Benefits Increasing demand for High Performance DB Engines Teradata, Netezza, Exadata, Greenplum Increasing demand for Real Time Integration Need to accelerate Information access Providing TRUE ‘Single Source Of Truth’ Analytics getting closer to ‘Predictive’ than ‘historical’ Enhanced Knowledge Harvesting and Management

Vendors SAP - Business Objects Explorer ENDECA - Latitude ATTIVIO – Active Intelligence Engine INFORMATION BUILDERS – Webfocus Magnify FAST GOOGLE Analytics

THANK YOU Satish Ramanan Satish_ramanan@yahoo. com +91 9769417381