ENTERPRISE SEARCH AND ITS VALUE TO THE ENTERPRISE

ENTERPRISE SEARCH AND ITS VALUE TO THE ENTERPRISE or why search and retrieval of ‘relevant’ information is only the start in meeting the business need Lee Atkinson

Introduction A business view of enterprise search and its value to the enterprise. What are the issues? Is there a common problem domain that needs to be solved in many large organisations when trying to put in place a single 'view' of their business and then find what they need, when they need it. The presentation will examine the issues found in three main areas in realising enterprise search and extracting real business value from it; Core Competencies Find it Organise it Use it Internal Database External Licensed External General Information Gathering (structured Information and Organisation unstructured file types) Information Analysis Entity Profiling (single view) Business Applications Internal Unstructured News feeds and genre specific We will focus on a practical example of where simply having the ability to 'search' and be more 'relevant' has limitations in finding the right information at the right time, in the right context and enabling the information to be used. We will then go on to look at how search needs to encompass ALL of the data held in the enterprise and how certainty and completeness of results, more than relevancy, are the key requirements.

The Problem Domain The ‘reality’ of technology in large organisations and has led us to build a set of pre requisites that any solution has to be able to be deployed in and work within that ‘reality’. Organisation beyond the department level is unusual Chaos is part of a dynamic business, technology’s ability to support it always lags Data silos are a fact of life and wholesale replacement is not going to happen as a big bang. Even organisations that think strategically, deploy tactically to succeed Legacy applications are embedded in the heart of most global business Adapt, not rebuild where the end product achieves the same outcome. Integration and business change are the major problems in delaying the implementation of new technology. The more systems the more overhead in data reconciliation and mapping.

What search based solutions offer as an enabler Search based applications offer a different approach, one that can sit above the chaos and bring organisation to it. Relevancy and semantic search based business solutions offer a means of pulling together information into a common structure and not require changes to existing processes, systems or data storage and classification approach. Search can work with whatever is in place - silos, data quality etc - and create the information structures within the application without the need for any manual intervention. To the end user there is no difference – they get comprehensive information when they need it, how they need it. But most search based solutions are focused on the web and technology

FIND IT

Generic Search Enterprise search • Harder and harder to differentiate • Focus on sources and extraction • Relevancy at the margin – so what? • Increasing commoditisation Web 2. 0 Semantic Open source competitive delay Focused on web and new information sources • Completeness of internal information is fundamental • Being the most relevant, is not • Structured and unstructured is one and the same. • Having a relevant results set is not enough. • Near real time information gathering becomes a pre-requisite • Business process change is not an option if rapid deployment is a need Search as an information consolidator and organiser but on its own an enabler

Case study as an example Large Business – 14 offices, 1000+ people, each office different – but teams often from multiple offices, departments work on a single project Problem domain Need to collaborate across offices, no common structure or collaboration capability Images, CAD and unstructured data 90% of their information needs DMS purchased, but business change and information migration overhead meant that never implemented 8 TB External of Database 32 TB of internal unstructured Consolidated and Organised view Around an entity - Customer - Project - Supplier

Relevancy is the start point to information organisation 5/15/2007 Detection of new terms and entities from within corpus and expansion into taxonomies and entity lists Need to be able to organise around entity Extraction and Entity Profiling Categorisation Organisation against predefined categories and sub categories To be able to build inter relationships between categories such as; Type of cavity wall, projects it appeared, suppliers, customers, costs etc May also require combination of data mining with information retreival Classification and Sources Relevancy based Search Daft V 1 Search Optimisation Classification, Dictionaries (manual and automatic)

Enterprise Issues – once information is organised Needs to be near real time – critical systems, means upto date positions Needs to be complete information set, with audit trail and reconciliation A results set on screen fails the Business Value Proposition test Simply being smarter, quicker and more relevant is hard to build an ROI to business users Focus on business application to unlock value

ORGANISE IT

Piecing it together • Integrated single results set Takes search results as a start Delivers Business Framework • Defines results in context of entire point data set (what’s not there) • Adds business taxonomies • Pre determined ‘end state’ • Semantic, contextual, file types etc • Customer • All information applications are commodities that defined the • Projects • No difference between scope of the applications possible • People structured and unstructured • ‘Entities’ • Combines information into • Business Process • Analysis against rules • Exception based event management Core Competencies External Database 32 TB of internal unstructur ed Consolidated and Organised view Information Analysis Entity Profiling (single view) Business context R u l e s Business Applications Business Applications 8 TB of Business Application

USING IT AS A CORE BUSINESS RESOURCE

Search based business architecture – unlocking value Business Architecture 1 Information stays where it is 5 min updates 2 Blend DB and documents Its all information Business Application Real time MIS Core Competency Internal Database Business Application Business View Internal Unstructured External Licensed News feeds and genre specific Information Analysis (Bayesian, Entity Profiling Linguistic, (single view) Categorisation, Taxonomy, WEKA) Business rules define analysis Information enrichment SAN Single business view Single customer view Business Application Compliance Business Application Research Business Portal 3 Rules Engine External General Information Gathering Information (structured and Organisation unstructured 750 + file types) Multiple business applications Collaboration Portal Business Application Project View 4 Business Application Business Processes 5

ENTERPRISE SEARCH AND ITS VALUE TO THE ENTERPRISE or why search and retrieval of ‘relevant’ information is only the start in meeting the business need Lee Atkinson