IBM Information Integration Capabilities Overview of IBM Information
IBM Information Integration Capabilities Overview of IBM Information Server © 2005 IBM Corporation
Corporate View of Information Architecture Is Changing § Information is the key to Business Innovation – Organizations highly effective at driving information integration are 5 times more likely to drive value creation – Information architecture can’t exist in a vacuum – it needs to be tied to enterprise architecture 87% of CEOs believe fundamental change is required in next two years to drive innovation Over 60% of CEOs believe their organizations need to do a better job leveraging information Source: 2006 IBM Global CEO Survey 2
What is Driving the Change? – Gartner Perspective Efficiency Enterprise Agility Process Simplification n Differentiation Across the Enterprise Eliminate redundancy Drive to standardization Promote reuse and data quality n Enterprise n n Compliance n n Reduce risks with conflicting sources Make information transparent Trx. Customers Employees Partners Real Time Information "Infoglut" n n Manage expanding volume and velocity Control unstructured content Vendor Consolidation n n Products Reduce integration burdens n n Orgs. Enable closed-loop analytics Immediately integrate with partners, suppliers Single View E-Mail Reports Spend less on same functionality/technology M&A n Sense and respond Provide consistency, accuracy Support continuous information flows Rapid orchestrate processes Financials n Web Content n Create consistent and holistic view across all channels Manage relationships Revenue Optimization Databases Documents Media Management Across All Content n n Support top-line growth on crosssell/upsell Leverage global purchasing power Enterprise Information Management: Getting Value From Information Assets Gartner Business Intelligence Summit 2006 David Newman 6 -8 March 2006 3
Customer Business Issues § Too much information and not knowing what’s important – Not using demand signals to drive supply chain – Not using customer analysis to tailor marketing and sales – Not leveraging valuable unstructured information § Multiple versions of the truth – Problems managing customer, product and partner interactions – Regulatory compliance inhibited by poor transparency § Lack of trusted information – Incomplete, out-of-date, inaccurate, misinterpreted data – Difficult to understand or control how information is used § Lack of agility – Inability to take advantage of opportunities for innovation – Escalating costs due to inflexible systems and changing needs 4
How Gartner Defines the Requirement Business Process Composition Integrated Composition Technologies Business Services Repository Custom Applications Product Content & Data Management Customer Data Integration External Services Package Applications Enterprise Content Management Business Intelligence Applications Data Services • Data Movement • Data Enrichment • Data Stewardship • Data sourcing • Data Transformation • Content Integration • Data Access • Data Quality Metadata Management and Semantic Reconciliation • Models • Schemas • Repositories • Standards • Business Rules • Search • Classification Across Transactional, Operational and Analytical Sources Customer Master Data Product Master Data Asset Master Data External Data Sources Enterprise Data Warehouse Across Structured, Semi-Structured and Unstructured Content 5 Enterprise Information Management: Getting Value From Information Assets Gartner Business Intelligence Summit 2006 David Newman 6 -8 March 2006
The Construction of Our Platform IBM Mainframe Integration Replication Event Pub (pre-2002) Federation 2002 Integrated Matching Integrated Profiling Integrated Cleansing Content Integration 2003 2004 6 2005 Unstructured Information Mgmt 2006 Integrated Semi-structured Data Handling SOA Deployment Grid Deployment Metadata Integration Ascential Acquisition Ascential Architectural Unification IBM Information Server
The IBM Solution: IBM Information Server Delivering information you can trust IBM Information Server Unified Deployment Understand Cleanse Transform Deliver Discover, model, and govern information structure and content Standardize, merge, and correct information Combine and restructure information for new uses Synchronize, virtualize and move information for in-line delivery Unified Metadata Management Parallel Processing Rich Connectivity to Applications, Data, and Content 7
IBM Information Server Architecture UNIFIED USER INTERFACE Analysis Interface Development Interface Web Admin Interface COMMON SERVICES Metadata Services Unified Service Deployment Security Services UNIFIED PARALLEL PROCESSING Understand Cleanse Transform UNIFIED METADATA Deliver COMMON CONNECTIVITY Structured, Unstructured, Applications, Mainframe 8 Logging & Reporting Services Design Operational
Why Is it Important to Start with Understanding? § Where is my information? § How do I get it when I need it? § What does it mean? § Can I trust it? § How do I get it in the form I need? § How do I get it where it needs to go? § How do I control it? 9
Physical Metadata: IBM Information Analyzer § Data-centric analysis of application, database and -based sources file § Secure, detailed profiling of fields, across fields, and across sources Subject Matter Experts Understand Data Analysts IBM Information Analyzer Analyze source data structures, and monitor adherence to integration and quality rules § Creation of metadata from profiling results § Results instantly promotable across IBM Information Server Physical View 10
Business Metadata: IBM Business Glossary § Web-based authoring, managing & sharing of business metadata § Aligns the efforts of IT with the goals of the business § Provides business context to information technology assets Subject Matter Experts Understand Business Users IBM Business Glossary Create and manage business vocabulary and relationships, while linking to physical sources § Establishes responsibility and accountability Database = DB 2 GL Account Number Schema = NAACCT Table = DLYTRANS Column = ACCT_NO data type = char(11) 11 Technical Business The ten digit account number. Sometimes referred to as the account ID. This value is of the form L-FIIIIVVVV. Business View
Logical Metadata: Rational Data Architect § Data modeling for data structures and federations Subject Matter Experts § Federated data discovery § Metadata relationship discovery & mapping Architects Rational Data Architect Create and manage business vocabulary and relationships, while linking to physical sources § Impact analysis, and synchronization across models § SQL & XML generation capabilities 12 Data Modeling & Mapping
Role-Based Tools with Integrated Metadata Data Admin Implementers IBM Quality. Stage Database application and transformation development Architects Rational Data Architect Metadata and data-driven data modeling and management Subject Matter Experts, Data Stewards Analysts IBM Business Glossary IBM Information Analyzer Business definition & ontology mapped to physical data Data-driven analysis, reporting, monitoring, data rule and integration specification IBM Metadata Server § Simplify integration 13 § Facilitate change management & reuse § Increase compliance to standards § Increase trust and confidence in information
IBM Metadata Server – at the Core of IBM Information Server IBM Rational Data Architect IBM Business Glossary IBM Information Analyzer IBM Data. Stage Metadata Access Services Analysis Services IBM Metadata Server 14 IBM Quality. Stage Meta. Brokers
Graphical Impact Analysis and Lineage Provide Trust HTML View Path View Graphical Tree View 15
Why Should I Care About Cleansing Information? § Lack of information standards – Different formats & structures across different systems § Data surprises in individual fields – Data misplaced in the database § Information buried in free-form fields Kate A. Roberts 416 Columbus Ave #2, Boston, Mass 02116 Catherine Roberts Four sixteen Columbus APT 2, Boston, MA 02116 Mrs. K. Roberts 416 Columbus Suite #2, Suffolk County 02116 Name Tax ID Telephone J Smith DBA Lime Cons. Williams & Co. C/O Bill 1 st Natl Provident HP 15 State St. 228 -02 -1975 025 -37 -1888 34 -2671434 508 -466 -1200 6173380300 415 -392 -2000 3380321 Orlando WING ASSY DRILL 4 HOLE USE 5 J 868 A HEXBOLT 1/4 INCH WING ASSEMBY, USE 5 J 868 -A HEX BOLT. 25” - DRILL FOUR HOLES USE 4 5 J 868 A BOLTS (HEX. 25) - DRILL HOLES FOR EA ON WING ASSEM RUDER, TAP 6 WHOLES, SECURE W/KL 2301 RIVETS (10 CM) § Data myopia – Lack of consistent identifiers inhibit a single view § The redundancy nightmare – Duplicate records with a lack of standards 16 19 -84 -103 RS 232 Cable 6' M-F Cand. S CS-89641 6 ft. Cable Male-F, RS 232 #87951 C&SUCH 6 Male/Female 25 PIN 6 Foot Cable 90328574 90328575 90238495 90233479 90233489 90345672 IBM I. B. M. Inc. Int. Bus. Machines International Bus. M. Inter-Nation Consults I. B. Manufacturing 187 N. Pk. Str. Salem NH 01456 187 N. Pk. St. Salem NH 01456 187 No. Park St Salem NH 04156 187 Park Ave Salem NH 04156 15 Main Street Andover MA 02341 Park Blvd. Bostno MA 04106
Data Cleansing: IBM Quality. Stage § Specialized data quality functions seamlessly integrated with Data. Stage § Visual tools for defining complex matching and survivorship logic Subject Matter Experts Cleanse Data Analysts IBM Quality. Stage™ Standardize and correct source data fields, and match records together across sources to create a single view § Ensures clean, standardized, deduplicated information § Enables a single version of the truth Visual Match Rule Design 17
What Is Important About Transformation & Delivery? § Transformation is key to enabling information to be used in new business contexts – it needs to be metadata-driven § Designed for use by information experts using the understanding imparted by the metadata Data Analysts § Transformation and Delivery can be reused across multiple mechanisms Data Architects DBAs Subject Matter Experts Logic Reuse – Large volume batch movement – Real-time event-driven response – Service-oriented architecture Request Response – Federated query Query 18
Data Transformation & Movement: IBM Data. Stage § Codeless visual design of data flows with hundreds of built-in transformation functions § Optimized reuse of data integration objects Developers Transform Architects IBM Data. Stage® Transform and aggregate any volume of information in batch or real time through visually designed logic § Leverages parallel processing without requiring design changes § Capable of supporting batch and real-time operations 19 Hundreds of Built-in Transformation Functions
Data Federation: IBM Federation Server § Access diverse & distributed information as if it were in one system § Industry leading query optimization with single sign-on, unified views, and function compensation IBM Federation Server Federate Access and integrate heterogeneous information across multiple sources as if they were a single source Extend value of existing analytical applications by providing real-time access to integrated information § Transactional write capabilities across heterogeneous sources § Visual tools for federated data discovery & data modeling 20 Visual Federation Design
Federated Queries Make Integration as Easy as SQL SELECT parameters_return_billto_key as BILL_TO_KEY, billto_company_name, parameters_return_shipto_key as SHIP_TO_KEY, CASES_SHIPPED, GROSS_SALES, Single SQL Query Joins: URL FROM GETKEYSSOAP_GETKEYSREALTIME_NN, Web Service GLOBAL_SALES_TRAN_NN, XML Documents BILLTO_DIMENSION, Data Warehouse URL_INVOICES Unstructured Data WHERE and and 21 getkeysrealtime_ship_to_number = '13546' getkeysrealtime_ship_to_number = URL_INVOICES. shipno ltrim(rtrim(translate(ship_to_number, ' ', x'0 a'))) = getkeysrealtime_ship_to_number parameters_return_billto_key = billto_key ltrim(rtrim(translate(sales_order_number, ' ', x'0 a'))) = URL_INVOICES. orderno;
Rapid SOA Deployment: IBM Information Services Director § Packages information integration logic as services that insulate developers from underlying sources § Allows these services to be invoked as EJB, JMS, or Web services § Provides load balancing & fault tolerance for requests across multiple Information Servers Developers Architects IBM Information Services Director Flexibly deploy and manage reusable information services without hand coding § Provides foundation infrastructure for Information Services Rapid SOA Deployment 22
Common Programming Model ESB Web Services EJB JMS SCA/SDO IBM Information Services Director Shared Services Integrated Metadata Management Metadata Services & Service Registry Logging, Security Load Balancing, Availability Common Service Backbone Other Services Common Configuration, Installation Administration and Reporting IBM Information Server 23 Design Operational
Actionable Information Services Portal Call Center Portal Order History Service Customer Order History IBM Information Server Customer Master Service Customer Data Cleansing Service Customer Order Handling Process Server Customer Info Receive Request Check Order Policy Order Status Customer Review Order Check Account Ship Order 24 Customer
Service Oriented Architecture Information as a Service is Key Your business process platform needs an enterprise information management strategy. Gartner, April 2006 Portal Server ESB Process Server IBM Information Server You will waste your investment in SOA unless you have enterprise information that SOA can exploit. Gartner, March 2005 25
Actionable Information Services provide a basis for trust in information – providing visibility into lineage, relationships to other systems, and business definition IBM Information Server Order History Service Customer Order History Customer • Where does the information come from? • What happens to it along the way? • How does this fit into how the business defines things? • How do I know I’m using the right service? Other Data Sources 26 Content Repositories
Customers Achieve Significant Productivity Benefits 1 Example ETL Project Approx. Project Effort 30% Source System Analysis 50+% gain 20% Data Cleansing 50+% gain 20% Transformation Logic Construction 40+% gain 15% Data Management Services 20+% gain 10% Application System Connectivity 30+% gain 100% 1 Compared 27 to hand coding – gathered from IBM project studies
The IBM Information Server Advantage A Complete Information Infrastructure § A comprehensive, unified foundation for enterprise information architectures, scalable to any volume and processing requirement § Auditable data quality as a foundation for trusted information across the enterprise § Metadata-driven integration, providing breakthrough productivity and flexibility for integrating and enriching information § Consistent, reusable information services—along with application services and process services, an enterprise essential § Accelerated time to value with proven, industry-aligned solutions and expertise § Broadest and deepest connectivity to information across diverse sources: structured, unstructured, mainframe, and applications 28
29
- Slides: 29