Information Quality and Total Data Quality Management John

Information Quality and Total Data Quality Management • John Talburt • Acxiom New Products and Solutions • November 30, 2004

Data Quality The term "data quality" can best be defined as "the degree of fitness of data for a particular use. ” – Data quality is contextual – the quality of a fixed dataset can vary depending upon its application. – The terms Data Quality and Information Quality are interchangeable.

Pervasive Industry Problem “… enterprises haven't done a good job of collecting all the data that they should have, or they have it but it's not in sync across the enterprise… Synchronization will become an even bigger issue for businesses in the near future as they struggle to integrate the enormous amounts of data gathered …” Gartner: Poor Data Quality Dooms Many IT Projects May 14, 2004 Techweb--quote from Ted Friedman

Challenges • Data quality issues often recognized too late in the data integration process • Complex, dynamic data environments introduce many points of failure • Business impact of data is often not reflected in technology solutions • Data Sources can introduce changes that impact data stores or individual marts • Tactical solutions solve immediate problems without identifying & fixing underlying issues

Data Quality is Multi-Dimensional Huang, Lee, Wang, Quality Information and Knowledge, Prentice Hall, 1999. Quality Category Intrinsic IQ Contextual IQ Information Quality Dimensions Accuracy, objectivity, believability, reputation Relevancy, value-added, timeliness, completeness, amount of information Representational IQ Interpretability, ease of understanding, concise representation, consistency Accessibility IQ Access, security

Measuring Data Quality “ You cannot improve what you cannot measure. ”

Data Quality Metrics • Metrics are formulas or algorithms that summarize statistics about data into meaningful indicators of quality – Establish metrics within each dimension related to key customer “care-abouts” – Each metric has a goal, threshold, and/or tolerance level defined • Example - Completeness – 800, 000 valid telephone number in file (statistic) – 73% of 1, 100, 000 records (formula) – Score of 85 and GPA of 2. 97 (Acxiom Scorecard Metric for 50% failure, 73% Actual, 85% Goal, 100% possible)

Data Quality Management • Managing Information as a Product – Must understand the consumer’s information needs – Must manage information as the product of a well-defined production process – Must manage the life cycle of information products – Must have an information product manager (data steward) • Application of Total Quality Management (TQM) to the data manufacturing process – Total Data Quality Management (TDQM)

TDQM 1. 2. 3. 4. Quality Requirements (Goals) Performance Measurements Failure Analysis Implementation of Improvement

Benefits of TDQM • Infrastructure supplements “one-time” assessments • Data quality issues exposed through monitoring, rather than accidental discovery • Evidence captured to leverage data quality in the marketplace • Corporate stakeholders engage in DQ management process • Periodic, quantifiable measurements help – – – Reduce unnecessary production costs Reduce data expense Reduce liability exposure for errors and omissions Avoid lost revenue opportunities Improve customer relationships

Database View of Data Quality • Data Quality, Richard Wang, Mostapha Ziad, and Yang Lee, Kluwer Academic Publishers, 2001 • Extending the Relational Model to Capture Data Quality Attributes • Extending the E-R Model to Represent Data Quality Requirements • Automating Data Quality Judgment • Developing a Data Quality Algebra • MIT, European Union, and Purdue University Projects in Data Quality

Resources • MIT Information Quality Program http: //mitiq. mit. edu/MITIQ. asp • MIT Total Data Quality Management Program http: //web. mit. edu/tdqm/www/index. shtml • Proceedings: International Conference on Information Quality (ICIQ) 1996 -2004 http: //www. crg 2. com/iqconference/iciq/iqproceedin gs. htm • Reading in Information Quality (CD), R. Wang.

Questions John Talburt Jtalbu@acxiom. com Acxiom Corporation 1001 Technology Dr Little Rock, AR 72223