MAJOR ISSUES FROM DATAWAREHOUSING AND DATA MINING NAME

MAJOR ISSUES FROM DATAWAREHOUSING AND DATA MINING. NAME OF STUDENTS: TOSHIT KHOBRAGADE NISHANT NANDESHWAR 59 46 SUBJECT: - DATAWAREHOUSING AND MINING.

DATA MINING ISSUES v Mining Methodology and User Interaction v Performance Issues v. Diverse Data Types Issues

Mining Methodology and User Interaction Issues ØData mining query languages and ad hoc data mining − Data Mining Query language that allows the user to describe ad hoc mining tasks. ØPresentation and visualization of data mining results − Once the patterns are discovered it needs to be expressed in high level languages, and visual representations. These representations should be easily understandable. ØHandling noisy or incomplete data − The data cleaning methods are required to handle the noise and incomplete objects while mining the data regularities. If the data cleaning methods are not there then the accuracy of the discovered patterns will be poor. ØPattern evaluation − The patterns discovered should be interesting because either they represent common knowledge or lack novelty.

ØMining different kinds of knowledge in databases − Different users may be interested in different kinds of knowledge. Therefore it is necessary for data mining to cover a broad range of knowledge discovery task. ØInteractive mining of knowledge at multiple levels of abstraction − The data mining process needs to be interactive because it allows users to focus the search for patterns, providing and refining data mining requests based on the returned results. ØIncorporation of background knowledge − To guide discovery process and to express the discovered patterns, the background knowledge can be used.

Performance Issues ØThere can be performance-related issues such as follows − ØEfficiency and scalability of data mining algorithms − In order to effectively extract the information from huge amount of data in databases, data mining algorithm must be efficient and scalable. ØParallel, distributed, and incremental mining algorithms − The factors such as huge size of databases, wide distribution of data, and complexity of data mining methods.

Diverse Data Types Issues ØHandling of relational and complex types of data − The database may contain complex data objects, multimedia data objects, spatial data, temporal data etc. ØMining information from heterogeneous databases and global information systems − The data is available at different data sources on LAN or WAN.

Data Warehouse Issues ØDesign Issues ØExtraction Issue ØUnderestimation of resources of data loading ØHidden problems with source systems ØRequired data not captured ØIncreased end-user demands ØData ownership ØHigh maintenance ØComplexity of integration

Data Warehouse Design Issues Ø Small systems are easier to manage than large ones. Ø Adding attributes is easier than changing or deleting them.

Data Warehouse Extraction Issues Internal data often comes from separate operational databases. z Reconcile formats z Remove intelligent codes z Verify accuracy z Unify time stamps

Data Warehouse Underestimation of resources of data loading ØSome times we underestimate the time required to extract, clean, and load the data into the warehouse. Ø It may take the significant proportion of the total development time, although some tools are there which are used to reduce the time and effort spent on this process.

Data Warehouse. Hidden problems with source systems ØSome times hidden. problems associated with the source systems feeding the data warehouse may be identified after years of being undetected. Ø For example, when entering the details of a new property, certain fields may allow nulls which may result in staff entering incomplete property data, even when available and applicable.

Data Warehouse Ownership Issues ØData warehousing may change the attitude of end-users to the ownership of data. Ø Sensitive data that owned by one department has to be loaded in data warehouse for decision making purpose. Ø But some time it results in to reluctance of that department because it may hesitate to share it with others.

Data Warehouse Complexity of integration ØThe most important area for the management of a data warehouse is the integration capabilities. ØAn organization must spend a significant amount of time determining how well the various different data warehousing tools can be integrated into the overall solution that is needed. ØThis can be a very difficult task, as there a number of tools for every operation of the data warehouse.

Thank You