Introduction To Data Mining What Is Data Mining

  • Slides: 31
Download presentation
Introduction To Data Mining

Introduction To Data Mining

What Is Data Mining? • A tool • Extraction of interesting (non-trivial, implicit, previously

What Is Data Mining? • A tool • Extraction of interesting (non-trivial, implicit, previously unknown and potentially useful) patterns or knowledge from huge amount of data • Core of KDD • Integration of Multiple technologies

Part of KDD (Knowledge Discovery in Databases) Interpretation/ Evaluation Data Mining Preprocessing Knowledge Patterns

Part of KDD (Knowledge Discovery in Databases) Interpretation/ Evaluation Data Mining Preprocessing Knowledge Patterns Selection Preprocessed Data Target Data adapted from: U. Fayyad, et al. (1995), “From Knowledge Discovery to Data Mining: An Overview, ” Advances in Knowledge Discovery and Data Mining, U. Fayyad et al. (Eds. ), AAAI/MIT Press

Integration of Multiple Technologies Artificial Intelligence Machine Learning Database Management Statistics Visualization Algorithms Data

Integration of Multiple Technologies Artificial Intelligence Machine Learning Database Management Statistics Visualization Algorithms Data Mining Other knowledge

Why Data Mining? • We are drowning in data (Data explosion problem ), but

Why Data Mining? • We are drowning in data (Data explosion problem ), but starving for knowledge! • Solution: Data warehousing and data mining – Data warehousing and on-line analytical processing – Mining interesting knowledge (rules, regularities, patterns, constraints) from data in large databases • A lot of potential applications – Market analysis and management • Target marketing, customer relationship management (CRM), market basket analysis, cross selling, market segmentation – Risk analysis and management • Forecasting, customer retention, improved underwriting, quality control, competitive analysis – Health care …

Data mining process State the problem + hypothesis Knowledge-base

Data mining process State the problem + hypothesis Knowledge-base

Knowledge from Data Mining • Association rules • Sequential Association • Classification rules •

Knowledge from Data Mining • Association rules • Sequential Association • Classification rules • Clustering • Deviation Detection • …

Association Rules • Identify association in the data: E. g. market-basket analysis • Indicate

Association Rules • Identify association in the data: E. g. market-basket analysis • Indicate significance of each “Find groups of items commonly purchased together” (correlation [A, B] and causality[A->B]) association (only interesting if its confidence exceed a certain measure) • Not all the Association is interesting (too trivial, negative association) – People who purchase fish are likely to purchase wine

Sequential Associations • Find event sequences that are unusually likely • Requires “training” event

Sequential Associations • Find event sequences that are unusually likely • Requires “training” event list, known “interesting” events • Must be robust in the face of additional “noise” events Uses: • Failure analysis and prediction Technologies: • Dynamic programming (Dynamic time warping) • “Custom” algorithms “Find common sequences of warnings/faults within 10 minute periods” – Warn 2 on Switch C preceded by Fault 21 on Switch B – Fault 17 on any switch preceded by Warn 2 on any switch

Classification rules • Classify a set of data based on their values in certain

Classification rules • Classify a set of data based on their values in certain attributes • Requires “training data”: have predefined attributes Uses: • Profiling Technologies: • Generate decision trees (results are human understandable) • Neural Nets “Route documents to most likely interested parties” – English or non-english? – Domestic or Foreign?

Clustering • Group a set of data base on the conceptual clustering principle(i. e.

Clustering • Group a set of data base on the conceptual clustering principle(i. e. maximizing the intraclass similarity and minimizing the interclass similarity) • No “training data”: Without predefined attributes Uses: • Demographic analysis Technologies: • Self-Organizing Maps • Probability Densities • Conceptual Clustering “Group people with similar travel profiles” – – – George, Patricia Jeff, Evelyn, Chris Rob

Deviation Detection • Find unexpected values, outliers Uses: • Failure analysis • Anomaly discovery

Deviation Detection • Find unexpected values, outliers Uses: • Failure analysis • Anomaly discovery for analysis Technologies: • clustering/classification methods • Statistical techniques • visualization • “Find unusual occurrences in IBM stock prices”

Popular Data Mining Techniques • Supervised – Decision trees – Rule induction – Regression

Popular Data Mining Techniques • Supervised – Decision trees – Rule induction – Regression models – Neural Networks … • Unsupervised —K-means clustering —Self organized maps …

Supervised vs. Unsupervised • Supervised algorithms » Learning by example: – Use training data

Supervised vs. Unsupervised • Supervised algorithms » Learning by example: – Use training data which the value of the response variable is already known – Create a model by running the algorithm on the training data – Identify a class label for the incoming new data » Driven by a real business problems and historical data • Unsupervised algorithms » Do not use training data. » Patterns may not be known in advance

Supervised Algorithms

Supervised Algorithms

Decision Trees • A tree structure where non-terminal nodes represent • tests on one

Decision Trees • A tree structure where non-terminal nodes represent • tests on one or more attributes and terminal nodes reflect decision outcomes. Advantages of decision trees —Understandable —Relatively fast —Easy to translate into SQL queries • Disadvantages of decision trees — Limited to one output attribute — Decision tree algorithms are not so stable • Types of decision trees —CHAID: Chi-Square Automatic Interaction Detection —CART: Classification and Regression Trees …

Figure 1. 1 A decision tree for the data in Table 1. 1

Figure 1. 1 A decision tree for the data in Table 1. 1

Rule induction IF = Antecedent THEN = • The extraction of useful • •

Rule induction IF = Antecedent THEN = • The extraction of useful • • independent if-then rules from data based on statistical significance If rules cause prediction confliction -> solve it according to confidence Advantage and disadvantage —Understandable —not cover all the possible situation Consequence E. g. IF Swollen Glands = Yes THEN Diagnosis = Strep Throat IF Swollen Glands = No & Fever = Yes THEN Diagnosis = Cold IF Swollen Glands = No & Fever = No THEN Diagnosis = Allergy

Neural Networks • Non-linear predictive models that learn through • • training and resemble

Neural Networks • Non-linear predictive models that learn through • • training and resemble biological neural networks in structure Means of efficiently modeling large and complex problems in which there may be hundreds of predictor variables that have many interactions Disadvantage – Difficult understand – Can require significant amounts of time to train, to prepare data –…

Figure 2. 2 A multilayer fully connected neural network

Figure 2. 2 A multilayer fully connected neural network

Regression Models • Statistical techniques • Using existing values to forecast what other values

Regression Models • Statistical techniques • Using existing values to forecast what other values will be. Y = a + b 1(X 1) + b 2(X 2) + b 3(X 3) + b 4(X 4) + b 5(X 5) … • A lot of types regression (linear regression, logistic regression …)

K-Means Clustering • Unsupervised algorithm • Steps of algorithm Choose a value for K,

K-Means Clustering • Unsupervised algorithm • Steps of algorithm Choose a value for K, the total number of clusters. Randomly choose K points as cluster centers. Assign the remaining instances to their closest cluster center. 4. Calculate a new cluster center for each cluster. 5. Repeat steps 3 -5 until the cluster centers do not change. 1. 2. 3.

A Hypothesis for the Credit Card Promotion Database A combination of one or more

A Hypothesis for the Credit Card Promotion Database A combination of one or more of the dataset attributes differentiate Acme Credit Card Company card holders who have taken advantage of the life insurance promotion and those card holders who have chosen not to participate in the promotional offer.

Figure 2. 3 An unsupervised cluster of the credit card database

Figure 2. 3 An unsupervised cluster of the credit card database

Choosing a Data Mining Technique • Know which kind knowledge you want to get

Choosing a Data Mining Technique • Know which kind knowledge you want to get • Know your data --What is the interaction between input and output attributes? --What is the Distribution of the Data? --Which Attributes Best Define the Data? • Know the difference among different data mining techniques

Questions to Determine Data Mining Applicability 1. 2. 3. 4. Can the problem be

Questions to Determine Data Mining Applicability 1. 2. 3. 4. Can the problem be clearly defined? Does potentially meaningful data exist? Does data contain hidden knowledge or is it just filled with facts? Is the “juice worth the squeeze? ”

Data Mining vs. OLAP • Discovery-based • • (deductive process) Mine data warehouse and

Data Mining vs. OLAP • Discovery-based • • (deductive process) Mine data warehouse and others Can provide information you didn’t expect • Verification-based • • (inductive process) DSS tool for data warehouse Pre-defined queries

Data Mining vs. Data Query • For hidden knowledge • Try to get the

Data Mining vs. Data Query • For hidden knowledge • Try to get the answer as • • accurate as possible Results are the analysis of the data Data need to be prepare before producing results • For specific question • Answer to query is • • 100% accurate if data correct Results are subset of data Need not prepare data