KDD 99 Knowledge Discovery Contest Advisor Dr Hsu

KDD’ 99 Knowledge Discovery Contest Advisor: Dr. Hsu Graduate: Yu-Wei Su 10/31/2020 Intelligent Database System Lab, IDSL

Outline l l l l Motivation Objective Contest target KDD’ 99 Competition: Knowledge Discovery Contest Knowledge Discovery in a Charitable Organization’s Donor Database Profiling Your Customers Using Bayesian Networks Opinion 10/31/2020 Intelligent Database System Lab, IDSL

Motivation l l Direct mail to all customers are inefficiency and high costs Utilizing unsupervised clustering method instead of supervised classification method in the 1998 competition 10/31/2020 Intelligent Database System Lab, IDSL

Objective l l Discovering higher-level knowledge from data Maximizing the profit for predictive model 10/31/2020 Intelligent Database System Lab, IDSL

Contest target l l l The database is the same used in the 1998 competition Data is implemented by an American Charity in the June ’ 97 renewal campaign Techniques included l l 10/31/2020 Unsupervised clustering Knowledge-driven segmentation Association rule discovery Causal modeling Intelligent Database System Lab, IDSL

Contest target( cont’) l Each team had to build a profit-maximizing predictive model for the ’ 98 competition task 10/31/2020 Intelligent Database System Lab, IDSL

KDD’ 99 Competition: Knowledge Discovery Contest l l l Introduction Exploratory data analysis A two-stage prediction model Understanding the model Conclusion 10/31/2020 Intelligent Database System Lab, IDSL

Introduction l l The paper discusses SAS Institute’s findings Expand on the ’ 98 KDD cup competition l l 10/31/2020 To reveal unusual data anomalies A two-stage prediction model yields superior results to those in ’ 98 Use decision tree to better understanding Apply a confidence interval to judge model performance Intelligent Database System Lab, IDSL

Introduction( cont’) l l The models were built using a 95412 case training data set with known response To judge model efficacy, expected gift amount were calculated for a validation data set with concealed response 10/31/2020 Intelligent Database System Lab, IDSL

Exploratory data analysis l Successful statistical prediction models’ elements l l l 10/31/2020 Problem-specific knowledge Historical data Analytical savvy Intelligent Database System Lab, IDSL

Exploratory data analysis l Four anomalies are immediately apparent 10/31/2020 Intelligent Database System Lab, IDSL

Exploratory data analysis( cont’) 10/31/2020 Intelligent Database System Lab, IDSL

Exploratory data analysis( cont’) 10/31/2020 Intelligent Database System Lab, IDSL

A two-stage prediction model l l To accurately estimate the probability distribution of gift amount to a potential donor Prediction is done in two way l l 10/31/2020 Directly estimating expected gift Separately estimating expected donation probability and the expected gift amount and multiplying them Intelligent Database System Lab, IDSL

A two-stage prediction model( cont’) l l Task was done by using two multi-layer perceptron(MLP) neural networks Gift amount model l l 10/31/2020 First, using the cases where gift occurred(5% of the data) Input reflecting historical patterns in the gift amount( fitting class-probability decision tree) Intelligent Database System Lab, IDSL

A two-stage prediction model( cont’) l Gift probability model l l 10/31/2020 Using all the cases Input reflecting recency, frequency, amount(RFA) and demographic data and the patterns noted in the exploratory data analysis Intelligent Database System Lab, IDSL

A two-stage prediction model( cont’) l The first stage MLP l l 10/31/2020 Input layer with five inputs fully connected to 20 hidden units and them fully connected to a target unit 4843 cases with TAGET_B=1 Intelligent Database System Lab, IDSL

A two-stage prediction model( cont’) 10/31/2020 Intelligent Database System Lab, IDSL

A two-stage prediction model( cont’) l The second stage MLP l l Input layer with eight inputs fully connected to 20 hidden units and them fully connected to a target unit Expected gift amount for each case was calculate as the product from the first- and second-stage models 10/31/2020 Intelligent Database System Lab, IDSL

A two-stage prediction model( cont’) l l Net revenue of the model using validation data is $14877. 77 This is $165. 53 more than the Gold-medal winner at KDD-98 10/31/2020 Intelligent Database System Lab, IDSL

Understanding the model 10/31/2020 Intelligent Database System Lab, IDSL

Brief summary l l Smaller mailing size will lead to smaller variability in expected total gift because the variance sum will have fewer terms Further development incorporates both profit maximization and risk minimization as determinants of optimum mailing depth 10/31/2020 Intelligent Database System Lab, IDSL

Knowledge Discovery in a Charitable Organization’s Donor Database l l Introduction Main results Detailed results comments 10/31/2020 Intelligent Database System Lab, IDSL

Introduction l l Data set contains about 95000 customers, with an average net donation of slightly over 11 cents per customers and a total net donation of around $10500 from the “mail to all” policy The task utilizing standard 2 -class knowledge discovery and with Value Weighted Analysis(VWA) 10/31/2020 Intelligent Database System Lab, IDSL

Main results l l Maximal net profit of $15515 when checked against the evaluation data set, compared to KDD-Cup 98’ best result of $14712 net profit Built a “white-box” model comprised of 11 customer segments and bring a combined net donation of $13397 for the evaluation data set 10/31/2020 Intelligent Database System Lab, IDSL

Main results( cont’) l Donation segments with highly profitable and actionable l l Approximately 14000 people who live in an area where over 5% of renters pay over $400 per month Have donated over $100 in the past Have an average donation of over $12 Account for $8200 net donation in the training set 10/31/2020 Intelligent Database System Lab, IDSL

Main results( cont’) 10/31/2020 Intelligent Database System Lab, IDSL

Main results( cont’) l l Identifying donors is a different task than maximizing donation The variability of profit results l l 10/31/2020 Profit difference less than $500 cannot be considered significant Difference of $2000 in profit is not significant on different data sets Intelligent Database System Lab, IDSL

Main results( cont’) l Main discovery & modeling approach was a 1 -stage 2 -class model based on VWA 10/31/2020 Intelligent Database System Lab, IDSL

Detailed results l l l The most significant variables for predicting a customer’s donation behavior are the previous donation behavior summaries The NK phenomenon US-census data turns out to be quite strongly connected to the donation performance of the population 10/31/2020 Intelligent Database System Lab, IDSL

Detailed results( cont’) l l 5 models was chosen at last Two “white box” models and one relatively simple model, based on 40 variables and two candidates for “best overall” model 10/31/2020 Intelligent Database System Lab, IDSL

Detailed results( cont’) l Building the white-box model l 10/31/2020 Total net donation of 11 segments if $13397 for 55086 customers Intelligent Database System Lab, IDSL

Detailed results( cont’) 10/31/2020 Intelligent Database System Lab, IDSL

Detailed results( cont’) l Best single model l Selected 31 original variables, plus 9 additional demographic summary variables Optimal point 10/31/2020 Predict point Intelligent Database System Lab, IDSL

Detailed results( cont’) 10/31/2020 Intelligent Database System Lab, IDSL

Detailed results( cont’) l Improving prediction by averaging 10/31/2020 Intelligent Database System Lab, IDSL

Brief summary l l A good leave-out test set performance can hardly be considered a reliable indication of good future performance The net profit has a very large variance 10/31/2020 Intelligent Database System Lab, IDSL

Profiling Your Customers Using Bayesian Networks l l Introduction Data manipulation and preprocessing Result Conclusion 10/31/2020 Intelligent Database System Lab, IDSL

Introduction l l Build two causal models to understand the characteristics of respondents to direct mail fund raising campaigns The first model( response-net) captures the dependency of the probability of response to the mailing campaign on the independent variables l 10/31/2020 96376 lapsed donors data set Intelligent Database System Lab, IDSL

Introduction( cont’) l The second network( donation-net) models the dependency of the dollar amount of the gift l 10/31/2020 5%respondents to the ’ 97 mailing campaign Intelligent Database System Lab, IDSL

Data manipulation and preprocessing l l l Remove redundant variables and more than 99% of missing values and variables with only one state All continuous variables were discretized into four bins of equal length 30 variables at last were available 10/31/2020 Intelligent Database System Lab, IDSL

Data manipulation and preprocessing( cont’) l These variables can be divided into three group l l l 10/31/2020 Variables about personal information about the donors Variable about information of donors neighborhood as socio-economic and urbanicity indicators Variables about history and promotion history file of the donors Intelligent Database System Lab, IDSL

Result-profiling respondents 10/31/2020 Intelligent Database System Lab, IDSL

Result-profiling donors 10/31/2020 Intelligent Database System Lab, IDSL

Result l l Profit prediction Both two model can be used to predict the expected profit 10/31/2020 Intelligent Database System Lab, IDSL

Brief summary l l Shown an application of Bayesian methods to a Knowledge Discovery task To maintain a high response rate to direct fund raising is to continuously update the database of donors 10/31/2020 Intelligent Database System Lab, IDSL

Opinion l l The aspect of this papers are too higher concepts Simple methodology can make great achievement 10/31/2020 Intelligent Database System Lab, IDSL