n n Person Basket A Chips Salsa Cookies

市场购物篮分析 n n 分析事务数据库表 Person Basket A Chips, Salsa, Cookies, Crackers, Coke, Beer B Lettuce, Spinach, Oranges, Celery, Apples, Grapes C Chips, Salsa, Frozen Pizza, Frozen Cake D Lettuce, Spinach, Milk, Butter 我们是否可假定? n Chips => Salsa Lettuce => Spinach 2020/12/3 AA 12 关联规则 史忠植 4

市场购物篮分析 事务 ID 购物篮 A Chips, Salsa, Cookies, Crackers, Coke, Beer B Lettuce, Spinach, Oranges, Celery, Apples, Grapes C Chips, Salsa, Frozen Pizza, Frozen Cake D Lettuce, Spinach, Milk, Butter, Chips I是什么? 事务ID B的T是什么? s(Chips=>Salsa) 是什么? c(Chips=>Salsa)是什么? 2020/12/3 AA 12 关联规则 史忠植 11

生成频繁项集 n Naïve algorithm n <- |D| for each subset s of I do l <- 0 for each transaction T in D do if s is a subset of T then l <- l + 1 if minimum support <= l/n then add s to frequent subsets 2020/12/3 AA 12 关联规则 史忠植 16

The Apriori 算法—一个示例 Database TDB Tid Items 10 A, C, D 20 B, C, E 30 A, B, C, E 40 B, E Itemset {A, C} {B, E} {C, E} sup {A} 2 {B} 3 {C} 3 {D} 1 {E} 3 C 1 1 st scan C 2 L 2 Itemset sup 2 2 3 2 Itemset {A, B} {A, C} {A, E} {B, C} {B, E} {C, E} sup 1 2 3 2 L 1 Itemset sup {A} 2 {B} 3 {C} 3 {E} 3 C 2 2 nd scan Itemset {A, B} {A, C} {A, E} {B, C} {B, E} {C, E} C 3 2020/12/3 Itemset {B, C, E} 3 rd scan L 3 Itemset {B, C, E} AA 12 关联规则 史忠植 sup 2 22

Apriori 算法 Algorithm: Apriori 输入: Database, D, of transactions; minimum support threshold, min_sup. 输出: L, freuqent itemsets in D. 过程: Ck: Candidate itemset of size k Lk : frequent itemset of size k L 1 = find_frequent_1_itemsets(D); for (k = 2; Lk+1 != ; k++) do begin{ Ck = apriori_gen(Lk-1 , min_sup); for each transaction t in database D do{//scan D for counts Ct =subset(Ck , t); // get the subsets of t that are candidates For each candidate c Ct c. count++; } Lk = candidates in Ck with min_support }end return L= k Lk; 2020/12/3 AA 12 关联规则 史忠植 23

Apriori 算法 Procedure apriori_gen(Lk-1: frequent (k-1)-itemsets; min_sup: minimum support threshold ) for each itemset l 1 Lk-1 for each itemset l 2 Lk-1 if(l 1[1]=l 2[1]) (l 1[2]=l 2[2]) … (l 1[k-1]=l 2[k-1]) Then{ c=join(l 1, l 2)//join step: generate candidates if has_infrequent_subset(c, Lk-1 ) then delete c; //prune step: remove unfruitful candidate else add c to Ck } return Ck 2020/12/3 AA 12 关联规则 史忠植 24

Apriori 算法 Join is generate candidates set of itemsets Ck from 2 itemsets in Lk-1 Procedure join(p, q) insert into Ck select p. item 1, p. item 2, . . . , p. itemk-1, q. itemk-1 from Lk-1 p, Lk-1 q where p. item 1=q. item 1, . . . , p. itemk-2=q. itemk-2, p. itemk-1 = q. itemk-1 2020/12/3 AA 12 关联规则 史忠植 25

Apriori 算法 Procedure has_infrequent_subset(c: candidate k-itemset; Lk-1: frequent (k-1)itemsets; )//use prior knowledge for each (k-1)-subset s of c if s Lk-1 then return TRUE; return FALSE. 2020/12/3 AA 12 关联规则 史忠植 26

如何生成候选项集? n 假定Lk-1中的项以一定顺序排列 n 步骤 1: 自连接 Lk-1 insert into Ck select p. item 1, p. item 2, …, p. itemk-1, q. itemk-1 from Lk-1 p, Lk-1 q where p. item 1=q. item 1, …, p. itemk-2=q. itemk-2, p. itemk-1 < q. itemk-1 n 步骤 2: 剪枝 forall itemsets c in Ck do forall (k-1)-subsets s of c do if (s is not in Lk-1) then delete c from Ck 2020/12/3 AA 12 关联规则 史忠植 28

频繁 1 -项集 事务数据库 支持度计数 频繁1 -项集 TID Items 1 I 1, I 2, I 5 2 I 2, I 4 {I 1} 6 3 I 2, I 3, I 6 {I 2} 7 4 I 1, I 2, I 4 {I 3} 6 5 I 1, I 3 {I 4} 2 6 I 2, I 3 7 I 1, I 3 {I 5} 2 8 I 1, I 2, I 3, I 5 {I 6} 1 9 I 1, I 2, I 3 Itemset n 2020/12/3 Support count Itemset Support count {I 1} 6 {I 2} 7 {I 3} 6 {I 4} 2 {I 5} 2 最小支持度为 20% (计数为 2) AA 12 关联规则 史忠植 36

FP-树 构建 按支持度降序排列 2020/12/3 Itemset Support count {I 1} 6 {I 2} 7 {I 1} 6 {I 3} 6 {I 4} 2 {I 5} 2 AA 12 关联规则 史忠植 37

FP-树 构建 I 2 2 I 1 0 I 3 0 I 4 1 I 5 0 null (I 2, 2) (I 1, 1) (I 4, 1) (I 5, 1) 2020/12/3 AA 12 关联规则 史忠植 TID Items 1 I 1, I 2, I 5 2 I 2, I 4 3 I 2, I 3, I 6 4 I 1, I 2, I 4 5 I 1, I 3 6 I 2, I 3 7 I 1, I 3 8 I 1, I 2, I 3, I 5 9 I 1, I 2, I 3 39

FP-树 构建 I 2 7 I 1 6 I 3 6 I 4 2 I 5 2 null (I 1, 2) (I 2, 7) (I 1, 4) (I 3, 2) (I 4, 1) (I 3, 2) (I 5, 1) (I 3, 2) (I 4, 1) TID Items 1 I 1, I 2, I 5 2 I 2, I 4 3 I 2, I 3, I 6 4 I 1, I 2, I 4 5 I 1, I 3 6 I 2, I 3 7 I 1, I 3 8 I 1, I 2, I 3, I 5 9 I 1, I 2, I 3 (I 5, 1) 2020/12/3 AA 12 关联规则 史忠植 40

FP-树 构建 Insert_Tree([p|P], T) If T has a child N such that N. item-name= p. item-name, then increment N’s count by 1; else create a new node N, and let its count be 1, its parent link be linked to T, and its node link to the nodes with the same item-name via the node-link structure. If P is nonempty, call insert_tree(P, N) recursively. n 2020/12/3 AA 12 关联规则 史忠植 42

挖掘 FP-Tree I 2 7 I 1 6 I 3 6 I 4 2 I 5 前缀路径 (I 2 I 1, 1) (I 2 I 1 I 3, 1) 条件路径 (I 2 I 1, 2) 条件 FP-tree null (I 1, 2) (I 2, 7) null (I 4, 1) (I 1, 4) 2 (I 3, 2) (I 5, 1) (I 3, 2) (I 2, 2) (I 4, 1) (I 1, 2) (I 2 I 1 I 5, 2), (I 2 I 5, 2), (I 1 I 5, 2) (I 5, 1) 2020/12/3 AA 12 关联规则 史忠植 44

挖掘 FP-Tree 条件FP-tree 项 条件模式基 I 5 {(I 2 I 1: 1), (I 2 I 1 I 3: 1)} <I 2: 2, I 1: 2> I 2 I 5: 2, I 1 I 5: 2, I 2 I 1 I 5: 2 I 4 {(I 2 I 1: 1), (I 2: 1)} I 3 {(I 2 I 1: 2, (I 2: 2), (I 1: 2)} <I 2: 4, I 1: 2>, I 2 I 3: 4, <I 1: 2> I 1 I 3: 2, I 2 I 1 I 3: 2 {(I 2: 4)} <I 2: 4> I 2 I 1: 4 I 1 2020/12/3 <I 2: 2> AA 12 关联规则 史忠植 生成的频繁模式 I 2 I 4: 2 45

挖掘 FP-Tree Procedure FP_growth(Tree, ) If Tree contains a single path P then (2) for each combination (denote as ) of the nodes in the path P (3) generate pattern with support = minisup of nodes in ; (4) Else for each ai in the header of Tree { (5) generate pattern =ai with support =ai. support; (6) construct , s conditional pattern base and then ’conditional FP_tree Tree ; (7) IF Tree ø then (8) call FP_growth(Tree , ); } (1) 2020/12/3 AA 12 关联规则 史忠植 46

由事务数据库构建FP-树 TID 100 200 300 400 500 Items bought (ordered) frequent items {f, a, c, d, g, i, m, p} {f, c, a, m, p} {a, b, c, f, l, m, o} {f, c, a, b, m} {b, f, h, j, o, w} {f, b} {b, c, k, s, p} {c, b, p} {a, f, c, e, l, p, m, n} {f, c, a, m, p} 1. 扫描DB一次, 找到频繁1 项 (单一项模式) 2. 按支持度降序对频繁项 排序为 F-list 3. 再次扫描DB, 构建FPtree {} Header Table Item frequency head f 4 c 4 a 3 b 3 m 3 p 3 F-list=f-c-a-b-m-p 2020/12/3 min_support = 3 AA 12 关联规则 史忠植 f: 4 c: 3 c: 1 b: 1 a: 3 b: 1 p: 1 m: 2 b: 1 p: 2 m: 1 47

通过 DB 投影(Projection)使FP-growth可伸缩 n n FP-tree 不能全放入内存? —DB 投影 首先将一个数据库划分成一个由若干投影(Projected) 数据库组成的集合 然后对每个投影数据库构建和挖掘 FP-tree Parallel projection vs. Partition projection 技术 n Parallel projection is space costly 2020/12/3 AA 12 关联规则 史忠植 52

Partition-based Projection n n Parallel projection 需要很多磁 盘空间 Partition projection 节省磁盘 空间 p-proj DB fcam cb fcamp fcabm fb cbp fcamp m-proj DB b-proj DB fcab fca am-proj DB fc fc fc 2020/12/3 Tran. DB f cb … a-proj DB fc … cm-proj DB f f f AA 12 关联规则 史忠植 c-proj DB f … f-proj DB … … 53

FP-Growth vs. Apriori: 支持度的可伸缩性 Data set T 25 I 20 D 10 K 2020/12/3 AA 12 关联规则 史忠植 56

FP-Growth vs. Tree-Projection: 支持度的可伸缩性 Data set T 25 I 20 D 100 K 2020/12/3 AA 12 关联规则 史忠植 57

挖掘多种规则或规律 n 多层(Multi-level), 量化(quantitative)关联规则, 相关 (correlation)和因果(causality), 比率(ratio)规则, 序 列 (sequential) 模式, 浮现(emerging)模式, temporal associations, 局部周期(partial periodicity) n 分类(classification), 聚类(clustering), 冰山立方体( iceberg cubes), 等等. 2020/12/3 AA 12 关联规则 史忠植 61

量化关联规则 n 装箱(Binning): 划分 过程被认为是装箱, 即称作间隔为箱柜 (bins). 三种常用装箱策略是: Equiwidth binning, where the intervalsize of each bin is the same Equidepth: where each bin has approximately the same number of tuples assigned to it Homogeneity-based binning, where bin size is determined so that the tuples in each bin are uniformly distributed. 2020/12/3 AA 12 关联规则 史忠植 71

量化关联规则 n 找出频繁谓词集: Once the 2 -D array containing the count distribution for each category is set up, this can be scanned in order to find the frequent predicate sets that also satisfy minimum confidence. n 聚类关联规则: age(X, ” 34 -35”) income(X, ” 31 K 50 K”) buys(X, ”high resolution TV”) 2020/12/3 AA 12 关联规则 史忠植 72

ARCS (Association Rules Clustering System) ARCS 1. Binning 2. Frequent Items Set 3. Clustering 4. Optimization 2020/12/3 AA 12 关联规则 史忠植 73

规则生成: 基本概念 n 规则项(Ruleitem) <condset, y> : 条件集 condset 是一个项集, y 是一个类标签(class label) 每个规则项表示一个规则: condset->y n 条件集支持度(condsup. Count) n n 规则支持度(rulesup. Count) n n n 2020/12/3 D中包含condset的事例(case)数 D中包含condset 及标注类别 y的事例(case)数 支持度Support=(rulesup. Count/|D|)*100% 可信度Confidence=(rulesup. Count/condsup. Count)*100% AA 12 关联规则 史忠植 79

规则生成: 一个示例 n 一个规则项: <{(A, 1), (B, 1)}, (class, 1)> n 假定 n n 则 (A, 1), (B, 1) -> (class, 1) n n 2020/12/3 条件集 condset (condsup. Count) 的支持度计数为 3, 规则项ruleitem (rulesup. Count) 的支持度计数为 2, |D|=10 [supt=20% (rulesup. Count/|D|)*100% confd=66. 7% (rulesup. Count/condsup. Count)*100%] AA 12 关联规则 史忠植 81

RG: 算法 1 F 1 = {large 1 -ruleitems}; 2 CAR 1 = gen. Rules (F 1 ); 3 pr. CAR 1 = prune. Rules (CAR 1 ); //count the item and class occurrences to determine the frequent 1 -ruleitems and prune it 4 for (k = 2; F k-1 Ø; k++) do 5 C k = candidate. Gen (F k-1 ); //generate the candidate ruleitems Ck using the frequent ruleitems Fk-1 6 for each data case d D do //scan the database 7 C d = rule. Subset (C k , d); //find all the ruleitems in Ck whose condsets are supported by d 8 for each candidate c C d do 9 c. condsup. Count++; 10 if d. class = c. class then c. rulesup. Count++; //update various support counts of the candidates in C k 11 end 12 end 2020/12/3 AA 12 关联规则 史忠植 82

RG: 算法(cont. ) 13 F k = {c C k | c. rulesup. Count minsup}; //select those new frequent ruleitems to form Fk 14 CAR k = gen. Rules(F k ); //select the ruleitems both accurate and frequent 15 pr. CAR k = prune. Rules(CAR k ); 16 end 17 CARs = k CAR k ; 18 pr. CARs = k pr. CAR k ; 2020/12/3 AA 12 关联规则 史忠植 83

M 1: Algorithm n n n n 1 R = sort(R); //Step 1: sort R according to the relation “ ” 2 for each rule r R in sequence do 3 temp = Ø; 4 for each case d D do //go through D to find those cases covered by each rule r 5 if d satisfies the conditions of r then 6 store d. id in temp and mark r if it correctly classifies d; 7 if r is marked then 8 insert r at the end of C; //r will be a potential rule because it can correctly classify at least one case d 9 delete all the cases with the ids in temp from D; 10 selecting a default class for the current C; //the majority class in the remaining data 11 compute the total number of errors of C; 12 end 13 end // Step 2 14 Find the first rule p in C with the lowest total number of errors and drop all the rules after p in C; 15 Add the default class associated with p to end of C, and return C (our classifier). //Step 3 2020/12/3 AA 12 关联规则 史忠植 86

M 1: 满足的两个条件 n n Each training case is covered by the rule with the highest precedence among the rules that can cover the case. Every rule in C correctly classifies at least one remaining training case when it is chosen. 2020/12/3 AA 12 关联规则 史忠植 87

MOUCLAS n Assumption: MOUCLAS algorithm assumes that the initial association rules can be agglomerated into clustering regions n The implication of the form of the Mouclas Pattern (so called MP) : Cluster(D)t y where Cluster(D)t is a cluster of D, t = 1 to m, and y is a class label. 2020/12/3 AA 12 关联规则 史忠植 88

MOUCLAS n Definitions: Frequency of Mouclas Patterns Accuracy of Mouclas Patterns Reliability of Mouclas Patterns n Task of Mouclas: 2020/12/3 To discover MPs that have support and confidence greater than the user-specified minimum support threshold (called minsup), and minimum confidence threshold (called minconf) and minimum reliability threshold (called min. R) respectively, and to construct a classifier based upon MPs. AA 12 关联规则 史忠植 89

The MOUCLAS algorithm n Two steps: Step 1. Discovery of the frequent and accurate and reliable MPs. Step 2. Construction of a classifier, called De-MP, based on MPs. 2020/12/3 AA 12 关联规则 史忠植 90

The MOUCLAS algorithm The core of the first step in the Mouclas algorithm is to find all cluster_rules that satisfy minsup and minconf and min. R. Let C denote the dataset D after dimensionality reduction processing. A cluster_rule represents a MP, namely a rule: cluset y, where cluset is a set of itemsets from a cluster Cluster(C)t, y is a class label, y Y. 2020/12/3 AA 12 关联规则 史忠植 91

The MOUCLAS algorithm Algorithm of the first step: Mouclas Mining frequent and accurate and reliable Mouclas patterns (MPs) n n Input: A training transaction database, D; minimum support threshold (minsup); minimum confidence threshold (minconf) ; minimum reliability threshold (min. R) Output: A set of frequent and accurate and reliable Mouclas patterns (MPs) 2020/12/3 AA 12 关联规则 史忠植 92

The MOUCLAS algorithm n Methods: (1) Reduce the dimensionality of transactions d, which efficiently reduces the data size by removing irrelevant or redundant attributes (or dimensions) from the training data, and (2) Identify the clusters of database C for all transactions d after dimensionality reduction on attributes Aj in database C, based on the Mountain function, which is a fuzzy set membership function, and specially capable of transforming quantitative values of attributes in transactions into linguistic terms, and (3) Generate a set of MPs that are both frequent and accurate, namely, which satisfy the user-specified minimum support (called minsup) and minimum confidence (called minconf) and minimum reliability (called min. R) constraints. 2020/12/3 AA 12 关联规则 史忠植 93

The MOUCLAS algorithm 1 X = reduce. Dim (I); // reduce the dimensionality on the set of all items I of in D 2 Cluster(C)t = gen. Cluster (C); // identify the complete clusters of C 3 for each Cluster(C)t do E = gen. Clusterrules(cluset, class); // generate a set of candidate cluster_rules 4 for each transaction d C do 5 Ed = gen. Sub. Clusterrules (E, d); // find all the cluster_rules in E whose cluset are supported by d 6 for each e Ed do 7 e. clusup. Count++; // accumulate the clusup. Count of the cluset of cluster_rule e 8 if d. class = e. class then e. cisup. Count++ // accumulate the cisup. Count of cluster_rule e supported by d 9 end 10 end 11 F = {e E e. cisup. Count minsupi }; // construct the set of frequent cluster_rules 12 MP = gen. Rules (F); //generate MP using the gen. Rules function by minconf and min. R 13 end 14 MPs = ∪ MP; // discover the final set of MPs 2020/12/3 AA 12 关联规则 史忠植 94

The MOUCLAS algorithm The task of the second step in Mouclas algorithm: n n Using a heuristic method to generate a classifier, named De. MP, where the discovered MPs can cover D and are organized according to a decreasing precedence based on their confidence and support. Suppose R be the set of frequent and accurate and reliable MPs which are generated in the past step, and MPdefault_class denotes the default class, which has the lowest precedence. We can then present the De-MP classifier in the form of <MP 1, MP 2, …, MPn, MPdefault_class>, where MPi R, i = 1 to n, MPa ≻MPb if n b > a 1 and a, b i, C ∪ cluset of MPi 2020/12/3 AA 12 关联规则 史忠植 95

The MOUCLAS algorithm Algorithm: Mouclas constructing De-MP Classifier n Input: A training database after dimensionality reduction, C; The set of frequent and accurate and reliable Mouclas patterns (MPs) n Output: De-MP Classifier n Methods: (1) Identify the order of all discovered MPs based on the definition of precedence and sequence them according to decreasing precedence order. (2) Determine possible MPs for De-MP classifier from R following the descending sequence of MPs. (3) Discard the MPs which cannot contribute to the improvement of the accuracy of the De-MP classifier and keep the final set of MPs to construct the De-MP classifier. 2020/12/3 AA 12 关联规则 史忠植 96

The MOUCLAS algorithm 1 R = sort(R); // sort MPs based on their precedence 2 for each MP R in sequence do 3 temp = ; 4 for each transaction d C do 5 if d satisfies the cluset of MP then 6 store d. ID in temp; 7 if MP correctly classifies d then 8 insert MP at the end of L; 9 delete the transaction who has ID in temp from C; 10 selecting a default class for the current L; // determine the default class based on majority class of remaining transactions in C 11 end 12 compute the total number of errors of L; // compute the total number of errors that are made by the current L and the default class 13 end 14 Find the first MP in L with the lowest total number of errors and discard all the MPs after the MP in L; 15 Add the default class associated with the above mentioned first MP to end of L; 16 De-MP classifier = L 2020/12/3 AA 12 关联规则 史忠植 97

Example of MOUCLAS application The well logging data sets include attributes (well logging curves) of GR (gamma ray), RDEV (deep resistivity), RMEV (shallow resistivity), RXO (flushed zone resistivity), RHOB (bulk density), NPHI (neutron porosity), PEF (photoelectric factor) and DT (sonic travel time). A hypothetically useful MP may suggest a relation between well logging data and the class label of oil/gas formation since. 2020/12/3 AA 12 关联规则 史忠植 98

Mining Distance-based Association Rules n n Binning methods do not capture the semantics of interval data Distance-based partitioning, more meaningful discretization considering: n density/number of points in an interval n “closeness” of points in an interval 2020/12/3 AA 12 关联规则 史忠植 99

Interestingness Measure: Correlations (Lift) n play basketball eat cereal [40%, 66. 7%] is misleading n The overall percentage of students eating cereal is 75% which is higher than 66. 7%. n play basketball not eat cereal [20%, 33. 3%] is more accurate, although with lower support and confidence n 2020/12/3 Measure of dependent/correlated events: lift Basketball Not basketball Sum (row) Cereal 2000 1750 3750 Not cereal 1000 250 1250 Sum(col. ) 3000 2000 5000 AA 12 关联规则 史忠植 100

相关规则(Correlation Rules) n n “Beyond Market Baskets, ” Brin et al. 假设执行关联规则挖掘 tea => coffee 20% support 80% confidence 2020/12/3 c c row t 20 5 25 t 70 5 75 col 90 10 100 but 90% of the people buy coffee anyway! AA 12 关联规则 史忠植 102

相关规则 n n 2 统计量定义为 Look up for significance value in a statistical textbook n There are k-1 degrees of freedom n If test fails cannot reject independence, otherwise contigency table represents dependence. 2020/12/3 AA 12 关联规则 史忠植 105

示例 n n n n Back to tea and coffee c c row t 20 5 25 t 70 5 75 90 10 100 E[t] = 25, E[t]=75, E[c]=90, E[c]=10 col E[tc]=100 * 25/100 * 90 /100=22. 5 O(tc) = 20 Contrib. to 2 = (20 - 22. 5)2 / 22. 5 = 0. 278 Calculate for the rest to get 2=2. 204 Not significant at 95% level (3. 84 for k=2) Cannot reject independence assumption 2020/12/3 AA 12 关联规则 史忠植 106

兴趣度（Interest） n n If 2 test shows significance, then want to find most interesting cell(s) in table I(r) = O(r)/E[r] n Look for values far away from 1 n I(tc) = 20/22. 5 = 0. 89 n I(tc) = 5/2. 5 = 2 n I(tc) = 70/67. 5 = 1. 04 n I(tc) = 5/7. 5 = 0. 66 2020/12/3 AA 12 关联规则 史忠植 107

其它度量(Measures) n TID Items 1 I 1, I 2, I 5 2 I 2, I 4 3 I 2, I 3, I 6 4 I 1, I 2, I 4 5 I 1, I 3 6 I 2, I 3 7 I 1, I 3 8 I 1, I 2, I 3, I 5 9 I 1, I 2, I 3 2020/12/3 作用度(Lift) n 度量项之间的相关性, 但 没有检验 AA 12 关联规则 史忠植 109

其它度量(Measures) n 可信度(Conviction) n 度量一个规则的强度 2020/12/3 AA 12 关联规则 史忠植 TID Items 1 I 1, I 2, I 5 2 I 2, I 4 3 I 2, I 3, I 6 4 I 1, I 2, I 4 5 I 1, I 3 6 I 2, I 3 7 I 1, I 3 8 I 1, I 2, I 3, I 5 9 I 1, I 2, I 3 110

基于约束(Constraints)的数据挖掘 n n n Finding all the patterns in a database autonomously? — unrealistic! n The patterns could be too many but not focused! Data mining should be an interactive process n User directs what to be mined using a data mining query language (or a graphical user interface) Constraint-based mining n User flexibility: provides constraints on what to be mined n System optimization: explores such constraints for efficient mining—constraint-based mining 2020/12/3 AA 12 关联规则 史忠植 112

数据挖掘中的约束(Constraints) n Knowledge type constraint: n n Data constraint — using SQL-like queries n n in relevance to region, price, brand, customer category Rule (or pattern) constraint n n find product pairs sold together in stores in Vancouver in Dec. ’ 00 Dimension/level constraint n n classification, association, etc. small sales (price < $10) triggers big sales (sum > $200) Interestingness constraint n 2020/12/3 strong rules: min_support 3%, min_confidence 60% AA 12 关联规则 史忠植 113

约束(Constrained)挖掘 vs. 基于约束的搜索 n Constrained mining vs. constraint-based search/reasoning n n n Both are aimed at reducing search space Finding all patterns satisfying constraints vs. finding some (or one) answer in constraint-based search in AI Constraint-pushing vs. heuristic search It is an interesting research problem on how to integrate them Constrained mining vs. query processing in DBMS n n 2020/12/3 Database query processing requires to find all Constrained pattern mining shares a similar philosophy as pushing selections deeply in query processing AA 12 关联规则 史忠植 114

约束频繁项集挖掘: 一种挖掘查询优化问 题 n Given a frequent pattern mining query with a set of constraints C, the algorithm should be n n n A naïve solution n n sound: it only finds frequent sets that satisfy the given constraints C complete: all frequent sets satisfying the given constraints C are found First find all frequent sets, and then test them for constraint satisfaction More efficient approaches: n n 2020/12/3 Analyze the properties of constraints comprehensively Push them as deeply as possible inside the frequent pattern computation. AA 12 关联规则 史忠植 115

基于约束挖掘的非单调性 TDB (min_sup=2) n Anti-monotonicity n n When an itemset S violates the constraint, so does any of its superset sum(S. Price) v is anti-monotone sum(S. Price) v is not anti-monotone Example. C: range(S. profit) 15 is antimonotone TID Transaction 10 a, b, c, d, f 20 b, c, d, f, g, h 30 a, c, d, e, f 40 c, e, f, g Item Profit a 40 b 0 c -20 n Itemset ab violates C d 10 n So does every superset of ab e -30 f 30 g 20 h -10 2020/12/3 AA 12 关联规则 史忠植 116

什么约束是非单调的( Anti-Monotone)? 2020/12/3 Constraint Antimonotone v S No S V no S V yes min(S) v no min(S) v yes max(S) v no count(S) v yes count(S) v no sum(S) v ( a S, a 0 ) yes sum(S) v ( a S, a 0 ) no range(S) v yes range(S) v no avg(S) v, { , , } convertible support(S) yes support(S) no AA 12 关联规则 史忠植 117

什么约束是单调的( Monotone)? 2020/12/3 Constraint Monotone v S yes S V no min(S) v yes min(S) v no max(S) v yes count(S) v no count(S) v yes sum(S) v ( a S, a 0 ) no sum(S) v ( a S, a 0 ) yes range(S) v no range(S) v yes avg(S) v, { , , } convertible support(S) no support(S) yes AA 12 关联规则 史忠植 118

简洁(Succinctness) n Succinctness: n n n Given A 1, the set of items satisfying a succinctness constraint C, then any set S satisfying C is based on A 1 , i. e. , S contains a subset belonging to A 1 Idea: Without looking at the transaction database, whether an itemset S satisfies constraint C can be determined based on the selection of items n min(S. Price) v is succinct n sum(S. Price) v is not succinct Optimization: If C is succinct, C is pre-counting pushable 2020/12/3 AA 12 关联规则 史忠植 119

什么约束是简洁的( Succinct? ） 2020/12/3 Constraint Succinct v S yes S V yes min(S) v yes max(S) v yes count(S) v weakly sum(S) v ( a S, a 0 ) no range(S) v no avg(S) v, { , , } no support(S) no AA 12 关联规则 史忠植 120

Apriori 算法 — 示例 Database D L 1 C 1 Scan D C 2 Scan D L 2 C 3 2020/12/3 Scan D L 3 AA 12 关联规则 史忠植 121

Naïve Algorithm: Apriori + Constraint Database D L 1 C 1 Scan D C 2 Scan D L 2 C 3 Scan D L 3 Constraint: Sum{S. price < 5} 2020/12/3 AA 12 关联规则 史忠植 122

The Constrained Apriori Algorithm: Push an Anti-monotone Constraint Deep Database D L 1 C 1 Scan D C 2 Scan D L 2 C 3 Scan D L 3 Constraint: Sum{S. price < 5} 2020/12/3 AA 12 关联规则 史忠植 123

约束的 Apriori 算法: Push a Succinct Constraint Deep Database D L 1 C 1 Scan D C 2 Scan D L 2 C 3 Scan D L 3 Constraint: min{S. price <= 1 } 2020/12/3 AA 12 关联规则 史忠植 124

转换“强 ”约束 (Tough Constraints) TDB (min_sup=2) n n TID Transaction Convert tough constraints into antimonotone or monotone by properly ordering items 10 a, b, c, d, f 20 b, c, d, f, g, h 30 a, c, d, e, f Examine C: avg(S. profit) 25 40 c, e, f, g n Order items in value-descending order n n If an itemset afb violates C n n 2020/12/3 <a, f, g, d, b, h, c, e> So does afbh, afb* It becomes anti-monotone! AA 12 关联规则 史忠植 Item Profit a 40 b 0 c -20 d 10 e -30 f 30 g 20 h -10 125

可变约束(Convertible Constraints) n Let R be an order of items n Convertible anti-monotone n n n If an itemset S violates a constraint C, so does every itemset having S as a prefix w. r. t. R Ex. avg(S) v w. r. t. item value descending order Convertible monotone n n 2020/12/3 If an itemset S satisfies constraint C, so does every itemset having S as a prefix w. r. t. R Ex. avg(S) v w. r. t. item value descending order AA 12 关联规则 史忠植 126

可变约束的挖掘 n n TDB (min_sup=2) C: avg(S. profit) 25 List of items in every transaction in value descending order R: <a, f, g, d, b, h, c, e> n C is convertible anti-monotone w. r. t. R n Scan transaction DB once n n 2020/12/3 remove infrequent items n Item h in transaction 40 is dropped Itemsets a and f are good AA 12 关联规则 史忠植 TID Transaction 10 a, f, d, b, c 20 f, g, d, b, c 30 a, f, d, c, e 40 f, g, h, c, e Item Profit a 40 f 30 g 20 d 10 b 0 h -10 c -20 e -30 127

强可变约束( Convertible Constraints) n avg(X) 25 is convertible anti-monotone w. r. t. item value descending order R: <a, f, g, d, b, h, c, e> n n avg(X) 25 is convertible monotone w. r. t. item value ascending order R-1: <e, c, h, b, d, g, f, a> n n If an itemset af violates a constraint C, so does every itemset with af as prefix, such as afd If an itemset d satisfies a constraint C, so does itemsets df and dfa, which having d as a prefix Item Profit a 40 b 0 c -20 d 10 e -30 f 30 g 20 h -10 Thus, avg(X) 25 is strongly convertible 2020/12/3 AA 12 关联规则 史忠植 128

什么约束是可改变的( Convertible)? Constraint Convertible antimonotone Convertible monotone Strongly convertible avg(S) , v Yes Yes median(S) , v Yes Yes sum(S) v (items could be of any value, v 0) Yes No No sum(S) v (items could be of any value, v 0) No Yes No sum(S) v (items could be of any value, v 0) Yes No No …… 2020/12/3 AA 12 关联规则 史忠植 129

处理多个约束 n n n Different constraints may require different or even conflicting item-ordering If there exists an order R s. t. both C 1 and C 2 are convertible w. r. t. R, then there is no conflict between the two convertible constraints If there exists conflict on order of items n n 2020/12/3 Try to satisfy one constraint first Then using the order for the other constraint to mine frequent itemsets in the corresponding projected database AA 12 关联规则 史忠植 130

将它们组合一起－一个综合视图 Constraint Antimonotone Monotone Succinct v S no yes yes S V yes no yes min(S) v yes no yes max(S) v no yes count(S) v yes no weakly count(S) v no yes weakly sum(S) v ( a S, a 0 ) yes no no sum(S) v ( a S, a 0 ) no yes no range(S) v yes no no range(S) v no yes no avg(S) v, { , , } convertible no support(S) yes no no support(S) no yes no 2020/12/3 AA 12 关联规则 史忠植 131

约束的分类 Monotone Antimonotone Succinct Strongly convertible Convertible anti-monotone Convertible monotone Inconvertible 2020/12/3 AA 12 关联规则 史忠植 132

References n n n Jiawei Han and Micheline Kamber. Data Mining: Concepts and Techniques. Morgan Kaufmann Publishers R. Agrawal, T. Imielinski, and A. Swami. Mining association rules between sets of items in large databases. SIGMOD'93, 207 -216, Washington, D. C. J. Han, J. Pei, and Y. Yin: “Mining frequent patterns without candidate generation”. In Proc. ACMSIGMOD’ 2000, pp. 1 -12, Dallas, TX, May 2000. 2020/12/3 AA 12 关联规则 史忠植 134

References n n S. Brin, R. Motwani, and C. Silverstein. Beyond market basket: Generalizing association rules to correlations. SIGMOD'97, 265 -276, Tucson, Arizona. J. Han, L. V. S. Lakshmanan, and R. T. Ng, "Constraint-Based, Multidimensional Data Mining", COMPUTER (special issues on Data Mining), 32(8): 46 -50, 1999. n Craig A. Struble. Association Rule Mining. Slides, Marquette University n Yalei Hao. Markus Stumptner. Gerald Quirchmayr. Qing He. Data Mining by MOUCLAS Algorithm for Petroleum Reservoir Characterization from Well Logging Data. AIAI 2004. 2020/12/3 AA 12 关联规则 史忠植 135

www. intsci. ac. cn/shizz/ Questions? ! 2020/12/3 AA 12 关联规则 史忠植 136