Efficiently Mining Long Patterns from Databases by Roberto

Efficiently Mining Long Patterns from Databases by Roberto J. Bayardo Jr. IBM Research Center - Greeshma Neglur - Devangana Tarafdar 10/25/2020 1

Contents. . n n n n Introduction Max-Miner Overview Techniques employed Formalizing Max-Miner Support Lower Bounding (Max-Miner and Apriori) Additional constraints on Max-Miner Evaluation & Experimental Results Summary 10/25/2020 2

Introduction n Finding patterns in databases is the fundamental operation behind most data-mining tasks. Most pattern mining algorithms (Apriori Like) operate efficiently on databases where the longest patterns are relatively short. Examples of databases with long patterns: n Biological databases (DNA, proteins etc) n Databases with questionnaire results 10/25/2020 3

Disadvantages of Apriori Like Algorithms n n n Employ bottom-up search to enumerate every single frequent itemset (smaller sets to bigger sets) To produce a frequent itemset of length ‘ℓ’ all 2ℓ subsets must be produced as they must also be frequent Exponential complexity restricts them to discovering only short patterns 10/25/2020 4

Basic concepts and definitions n n Data-set : set of transactions over a finite item domain Itemset : A set of items from the finite item domain Support of an Itemset I [Sup(I)] : The number of transactions that contain I Frequent Itemset : Is one whose support >= minimum support(min. Sup) as specified by the user 10/25/2020 5

Maximal Frequent itemsets n n Maximal frequent itemset is one that has no superset that is frequent Example: Frequent itemsets Maximal Frequent itemsets {1, 2, 3} {1, 2, 3, 4} {3, 4, 5} n {1, 2, 3} – not maximal as its superset {1, 2, 3, 4} is frequent 10/25/2020 6

Max-Miner overview n n Max-Miner is an improvement over Apriori Like Algorithms How ? n n n efficiently extracts only maximal frequent itemsets Implicitly outputs all frequent itemsets as any frequent itemset is a subset of a maximal frequent itemset Performance n n Results in an improvement of two or more orders of magnitude over Apriori on long pattern datasets Scales linearly in the number of maximal patterns irrespective of length of longest pattern 10/25/2020 7

Set enumeration trees n Illustrate how sets of items are to be completely enumerated in a search problem n Example for Item Domain = {1, 2, 3, 4} {} 1 1, 2, 3 1, 2, 4 1, 2, 3, 4 10/25/2020 (25%) 1, 4 1, 3, 4 2 (50%) 2, 3 3 2, 4 (58%) 4 (67%) 3, 4 2, 3, 4 Gen-Sub Item-order 8

Set enumeration tree. . . n n n Max-Miner expands sets over an ordered and finite Item domain (D) Max-Miner uses heuristic techniques to order the items and dynamically reorder them on a per-node basis The order imposed on the item domain affects the parent/child relationships but not the completeness 10/25/2020 9

Techniques Employed … {} n n Each node in the tree is represented by a Candidate group (g) has two itemsets n n 1, 2 Head : h(g) , is the itemset enumerated by the node Tail : t(g), is the ordered 1, 2, 3 itemset containing all items not in h(g) that can potentially appear in any subnode 1, 2, 3, 4 10/25/2020 2 1 1, 3 1, 2, 4 h(g)={1} t(g)={2, 3, 4} h(g)={1, 2} t(g)={3, 4} h(g)={1, 2, 3} t(g)={4} h(g)={1, 2, 3, 4} t(g)={} 10

Techinques Employed… n n Ordering of the tail items reflects how the sub-nodes are to be expanded Counting the support of g : n n Support of itemsets h(g) U t(g) Support of h(g) U {i} for all i Є t(g) Support of itemsets other than h(g) are used for pruning 10/25/2020 11

Techniques employed n n n Subset pruning (Apriori Like) n If a subset is infrequent then its superset will also be infrequent - do not expand node Abandons strict bottom-up traversal of search space, attempts to look ahead How ? n Superset pruning (looking ahead) n 10/25/2020 If an itemset is frequent, then all its subsets will be frequent but not maximal – do not expand node 12

Techniques Employed…. n Subset pruning is done before expanding sub-nodes, by removing any item i , (i Є t(g)) from g, if h(g) U {i} is infrequent. n If h(g) U {i} is infrequent then any head of a sub node containing ‘i’ is also infrequent. n An example using the set enumeration tree described earlier is shown : {} h(g)={1} t(g)={2, 3, 4} 1 1, 2, 3 1, 2, 4 10/25/2020 1, 2, 3, 4 1, 3, 4 2 3 4 1, 4 {1, 3} is infrequent , hence {3} is removed from t(g) 13

Techniques Employed…. § Superset pruning is done by halting sub-node expansion at any g for which h(g) U t(g) is frequent. Why can this be done ? n h(g) U t(g) is frequent implies that any itemset as specified by sub-node is frequent but not maximal. § An example using the set enumeration tree described earlier is shown : {2, 3, 4} is frequent , hence ‘look ahead’ {} and halt expansion of subnodes h(g)={2} t(g)={3, 4} 1 2 2, 3 3 4 2, 3, 4 10/25/2020 14

Running Example n n Item_Domain = {5, 8, 9, 2, 1, 3, 10, 7, 6, 4} Minsup = 25% Transactions {2, 3, 4} {1, 2, 4} {5, 7, 3} {6, 2, 3, 4} {1, 2, 4} n Data Set of transactions {2, 3, 4} {7, 8, 3} {9, 10, 3} {6, 4} {1, 2, 4} {8, 9, 3} 10/25/2020 {10, 4} 15

Formalizing Max-Miner n The Max-Miner algorithm consists of 3 main sections n The main function Max-Miner : This accepts a data set and a minimum support. n Gen-Initial-Groups : This perform an initial scan over the dataset and identifies the item domain and begins the tree enumeration process. n Gen-Sub-Nodes : This generates the actual sub node. 10/25/2020 16

Formalizing Max-Miner 10/25/2020 Example-1 Example-2 Gen-Sub-Nodes 17

Formalizing Max-Miner 10/25/2020 18

Running Example n GEN-INITIAL-GROUPS(T, C) h(g)={1} t(g)={2, 3, 4} {} I = {5, 8, 9, 2, 1, 3, 10, 7, 6, 4} F 1 = {1 (25%), 2 (after ordering) n n (50%), 3(58%), 4(67%)} C = {( 1 , 2, 3, 4), ( 2 , 3, 4), ( 3 , 4)} 1 2 h(g)={2} t(g)={3, 4} 3 4 h(g)={3} t(g)={4} Return (4) 10/25/2020 19

Running Example. . n Back in Max-Miner (after Gen-Initial-Groups) F = {(4)} C= {( 1 , 2, 3, 4), -> 0% [sup(h(g) U t(g)] ( 2 , 3, 4), -> 25% (as above) ( 3 , 4)} -> 25% (as above) F = {(4), (2 , 3, 4), (3, 4)} Cnew = {} n Gen-Sub-Nodes(( 1 , 2, 3, 4), Cnew) n n 1, 2 sup(h(g) U {i}) sup(1, 2) = 25% sup(1, 3) = 0% Sup(1, 4) = 25% {} 1 1, 3 2 1, 4 3, 4 (after subset-pruning) n n h(g) = {1} , t(g) = {2, 4} g’ = {( 1, 2 , 4)} 2, 3, 4 C = {( 1, 2 , 4)} Return (1, 4) 10/25/2020 Max-Miner 20

Formalise Max-Miner 10/25/2020 Gen-Sub. LB 21

Running Example. . n n n Back in Max-Miner (after Gen-Sub-Nodes) F = {(4), (2, 3, 4), (1, 4)} C = {( , 4)} 1, 2 After removing F = {(2, 3, 4) , (1, 4)} C = {( , 4)} 1, 2 of while loop Next iteration C = {( , 4)} -> 25% (sup (h(g) U t(g)) 1, 2 , (1, 4) , (1, 2, 4)} F = {(2, 3, 4) Final tree generated {} 1 2 3 1, 2 F = {(2, 3, 4) , (1, 2, 4)} C = {} - termination Max-Miner 10/25/2020 Set-Enum 22

Item Ordering Policies n n Motivation – increase the effectiveness of superset-frequency pruning [when h(g) U t(g) is frequent] Superset-pruning possible when many candidate groups have h(g) U t(g) to be frequent Force most frequent items to appear in most candidate groups as they are more likely to be part of long frequent itemsets Reorder items to position the most frequent items last (eg : item (4) in the set enumeration tree Set-enum 10/25/2020 23

Support Lower Bounding n n Number of transactio ns dropped when ‘e ’ is added to Is Advantage : reduces number of candidate groups and facilitates additional super-set pruning Idea is to compute a lower-bound on the support of an itemset by exploiting the available support of its subsets n drop(Is , e) = sup(Is) – sup(Is U {e}) [ ‘e’ is an item not in Is or I and Is I] n n Similarly for I : sup(I U {e}) = sup(I) – drop(I, e) Lower-bound on sup(I U {e}) can be computed from sup(I) and upper-bound on drop(I, e) 10/25/2020 24

Support Lower-Bounding n Theorem: n n sup(I) – drop(Is, e) is a lower-bound on the sup(I U {e}) Proof : n n e e Space of all transactions occupied by I is entirely contained within the space occupied by Is Set of transactions dropped by extending itemset I with ‘e’ is a subset of the set of transactions dropped by extending Is with ‘ e ’ n n drop(Is , e) is an upper-bound on drop(I, e) Extending the concept : T is a itemset disjoint from I sup(I U T) = sup(I) - ∑ drop(Is , e) eЄT 10/25/2020 25

Support Lower-Bounding in Max-Miner 10/25/2020 Gen-Sub 26

Support Lower-Bounding Apriori (Apriori-LB) n n An itemset of length ‘k’ is a candidate iff every k-1 item subset was found frequent during the previous database pass Support of candidate itemsets (Ck) of length ‘k’ is computed during database pass ‘k’ Apriori-LB lower-bounds the support of a candidate itemset above or equal to minsup, using the support of subsets, and hence avoids explicit calculation of support Savings resulting from counting the support of fewer candidate itemsets can be substantial 10/25/2020 27

Max-Miner-LO n n Finds the cardinality ‘ℓ’ of the longest itemset in the set of frequent itemsets ‘F’ after each database pass. Performs the following pruning operations on ‘F’: n n Prune any frequent itemsets in F shorter than ‘ℓ’ Prune any candidate groups g in C such that |h(g) U t(g)| < ‘ℓ’ 10/25/2020 28

Evaluation n n Evaluation was performed on datasets with relatively wide record lengths, categorically-valued attributes and substantial number of records The seven different datasets used were 10/25/2020 29

Comparison with Apriori and Apriori-LB n n Max-Miner outperforms Apriori and Apriori-LB because fewer candidate itemsets are considered Unable to run Apriori and Apriori-LB at lower supports because of memory constraints 10/25/2020 30

Evaluation 10/25/2020 31

Performance Max-Miner-LO n 10/25/2020 Additional constraints can make mining feasible even when the number of maximal frequent itemsets is too large and support is very low 32

Summary n n n Max-Miner applies several techniques for reducing the space of itemsets considered, through superset and subset frequency based pruning. It can be used to achieve tractable completeness at low supports on complex datasets. Some of these techniques can be used to modify and improve existing algorithms like Apriori. 10/25/2020 33

Thank You ! 10/25/2020 34