TopN Recommendation Algorithm Based on ItemGraph Allen Zhenjiang

Top-N Recommendation Algorithm Based on Item-Graph Allen, Zhenjiang LIN CSE, CUHK June 7, 2007 1

Outline n 1. Top-N Recommendation Problem n 2. Top-N Recommendation Algorithm n 3. Item-Graph Model and GCP-based Method q Item-Graph Model q Generalized Conditional Probability(GCP)-based Recommendation Algorithm n 4. Preliminary Experimental Results n 5. Conclusion and Future Work 2

1. Top-N Recommendation Problem n The Top-N Recommendation Problem q User. Item matrix Given the preference information of users, recommend a set of N items to a certain user that he might be interested in, based on the items he has selected. n E-commerce system example: Amazon. COM, customers vs products. Item 1 Item 2 Item 3 … Item m User 1 1 0 User 2 1 1 0 0 User n 0 1 New User 1 ? 1 … Active User ? ? Basket 3

Example: the Amazon. com Active User Basket Recommend ations 4

1. Top-N Recommendation Problem n Challenges in E-commerce Systems q q n Huge amounts of data: millions of users and/or items; Real-time return the results set; Limited new user’s preference information; Volatile users’ preference information. Contributions q Propose the Item-Graph model. n n q simple & incremental to reflect the relationship among items Develop the Generalized Conditional Probability-based top-N recommendation algorithm. n n item-centric based-on the Item-Graph model 5

2. Top-N Recommendation Algorithm n Two main paradigms q Content-based: recommend items based on the content (textual information) of items. n q Fab system [Balabanovic 97], Syskill & Webert system [Pazzani 97]. Collaborative Filtering (CF): recommend items by collecting taste information from other users. n Collaborative between users (link information). n More popular than content-based recommendation, since in many domains (such as music, restaurants) it is hard to extract useful features from items. n Tapestry system [Goldberg 92], Video Recommender [Hill 95], Ringo [Shardanand 95], Group. Lens [Konstan 97], Jester system [Goldberg 01], Amazon [Linden 03]. 6

2. Top-N Recommendation Algorithm n CF algorithms classified by strategy of using data q q Memory-based: make recommendations based on the entire collection of references of the users. n No pre-computing is needed, suffer serious scalability problem. n E. g. , Correlation-based [Resnick 94], Cosine-based [Breese 98]. Model-based: use the collection of user preferences to learn a model, which is then used to make recommendations. n Building a model off-line, more scalable. n E. g. , Cluster models [Ungar 98], Bayesian network model [Breese 98], Association Rule Mining approach [Lin 00]. 7

2. Top-N Recommendation Algorithm n CF algorithms classified by strategy of using objects q User-centric: look for similar (like-minded) users first and then make recommendation. n n q Item-centric: look for similar (or related) items first and then make recommendation. n n Similarity between users is relatively dynamic. Pre-computing user neighborhood may lead to poor predictions. Similarity between items is relatively static. Enables pre-computing of item-item similarity. Therefore, more scalable. The aim of our work q Model-based Item-centric CF top-N recommendation algorithm. 8

2. Top-N Recommendation Algorithm n Notations q q q n Item set I = {I 1, I 2, …, Im}. User set U = {U 1, U 2, …, Un}. User-Item matrix D = (Dn, m). Basket of the active user B I. Similarity score of x and y: sim(x, y). Formal definition of top-N recommendation problem q Given a user-item matrix D and a set of items B that have been purchased by the active user, identify an ordered set of items X such that |X| ≤ N, and X ∩B = 0. 9

2. Top-N Recommendation Algorithm n Two classical item-item similarity measures q Cosine-based (symmetric) sim(Ii, Ij) = cos(D*, i, D*, j) q (1) Conditional Probability(CP)-based (asymmetric) sim(Ii, Ij) = P(Ij | Ii) ≈ Freq(Ii Ij) / Freq(Ii) (2) Freq(X): the number of customers who have purchased the item set X. n The ranking score for item x RS(x) = ∑ b∈B sim(b, x) (3) 10

3. Item-Graph Model & GCP-based Method n Intuitions behind the Item-Graph q q q n The similarity between two items is proportional to the times of co-purchase of them. The similarity of item-pairs is transmissible. E. g. , 1 2 a b c Definition of the Item-Graph q Given a dataset D = (Dn, m), the Item-Graph is defined by a weighted & undirected graph G(V, E, W), where n n n V is the item set I. An edge (x, y)∈E if and only if items x and y have been copurchased. The weight of edge (x, y) is defined by the number of copurchase of items x and y. 11

3. Item-Graph Model & GCP-based Method n Updating the Item-Graph is easy q q Adding new user’s preference information T into the graph needs O(|T|2) operations, including adding edges and/or increasing weight of edges. E. g. , (a, b, c) a 2 b 1 c a 3 b 2 c 1 n Potentially direct application of the Item-Graph q q q Clustering the items. Measuring item-item similarity. Measuring importance of items. 12

3. Item-Graph Model & GCP-based Method n Ideas in Generalized Conditional Probability-based method q According to the definition of top-N recommendation problem, for any x in I-B, we just need to compute the “basket-based” conditional probability P(x|B) = Freq(x. B) / Freq(B). However, n n q q q Freq(x. B) or Freq(B) may not exist, or Freq(x. B) or Freq(B) are too small to make much sense. The CP-based method considers the sum of “ 1 -item”-based conditional probabilities P(x|y) instead, where x∈I-B, y∈B. However, the “multi-item”-based conditional probabilities may also contribute to the recommendation. E. g. , suppose the ranking scores of x and y computed by the CP-based method are equal, and we also know P(x|B)>P(y|B). Which one should be ranked higher, x or y? 13

3. Item-Graph Model & GCP-based Method n The Generalized Conditional Probability (GCP)-based recommendation algorithm q The ranking score of item x is defined by the sum of all possible “multi-item”-based conditional probabilities, that is, GCP(x|B) = ∑ S B P(x|S) ≈ ∑ S B (Freq(x. S) / Freq(S)). (4) q However, the number of subsets of B is 2|B|. q Use GCPd(x|B) instead (set d=2 in the following experiments) GCPd(x|B) = ∑ q S B, |S|≤ d P(x|S). (5) Freq(x. S) and Freq(S) can be extracted from the Item-Graph approximately. 14

3. Item-Graph Model & GCP-based Method n Extracting Freq(A) from Item-Graph approximately q q For an item set A, obtaining the exact Freq(A) may not be possible from the Item-Graph. Extracting approximate Freq(A) from the Item-Graph instead. n Find out the complete sub-graph of A (denoted by CSG(A)) in the Item-Graph, running time O(|A|2). n Freq(A) ≈ minimal weight of edges in CSG(A). n E. g. , a q for A = {a, b}, Freq(A) ≈ 3. q for B = {a, b, c}, Freq(B) ≈ 1. q P(c|ab) ≈ Freq(abc) / Freq(ab) ≈ 1 / 3. 3 b 2 c 1 15

4. Preliminary Experimental Results n Dataset q The Movie. Lens (http: //www. grouplens. org/data) n n A web-based movies recommender system; Contains multi-valued ratings that indicate how much each user liked a particular movie or not; Each user has rated at least 20 movies. We treat the ratings as an indication that the users have seen the movies (nonzero) or not (zero). Table 1: The characteristics of the Movie. Lens dataset # of Users # of Items Density 1 Average Basket Size 943 1 Density: 1682 6. 31% 106. 04 the percentage of nonzero entries in the user-item matrix. 16

4. Preliminary Experimental Results 1 n Evaluation Design q Split the dataset into a training and test set by n n q n randomly selecting one rated movie of each user to be part of the test set, use the remaining rated movies for training. Cosine(COS)-based, CP-based, GCP-based methods, 10 -runs average. Evaluation Metrics q Hit-Rate (HR) q HR = # of hits / n Average Reciprocal Hit-Rate (ARHR) ARHR = (∑i=1, h 1/pi) / n (6) (7) # of hits: the number of items in the test set that were also in the top-N lists. h is the number of hits that occurred at positions p 1, p 2, … , ph within the top-N lists (i. e. , 1 ≤ pi ≤ N). 17

4. Preliminary Experimental Results 1 n Performance of Top-N Recommendation Algorithms HR (left): x-axis: top-N items, y-axis: hit-rate of all users. ARHR (right): x-axis: top-N items, y-axis: average reciprocal hit-rate of all users. (For the GCP-based method, set d = 2. ) 18

4. Preliminary Experimental Results 2 n Testing the Parameter d in GCP Method q n Testing the effect of d ( d = 1, 2, 3 ). Evaluation: Online Shopping Simulation q Randomly selecting part of the user records to be the training set; q Use the remaining user records for training. q STEP 0: Constructing the item-graph based on the training set; q STEP 1: for each user in the training set q n randomly moving one item out of the user’s basket and make recommendation based on the remaining items in the basket; n computing the order of this item in the recommendation list; n updating the item-graph. STEP 2: Computing HR and ARHR metrics. 19

4. Preliminary Experimental Results 2 n Performance of Top-N Recommendation Algorithms HR (left): x-axis: top-N items, y-axis: hit-rate of all users. ARHR (right): x-axis: top-N items, y-axis: average reciprocal hit-rate of all users. 20

5. Conclusion and Future Work n Conclusion q Top-N Recommendation Problem and item-centric Algorithms n q Item-Graph model n n q Visualizing the relationship among items. Easy to update. Generalized Conditional Probability-based top-N recommendation algorithm n n Cosine-based, conditional probability-based Item-centric & based on the Item-Graph model Future Work q q Clustering items and measuring item-item similarities based on the Item. Graph model Speeding up the GCP method. 21

References n [Balabanovic 97] M. Balabanovic and Y. Shoham. Fab: Content-based, Collaborative Recommendation. Commun. ACM, 40(3): 66 -72, 1997. n [Breese 98] J. S. Breese, D. Heckerman, David and C. Kadie. Empirical Analysis of Predictive Algorithms for Collaborative Filtering. In Proceedings of the 14 th Conference on Uncertainty in Artificial Intelligence (UAI-98), pages 43 -52, San Francisco, 1998. n [Deshpande 04] M. Deshpande and G. Karypis. Item-based Top-N Recommendation Algorithms. ACM Trans. Inf. Syst. , 22(1): 143 -177, 2004. n [Lin 00] W. Lin. Association Rule Mining for Collaborative Recommender Systems. Thesis submitted for the Degree of M. S. in Computer Science. n [Linden 03] G. Linden, B. Smith and J. York. Amazon. com Recommendations: Itemto-Item Collaborative Filtering. IEEE Internet Computing, 7(1): 76 -80, 2003. n [Resnick 94] P. Resnick, N. Iacovou, M. Suchak, P. Bergstorm and J. Riedl. Group. Lens: An Open Architecture for Collaborative Filtering of Netnews. Proc. Computer Supported Cooperative Work Conf. , pages 175 -186, 1994. 22