Music Recommendation by Unified Hypergraph Combining Social Media

Music Recommendation by Unified Hypergraph: Combining Social Media Information and Music Content Jiajun Bu, Shulong Tan, Chun Chen, Can Wang, Hao Wu, Lijun Zhang and Xiaofei He Zhejiang University 1

Multi-type Media Fusion • Content analysis – text – Image – Audio – Video – …… • Social analysis Hypergraph – Friendship – Interest group – Resource collection – Tag – …… 2

Outlines • Music Recommendation • Social media information • Unified Hypergraph Model • Music Recommendation on Hypergraph (MRH) • Experimental results 3

Music Recommendation n We have huge amount of music available in music social communities It is difficult to find music we would potentially like Music Recommendation is needed! Recommended music by the Last. fm. 4

Traditional Music Recommendation n Traditional music recommendation methods only utilize limited kinds of social information n Collaborative Filtering (CF) only uses rating information n Acoustic-based method only utilizes acoustic features n Hybrid method just combines these two 5

Music Social Community [www. pandora. com] Social activities Actions which between users can do on resources 6

Introduction to Last. fm Users can listen to music. Users canare bookmark These music tracks resources bylikes tags. best. this user Users can also make friends. Users can join in groups 7

Social Media Information in Last. fm Memberships Friendships Tagging relations Listening relations Inclusion relations 8

Social Media Information n The rich social media information is valuable for music recommendation. ØTo build the users’ preference profiles. ØTo predict users’ interests from their friends. ØTo recommend music tracks by albums or artists. Ø… 9

How About Graph Model? n Use traditional graph to model social media information but fail to keep high-order relations in social media information (u 1, t 1, r 1) (u 1, t 2, r 2) (u 2, t 2, r 1) It is unclear whether u 2 bookmarks r 1, r 2, or both. 10

Unified Hypergraph Model n Using a unified hypergraph to model multi-type objects and the high-order relations ØEach edge in a hypergraph, called a hyperedge, is an arbitrary non-empty subset of the vertex set ØModeling each high-order relation by a hyperedge, so hypergraphs can capture high-order relations naturally (u 1, t 1, r 1) (u 1, t 2, r 2) (u 2, t 2, r 1) The high-order relations among the three types of objects can be naturally represented as triples. 11

Unified Hypergraph Construction tag Each type of relations corresponds to a certain type of hyperedges in the unified hypergraph. album tag album The six types of objects form the vertex set of the unified hypergraph. 12

Hyperedges Construction Details : a hyperedge corresponding to each pairwise friendship : a hyperedge corresponding to each tagging relation : a hyperedge corresponding to each group : a hyperedge for each album or artist : a hyperedge for each track-track similarity relation : a hyperedge for each user-track listening relation 13

Ranking on Unified Hypergraph Setting a user as the query Track List … Tracks have more strong “hyperpaths” to the query user will get higher ranking scores 14

Notation • A unified hypergraph • : Vertex-hyperedge incidence matrix 15

Notation-2 : the degree of a hyperedge is the number of vertices in the hyperedge : • : the degree of a vertex is the weight sum of all hyperedges the vertex belongs to: • • Dv , De and W : diagonal matrices consisting of hyperedge degrees, vertex degrees and hyperedge weights 16

Problem Definition • Given some query vertices from , rank the other vertices on the unified hypergraph according to their relevance to the queries. • : the ranking score of the i-th object • : the vector of ranking scores • : the query vector 17

Cost Function Vertices contained in many common hyperedges should have similar ranking scores Obtained ranking scores should be similar to pre-given labels The optimal ranking result is achieved when Q(f) is minimized 18

Matrix-vector Form 19

Optimal Solution Requiring that the gradient of Q(f) vanish gives the following this equation We define Note: all the matrices are highly sparse! 20

Music recommendation on Hypergraph (MRH) • The offline training phase: ØConstructing matrix H and W ØComputing matrix Dv and De ØCalculating , where • The online recommendation phase: ØBuilding the query vector y ØComputing the ranking results f* ØRecommending top tracks which not listened 21

General Ranking Framework Setting a user as the query User List … For friend recommendation Group List … For group recommendation Tag List … For topic recommendation Track List … For music recommendation Album List … For album recommendation Artist List … For artist recommendation 22

Personalized Tag Recommendation Tag List Setting a user and an resource as the queries … Personalized Tag recommendation for the target user and resource 23

Objects and Relations in Our Dataset Objects Relations 24

Compared Algorithms Information Used User-based Collaborative Filtering (CF) R 3 Acoustic-based music recommendation (AB) R 3, R 9 Ranking on Unified Graph (RUG) R 1, R 2, R 3, R 4, R 5, R 6, R 7, R 8, R 9 Our proposed music recommendation on Hypergraph method (MRH) • • • R 1: friendship relations R 2: membership relations R 3: listening relations R 4: tagging relations on tracks R 5: tagging relations on albums MRH-hybrid R 3, R 9 MRH-social R 1, R 2, R 3, R 4, R 5, R 6, R 7, R 8 MRH R 1, R 2, R 3, R 4, R 5, R 6, R 7, R 8, R 9 • • R 6: tagging relations on artists R 7: track-album inclusion relations R 8: album-artist inclusion relations R 9: similarities between tracks 25

Performance Comparison of recommendation algorithms in terms of MAP and F 1. Comparison of recommendation algorithms in terms of NDCG. It is clear that our proposed algorithm significantly outperforms the other recommendation algorithms 26

Precision-Recall Curves Comparing to MRH-social, MRH uses similarity relations among tracks Acoustic-based Our proposed (AB) method alleviates works these worst. problems. That is because MRH-hybrid acousticonly CF algorithm does not work well too. This is probably because The superiority ofmethod MRH over RUG indicates that the hypergraph isthe indeed a additionally. Wefor find that using acoustic information can improve uses based similarity method relations incurs the among semantic gap tracks and similarities listening based relations, on the user-track matrix in this ourmusic data setand is highly sparse better choice modeling complex relations in social media information recommendation result, when we only care top. CF ranking music tracks. acoustic content butare itespecially works not always much consistent better thanwith AB and human knowledge 27

Social Information Contribution MRH using listening relations MRH using listening The MRH baseline is MRH onlyrelations and tagging relations, and socialrelations and inclusion using listening relations Comparison of MRH on different subsets of social media information in terms of MAP and F 1. There is a little improvement at lower ranks obtained by social Tagging relations do not improve the performance. That is because there relations. Intuitively, the users’ tastes may bewe inferred from friendship inclusion relations among resources, music is Using a strong correlation between listening relationscan andrecommend tagging relations, and membershipasrelations. tracks in the same similar as theistracks performed by and thusorthe usagealbums, of taggingwell relations limited the same or similar artists. So the performance is improved greatly. 28

An Example Top 5 Recommended Tracks for No. 793 User No. 793 joins in the groups about metal and named The reason these Slipknot. Users in three tracks are these groups are recommended is fans of Metallica that user No. 793 (one of the four likes music come most popular heavy from System of a metal band. ) Down best. and Slipknot War? From: System of a Down Know From: System of a Down Dirty Window From: Metallica Deer Dance From: System of a Down Spit It Out From: Slipknot 29

Conclusion • We use the unified hypergraph model to fuse multi-type media, includes multi-type social media information and music content. ØSocial media information is very useful for music recommendation. ØHypergraphs can accurately capture the high -order relations among various types of objects. 30

31