FACTORIZATION MACHINE MODEL OPTIMIZATION AND APPLICATIONS 1 Yang

  • Slides: 24
Download presentation
FACTORIZATION MACHINE: MODEL, OPTIMIZATION AND APPLICATIONS 1 Yang LIU Email: yliu@cse. cuhk. edu. hk

FACTORIZATION MACHINE: MODEL, OPTIMIZATION AND APPLICATIONS 1 Yang LIU Email: yliu@cse. cuhk. edu. hk Supervisors: Prof. Andrew Yao Prof. Shengyu Zhang

OUTLINE Factorization machine (FM) �A generic predictor � Auto feature interaction Learning algorithm �

OUTLINE Factorization machine (FM) �A generic predictor � Auto feature interaction Learning algorithm � Stochastic gradient descent (SGD) �… Applications � Recommendation systems � Regression and classification �… 2

DOUBAN MOVIE 3

DOUBAN MOVIE 3

PREDICTION TASK ? ? e. g. Alice rates Titanic 5 at time 13 4

PREDICTION TASK ? ? e. g. Alice rates Titanic 5 at time 13 4

PREDICTION TASK 5

PREDICTION TASK 5

LINEAR MODEL – FEATURE ENGINEERING Linear SVM Logistic Regression 6

LINEAR MODEL – FEATURE ENGINEERING Linear SVM Logistic Regression 6

FACTORIZATION MODEL Interaction between variables 7

FACTORIZATION MODEL Interaction between variables 7

INTERACTION MATRIX W 8

INTERACTION MATRIX W 8

INTERACTION MATRIX W 9

INTERACTION MATRIX W 9

INTERACTION MATRIX ? W 10

INTERACTION MATRIX ? W 10

INTERACTION MATRIX W = V T V k 11

INTERACTION MATRIX W = V T V k 11

INTERACTION MATRIX W = V T V k 12

INTERACTION MATRIX W = V T V k 12

INTERACTION MATRIX W = V T V 13

INTERACTION MATRIX W = V T V 13

INTERACTION MATRIX W = V T V 14

INTERACTION MATRIX W = V T V 14

INTERACTION MATRIX W = V T V Factorization 15

INTERACTION MATRIX W = V T V Factorization 15

INTERACTION MATRIX W = Machine V T V Factorization 16

INTERACTION MATRIX W = Machine V T V Factorization 16

FM: PROPERTIES 17

FM: PROPERTIES 17

OPTIMIZATION TARGET 18

OPTIMIZATION TARGET 18

STOCHASTIC GRADIENT DESCENT (SGD) 19

STOCHASTIC GRADIENT DESCENT (SGD) 19

APPLICATIONS EMI Music Hackathon 2012 � Song recommendation Given: � Historical ratings � User

APPLICATIONS EMI Music Hackathon 2012 � Song recommendation Given: � Historical ratings � User demographics # features: 51 K # items in training: 188 K ? 20

RESULTS FOR EMI MUSIC FM: Root Mean Square Error (RMSE) 13. 27626 � Target

RESULTS FOR EMI MUSIC FM: Root Mean Square Error (RMSE) 13. 27626 � Target value [0, 100] � The best (SVD++) is 13. 24598 Details � Regression � Converges in 100 iterations � Time for each iteration: < 1 s Win 7, Intel Core 2 Duo CPU 2. 53 GHz, 6 G RAM 21

OTHER APPLICATIONS Ads CTR prediction (KDD Cup 2012) � Features User_info, Ad_info, Query_info, Position,

OTHER APPLICATIONS Ads CTR prediction (KDD Cup 2012) � Features User_info, Ad_info, Query_info, Position, etc. �# features: 7. 2 M � # items in training: 160 M � Classification � Performance: AUC: 0. 80178, the best (SVM) is 0. 80893 22

OTHER APPLICATIONS Hi. Cloud App Recommendation � Features App_info, Smartphone model, installed apps, etc.

OTHER APPLICATIONS Hi. Cloud App Recommendation � Features App_info, Smartphone model, installed apps, etc. �# features: 9. 5 M � # items in training: 16 M � Classification � Performance: Top 5: 8%, Top 10: 18%, Top 20: 32%; AUC: 0. 78 23

SUMMARY FM: a general predictor Works under sparsity Linear computation complexity Estimates interactions automatically

SUMMARY FM: a general predictor Works under sparsity Linear computation complexity Estimates interactions automatically Works with any real valued feature vector THANKS! 24

Factorization Machine Im Jerry Factorization Machine Factorization MethodsFactorization Machine Im Jerry Factorization Machine Factorization Methods
FACTORIZATION MACHINE MODEL OPTIMIZATION AND APPLICATIONS 1 YangFACTORIZATION MACHINE MODEL OPTIMIZATION AND APPLICATIONS 1 Yang
Factorization of Polynomials What is Factorization factorization orFactorization of Polynomials What is Factorization factorization or
Factorization by grouping the terms Factorization by GroupingFactorization by grouping the terms Factorization by Grouping
Prime Factorization Math Prime Factorization Of a NumberPrime Factorization Math Prime Factorization Of a Number
Prime Factorization 5 th Grade Math Prime FactorizationPrime Factorization 5 th Grade Math Prime Factorization
Factorization Factorization is writing a number as aFactorization Factorization is writing a number as a
Prime Factorization FACTOR TREE Prime Factorization Essential QuestionPrime Factorization FACTOR TREE Prime Factorization Essential Question
SEJARAH SPIRAL MODEL Proses model yang lain yangSEJARAH SPIRAL MODEL Proses model yang lain yang
Multiobjective Optimization Multiobjective Optimization Some optimization problems haveMultiobjective Optimization Multiobjective Optimization Some optimization problems have
linear optimization u nonlinear optimization u quadratic optimizationlinear optimization u nonlinear optimization u quadratic optimization
linear optimization u nonlinear optimization u quadratic optimizationlinear optimization u nonlinear optimization u quadratic optimization
Code Optimization II Machine Dependent Optimization Topics nCode Optimization II Machine Dependent Optimization Topics n
Engineering Optimization Applications 2010521 Optimization Problem Design andEngineering Optimization Applications 2010521 Optimization Problem Design and
Machine learning optimization Usman Roshan Machine learning MathematicalMachine learning optimization Usman Roshan Machine learning Mathematical
Grovers Algorithm in Machine Learning and Optimization ApplicationsGrovers Algorithm in Machine Learning and Optimization Applications
Introduction to tensor tensor factorization and its applicationsIntroduction to tensor tensor factorization and its applications
Matrix Factorization and its applications By Zachary 16Matrix Factorization and its applications By Zachary 16
RELATIONAL MODEL Model Relasional merupakan model data yangRELATIONAL MODEL Model Relasional merupakan model data yang
Sparse optimization with applications Sparse optimization sv glesSparse optimization with applications Sparse optimization sv gles
input Juicer Machine output Juicer Machine This Machineinput Juicer Machine output Juicer Machine This Machine
Machine Learnin Andrew Ng Machine Learning summary MachineMachine Learnin Andrew Ng Machine Learning summary Machine
Machine Instruction Characteristics Machine Instruction Characteristics Machine InstructionsMachine Instruction Characteristics Machine Instruction Characteristics Machine Instructions