Discriminative Probabilistic Models for Relational Data Ben Taskar

Discriminative Probabilistic Models for Relational Data Ben Taskar, Pieter Abbeel, Daphne Koller Guohua Hao

Tradition statistic classification Methods n n n Dealing with only ‘flat’ data – IID In many supervised learning tasks, entities to be labeled are related to each other in complex way and their labels are not independent This dependence is an important source of information to achieve better classification 3/6/2021 Guohua Hao

Collective Classification n Rather than classify each entity separately Simultaneously decide on the class label of all the entities together Explicitly take advantage of the correlation between the labels of related entitiies 3/6/2021 Guohua Hao

Undirected vs. directed graphical models n n Undirected graphical models do not impose the acyclicity constraint, but directed ones need acyclicity to define a coherent generative model Undirected graphical models are well suited for discriminative training, achieving better classification accuracy over generative training 3/6/2021 Guohua Hao

Our Hypertext Relational Domain Label Has. Word 1 . . . Label Has. Wordk Has. Word 1 Doc Has. Wordk Doc From To Link 3/6/2021 . . . Guohua Hao

Schema n A set of entity types n Attribute of each entity type n n n 3/6/2021 Content attribute E. X Label attribute E. Y Reference attribute E. R Guohua Hao

Instantiation n Provide a set of entities I (E) for each entity type E Specify the values of all the attribute of the entities, I. x, I. y, I. r is the instantiation graph, which is call relational skeleton in PRM 3/6/2021 Guohua Hao

Markov Network n Qualitative component – Cliques n Quantitative component – Potentials 3/6/2021 Guohua Hao

Cliques n A set of nodes for each in the graph G such that are connected by an edge in G 3/6/2021 Guohua Hao

Potentials n n The potential for the clique c defines the compatibility between values of variables in the clique Log-linearly combination of a set of features 3/6/2021 Guohua Hao

Probability in Markov Network n Given the values Markov Network 3/6/2021 of all nodes in the Guohua Hao

Conditional Markov Network n Specify the probability of a set of target variables Y given a set of conditioning variables X 3/6/2021 Guohua Hao

Relational Markov Network (RMN) n n Specifies the conditional probability over all the labels of all the entities in the instantiation given the relational structure and the content attributes Extension of the Conditional Markov Networks with a compact definition on a relational data set 3/6/2021 Guohua Hao

Relational clique template n n n F --- a set of entity variables (From) W--- the condition about the attributes of the entity variables (Where) S --- subset of attributes (content and label attribute) of the entity variables (Select) 3/6/2021 Guohua Hao

Relationship to SQL query SELECT doc 1. Category, doc 2. Category FROM doc 1, doc 2, Link link WHERE link. From=doc 1. key and link. To=doc 2. key Doc 1 Doc 2 Link 3/6/2021 Guohua Hao

Potentials n n Potentials are defined at the level of relational clique template The cliques of the same relational clique template have the same potential functions 3/6/2021 Guohua Hao

Unrolling the RMN n Given an instantiation of a relational schema, unroll the RMN as follows n n 3/6/2021 Find all the cliques in the unrolled the relational schema where the relational clique templates are applicable The potential of a clique is the same as that of the relational clique template which this clique belongs to Guohua Hao

link 1 Doc 2 link 2 Doc 3 3/6/2021 Guohua Hao

Probability in RMN 3/6/2021 Guohua Hao

3/6/2021 Guohua Hao

Learning RMN n n Given a set of relational clique templates Estimate feature weight w using conjugate gradient Objective function--Product of likelihood of instantiation and parameter prior Assume a shrinkage prior over feature weights 3/6/2021 Guohua Hao

Learning RMN (Cont’d) n The conjugate gradient of the objective function where 3/6/2021 Guohua Hao

Inference in RMN n Exact inference n n Intractable due to the network is very large and densely connected Approximate inference n Belief propagation 3/6/2021 Guohua Hao

Experiments n Web. KB dataset Four CS department websites n Five categories (faculty, student, project, course, other) n Bag of words on each page n Links between pages Experimental setup n Trained on three universities n Tested on fourth n n 3/6/2021 Guohua Hao

Flat Models n n Based only on the text content on the Web. Pages Incorporate meta-data 3/6/2021 Guohua Hao

Relational model n introduce relational clique template over the labels of two pages that are linked Doc 1 Doc 2 Link 3/6/2021 Guohua Hao

Relational model (Cont’d) n n relational clique template over the label of section and the label of the pages it is on Relational clique template over the label of the section containing the link and the label of the target page 3/6/2021 Guohua Hao

3/6/2021 Guohua Hao

Discriminative vs. Generative n n n Exit+Naïve Bayes: a complete generative model proposed by Getoor et al Exit+logistic: using logistic regression for the conditional probability distribution of page label given words Link: a fully discriminative training model 3/6/2021 Guohua Hao

3/6/2021 Guohua Hao