Representation Learning of Knowledge Graphs with Hierarchical Types

Representation Learning of Knowledge Graphs with Hierarchical Types Ruobing Xie, Zhiyuan Liu, Maosong Sun 2016/05/22

Outline • Introduction • Method • Experiments • Summary

Outline • Introduction • Method • Experiments • Summary

What is knowledge graph?

write

Hierarchical types

Outline • Introduction • Method • Experiments • Summary

Translation-based method • Project entities and relations into a continuous low-dimensional vector space • Consider relations as translating operations between head and tail entities

Enhanced energy function

Hierarchical type encoders • Recursive Hierarchy Encoder (RHE) • Weighted Hierarchy Encoder (WHE)

Hierarchical type encoders

Objective Formalization • Margin-based score function • with negative sampling

Type constraints • Soft type constraints in training (STC) • improve the probability of selecting entities which have the same types when negative sampling • Type constraints in evaluation (TCE) • remove all candidates which don’t follow the type constraints in evaluation

Soft type constraints William Shakespeare The Great Wall William Shakespeare Jane Austen

Outline • Introduction • Method • Experiments • Summary

Datasets • FB 15 k: A typical KG dataset extracted from Freebase (Bordes et al. , 2013) • FB 15 k+: A new dataset based on FB 15 K with low-frequency relations

Evaluation results on entity prediction

Evaluation results on entity prediction with TCE

Evaluation results on long-tail distribution

Outline • Introduction • Method • Experiments • Summary

Summary • We encode hierarchical type information into knowledge representation learning • Type information, either in form of projection matrices or type constraints, could provide significant supplements for KRL • It works especially in long-tail distribution

Thanks