Teaching Machines to Learn by Metaphors Omer Levy
Teaching Machines to Learn by Metaphors Omer Levy & Shaul Markovitch Technion – Israel Institute of Technology
Concept Learning by Induction
Few Examples
Transfer Learning Target (New) Source (Original)
Define: Related Concept
Transfer Learning Approaches • Common Inductive Bias • Common Instances • Common Features
Different Feature Space
Example -3 -2 0 2 3
Example -3 -2 0 0 2 3 4 9
Example -3 -2 0 0 2 3 4 9
Common Inductive Bias -3 -2 0 0 2 3 4 9
Common Inductive Bias -3 -2 0 0 2 3 4 9
Common Instances -3 -2 0 0 2 3 4 9
Common Features 3 2 -2 -3 4 9
New Approach to Transfer Learning
Our Solution: Metaphors
Metaphors Target (New) Source (Original)
Source Concept Learner Target Metaphor Learner +/-
Theorem •
The Metaphor Theorem •
Redefine Transfer Learning •
Redefine Transfer Learning •
Metaphor Learning Framework
Concept Learning Framework Search Algorithm Hypothesis Space Data Evaluation Function
Metaphor Learning Framework Source Search Algorithm Metaphor Space Target Evaluation Function
Metaphor Evaluation
Metaphor Evaluation •
Metaphor Evaluation •
Metaphor Evaluation
Metaphor Evaluation
Metaphor Evaluation
Metaphor Evaluation
Metaphor Spaces
Metaphor Spaces • General • Few Degrees of Freedom • Representation-Specific Bias
Geometric Transformations Я R
Dictionary-Based Metaphors cheese queso
Linear Transformations •
Which metaphor space should I use?
Which metaphor space should I use? Automatic Selection of Metaphor Spaces
Which metaphor space should I use? Automatic Selection of Metaphor Spaces Occam’s Razor
Which metaphor space should I use? Automatic Selection of Metaphor Spaces Occam’s Razor Structural Risk Minimization
Automatic Selection of Metaphor Spaces
Automatic Selection of Metaphor Spaces
Automatic Selection of Metaphor Spaces
Empirical Evaluation
Reference Methods Baseline • Target Only • Identity Metaphor • Merge State-of-the-Art • Frustratingly Easy Domain Adaptation – Daumé, 2007 • Multi. Task Learning – Caruana, 1997; Silver et al, 2010 • Tr. Ada. Boost – Dai et al, 2007
Digits: Negative Image
Digits: Negative Image
Digits: Negative Image
Digits: Higher Resolution
Digits: Higher Resolution
Digits: Higher Resolution
Wine
Wine
Qualitative Results Transfer Learning Target Task Instance Digits: Negative Image Digits: Higher Resolution Target Sample Size 1 2 5 10
Discussion
Recap • Problem: Concept learning with few examples • Solution: Metaphors
Recap •
Recap •
Recap •
Recap •
What if the concepts are not related?
What if the concepts are not related?
Metaphors are not a measure of relatedness
Metaphors are not a measure of relatedness Metaphors explain how concepts are related
Vision
METAPHORS Explaining how concepts are related since 2012.
- Slides: 71