Zeroshot Learning 2015 4 2 Mun Jonghwan Zeroshot

Zero-shot Learning Traindata : • Which image shows a cat?

Zero-shot Learning Traindata : • Which image shows a giraffe?

Zero-shot Learning • Which image shows a giraffe? • Description - has long neck?

External information • Attribute - C. H. Lampert, Attribute-based classification for zero-shot visual object

Direct Attribute Prediction (DAP)[1] Label Attribute Image • Learn attribute classifier from related classes

Direct Attribute Prediction (DAP)[1] 1. Vocabulary of attributes and class decriptions - giraffe has

Relative Attribute[2] Jared Smiling: Miley Smiling Age: Hugh Clive Scarlett Jared Miley S Clive

Attribute Label Embedding (ALE)[2] • Embedding to attribute space • Search for the class

Word Vector[3] • Use CNN feature • Embedding vector is collected automatically from text

Relative information from word vector Tiger : bobcat = strong : ? bobcat :

Relative information from word vector Word vector Some ranking information Attribute embedding

Slides: 16

Download presentation

Zeroshot Learning 2015. 4. 2 Mun Jonghwan

Zero-shot Learning Traindata : • Which image shows a cat?

Zero-shot Learning Traindata : • Which image shows a giraffe?

Zero-shot Learning • Which image shows a giraffe? • Description - has long neck? Is black? Is spot? lives in plain?

External information • Attribute - C. H. Lampert, Attribute-based classification for zero-shot visual object classification, TPAMI 13[1] - D. Parikh, Relative attributes, ICCV 2011[3] - Z. Akata, Label embedding for attribute-based classification, CVPR 13[2] • Word vector - A. Frome, Devise: A deep visual-semantic embedding model, NIPS 13[3] - Z. Akata, Evaluation of output embedding for fine-grained image classification, CVPR 15[4] • Hierarchy - Usually used as side information • Co-occurrence - T. Mensink, Costa: Co-occurrence statistics for zero-shot classification, CVPR 14

Direct Attribute Prediction (DAP)[1] Label Attribute Image • Learn attribute classifier from related classes • Use attribute-to-class mapping for prediction

Direct Attribute Prediction (DAP)[1] 1. Vocabulary of attributes and class decriptions - giraffe has properties X and Y but not Z 2. Train classifier for each attribute X, Y, Z - From visual examples of related classes 3. Make image attributes predictions 4. Combine into decision: this image is not giraffe

Relative Attribute[2]

Relative Attribute[2] Jared Smiling: Miley Smiling Age: Hugh Clive Scarlett Jared Miley S Clive Miley H J Age Infer image category using max-likelihood

Attribute Label Embedding (ALE)[2] • Embedding to attribute space • Search for the class with the highest compatibility

Word Vector[3] • Use CNN feature • Embedding vector is collected automatically from text corpora • Embedding to word vector space

Word Vector[3] country capital

Survey result[4]

Relative information from word vector Tiger : bobcat = strong : ? bobcat : tiger = small : ?

Relative information from word vector Word vector Some ranking information Attribute embedding

Thank you