Zeroshot Learning 2015 4 2 Mun Jonghwan Zeroshot
![Direct Attribute Prediction (DAP)[1] Label Attribute Image • Learn attribute classifier from related classes](https://slidetodoc.com/presentation_image_h/33b2bb874f2e4a246b5198b867e852d7/image-6.jpg "Direct Attribute Prediction (DAP)[1] Label Attribute Image • Learn attribute classifier from related classes")
![Direct Attribute Prediction (DAP)[1] 1. Vocabulary of attributes and class decriptions - giraffe has](https://slidetodoc.com/presentation_image_h/33b2bb874f2e4a246b5198b867e852d7/image-7.jpg "Direct Attribute Prediction (DAP)[1] 1. Vocabulary of attributes and class decriptions - giraffe has")
![Relative Attribute[2]](https://slidetodoc.com/presentation_image_h/33b2bb874f2e4a246b5198b867e852d7/image-8.jpg "Relative Attribute[2]")
![Relative Attribute[2] Jared Smiling: Miley Smiling Age: Hugh Clive Scarlett Jared Miley S Clive](https://slidetodoc.com/presentation_image_h/33b2bb874f2e4a246b5198b867e852d7/image-9.jpg "Relative Attribute[2] Jared Smiling: Miley Smiling Age: Hugh Clive Scarlett Jared Miley S Clive")
![Attribute Label Embedding (ALE)[2] • Embedding to attribute space • Search for the class](https://slidetodoc.com/presentation_image_h/33b2bb874f2e4a246b5198b867e852d7/image-10.jpg "Attribute Label Embedding (ALE)[2] • Embedding to attribute space • Search for the class")
![Word Vector[3] • Use CNN feature • Embedding vector is collected automatically from text](https://slidetodoc.com/presentation_image_h/33b2bb874f2e4a246b5198b867e852d7/image-11.jpg "Word Vector[3] • Use CNN feature • Embedding vector is collected automatically from text")
![Word Vector[3] country capital](https://slidetodoc.com/presentation_image_h/33b2bb874f2e4a246b5198b867e852d7/image-12.jpg "Word Vector[3] country capital")
![Survey result[4]](https://slidetodoc.com/presentation_image_h/33b2bb874f2e4a246b5198b867e852d7/image-13.jpg "Survey result[4]")
- Slides: 16
Zeroshot Learning 2015. 4. 2 Mun Jonghwan
Zero-shot Learning Traindata : • Which image shows a cat?
Zero-shot Learning Traindata : • Which image shows a giraffe?
Zero-shot Learning • Which image shows a giraffe? • Description - has long neck? Is black? Is spot? lives in plain?
External information • Attribute - C. H. Lampert, Attribute-based classification for zero-shot visual object classification, TPAMI 13[1] - D. Parikh, Relative attributes, ICCV 2011[3] - Z. Akata, Label embedding for attribute-based classification, CVPR 13[2] • Word vector - A. Frome, Devise: A deep visual-semantic embedding model, NIPS 13[3] - Z. Akata, Evaluation of output embedding for fine-grained image classification, CVPR 15[4] • Hierarchy - Usually used as side information • Co-occurrence - T. Mensink, Costa: Co-occurrence statistics for zero-shot classification, CVPR 14
Direct Attribute Prediction (DAP)[1] Label Attribute Image • Learn attribute classifier from related classes • Use attribute-to-class mapping for prediction
Direct Attribute Prediction (DAP)[1] 1. Vocabulary of attributes and class decriptions - giraffe has properties X and Y but not Z 2. Train classifier for each attribute X, Y, Z - From visual examples of related classes 3. Make image attributes predictions 4. Combine into decision: this image is not giraffe
Relative Attribute[2]
Relative Attribute[2] Jared Smiling: Miley Smiling Age: Hugh Clive Scarlett Jared Miley S Clive Miley H J Age Infer image category using max-likelihood
Attribute Label Embedding (ALE)[2] • Embedding to attribute space • Search for the class with the highest compatibility
Word Vector[3] • Use CNN feature • Embedding vector is collected automatically from text corpora • Embedding to word vector space
Word Vector[3] country capital
Survey result[4]
Relative information from word vector Tiger : bobcat = strong : ? bobcat : tiger = small : ?
Relative information from word vector Word vector Some ranking information Attribute embedding
Thank you