Selftaught Learning Transfer Learning from Unlabeled Data Rajat

Self-taught Learning: Transfer Learning from Unlabeled Data Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer, Andrew Y. Ng ICML 2007 Yu-chen Kao Department of Computer Science & Information Engineering National Taiwan Normal University 2010. 01. 28

Outline o Introduction o Self-taught Learning o Experiments 2

Introduction o Self-taught learning is a concept of learning that we can use unlabeled data to assist the learning from labeled data. o The unlabeled data doesn’t necessarily share the class labels of the labeled data. o Labeled data is more expensive than unlabeled data! 3

Introduction 4

Self-taught Learning: Motivation o Many randomly obtained samples, such as speech or images, will contain basic patterns that are similar to those we want to classify 0. 86 x = +0. 75 x +1. 45 x 5

Self-taught Learning: Approach 1. Find some common patterns from a massive of unlabeled data, to learn higher-level representations. 2. Use these patterns to represent our labeled training data, construct a new feature based on these patterns. 6

Self-taught Learning: Target Function [1/2] o First, we minimize this function to get the common patterns 7

Self-taught Learning: Target Function [2/2] o Then, we get our new feature by optimizing this: 8

Experiments: Music genre classification 9

Thank you!