Image Classification and Convolutional Networks Presented by T

Image Classification and Convolutional Networks Presented by T. H. Yang, S. H. Cho, L. W. Kao

Outline Convolution Visualizing What Neural Networks (Conv. Nets) Conv. Nets makes Conv. Nets ticks? Transfer Learning

What makes Conv. Nets ticks? 1). [Visualizing and Understanding Convolution Networks, Zeiler and Fergus, 2013] This CNN is very similar to Alex. Net!

What makes Conv. Nets ticks? 1). [Visualizing and Understanding Convolution Networks, Zeiler and Fergus, 2013] Depth is important!

What makes Conv. Nets ticks? Changing size of FC layers: little to no improvement Changing size of Conv layers: reasonable improvement !!

What makes Conv. Nets ticks? 2). [Very Deep Convolutional Networks for Large-Scale Image Recognition, Simonyan et al. , 2014]

What makes Conv. Nets ticks? - Summary

Classical Models - Goog. Le. Net -

Outline Convolution Visualizing What Neural Networks (Conv. Nets) Conv. Nets makes Conv. Nets ticks? Transfer Learning

Transfer Learning D E T S “You need a lot of data if you want to train/use CNNs” U B

Transfer Learning training items test items training items Humans can learn in many domains. Humans can also transfer from one domain to other domains. test items Transfer of learning across domains Traditional ML in multiple domains

Transfer Learning Process of Traditional ML training items Learning System Learning Process of Transfer Learning training items Learning System Knowledge Learning System

Transfer Learning

Transfer Learning Very little data Quite a lot of data Very similar dataset Very different dataset Use linear classifier on top layer Big Trouble!!! Finetune a few layers Finetune a larger number of layers

Transfer Learning T I “How does transfer learning E V work on CNNs? ” O R P

Transfer Learning 3). [Very Deep Convolutional Networks for Large-Scale Image Recognition, Simonyan et al. , 2014] Separate the database into two pieces A and B.

Transfer Learning

Transfer Learning Watch out for fragile co-adaptation!! Transfer learning sometimes even works better!!