Chap 11 Practice in CNN Contents Data Augmentation

Chap 11. Practice in CNN 이성호

Contents • Data Augmentation • Transfer learning • Convolution • Implementation Details

cf) How to improve Deep learning performance? • Improve Performance with Data Performace with Algorithms performace with Algorithm Tuning Performance with Ensembles

Improve performance with Data • Get More Data • Invent More Data • Rescale your data • Transform your data • Feature Selection

Data Augmentation

Data Augmentation • Horizontal Flip • Random crop, Scales • Color jittering • etc

Data Augmentation – Horizontal Flip

Data Augmentation – Random crops/scales

Data Augmentation – Color jittering

etc

Transfer Learning

Transfer Learning

Transfer Learning

Transfer Learning

Transfer Learning

Convolution 3*3 convolution 2회 receptive field = 5 * 5 3*3 convolution 3회 receptive field = 7 * 7

Convolution • 3 * 3 filter * 3 Receptive field = 7 * 7 • 7 * 7 filter Receptive field = 7 * 7 • Which is better? Input H * W * C, Convolution C filter, preserve depth(stride 1, padding to preserve H, W) # of computation 3 * 3 filter * 3 Computation: 3 * C * (3 * H * W * C) = 27 HWC^2 7 * 7 filter: C * 7 * H * W * C = 49 HWC^2

Convolution • Using small size filter, Fewer parameters, More nonlinearity, Less compute 따라서, 5*5 filter보다는 3*3이 좋다. 그렇지만, 1*1은 잘 사용하지 않는다.

Convolution • Why not try 1 * 1? receptive field가 자기 자신밖에 없으므로 • 그 대신에, bottleneck Sandwich를 통해 Parameter를 줄여주는 데 사용된다.

Convolution • Bottleneck Sandwich

Convolution

Stack convolution example

Convolution – im 2 col

Convolution - FFT •

Convolution - FFT

Convolution – Fast Algorithm

Convolution – Fast Algorithm