Lecture 10 Convolutional Neural Networks Alireza Akhavan Pour

  • Slides: 21
Download presentation
Lecture 10: Convolutional Neural Networks Alireza Akhavan Pour CLASS. VISION SRTTU – A. Akhavan

Lecture 10: Convolutional Neural Networks Alireza Akhavan Pour CLASS. VISION SRTTU – A. Akhavan Lecture 10 - 1 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

Learning to detect edges 1 0 -1 2 0 -2 1 0 -1 3

Learning to detect edges 1 0 -1 2 0 -2 1 0 -1 3 0 -3 10 0 -10 3 0 -3 sobel 3 0 1 2 7 4 1 5 8 9 3 1 2 7 2 5 1 3 0 1 3 1 7 8 4 2 1 6 2 8 2 4 5 2 3 9 SRTTU – A. Akhavan 45 ? 70? 73? Lecture 10 - 3 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

Valid and Same convolutions nxn * fxf -> n-f+1 x n-f+1 6 x 6

Valid and Same convolutions nxn * fxf -> n-f+1 x n-f+1 6 x 6 * 3 x 3 -> 4 x 4 “Valid”: “Same”: Pad so that output size is the same as the input size. n+2 p-f+1 x n+2 p-f+1 P=1 ﺑﺮﺍی ﻣﺜﺎﻝ ﻗﺒﻞ SRTTU – A. Akhavan Lecture 10 - 5 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

Strided convolution 2 3 3 4 7 43 4 4 6 34 2 4

Strided convolution 2 3 3 4 7 43 4 4 6 34 2 4 9 4 6 1 6 0 9 21 8 0 7 12 4 0 3 2 3 -13 4 40 8 -143 3 40 8 -134 9 40 7 43 7 8 0 3 6 0 6 3 0 4 2 4 -13 2 04 1 -134 8 0 4 3 -134 4 0 4 6 3 4 3 2 0 4 1 0 9 8 0 3 2 0 -1 1 0 3 -13 9 0 2 -13 1 0 4 3 1 1 2 12 12 SRTTU – A. Akhavan 3 1 -1 4 0 0 4 2 3 Lecture 10 - 6 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

Summary of convolutions padding p stride s SRTTU – A. Akhavan Lecture 10 -

Summary of convolutions padding p stride s SRTTU – A. Akhavan Lecture 10 - 7 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

Convolutions on RGB image 4 x 4 SRTTU – A. Akhavan Lecture 10 -

Convolutions on RGB image 4 x 4 SRTTU – A. Akhavan Lecture 10 - 8 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

Multiple filters 3 x 3 6 x 3 SRTTU – A. Akhavan 4 x

Multiple filters 3 x 3 6 x 3 SRTTU – A. Akhavan 4 x 4 Lecture 10 - 9 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

Pooling layer: Max pooling 5 x 5 x 2 1 3 2 9 1

Pooling layer: Max pooling 5 x 5 x 2 1 3 2 9 1 1 5 1 3 2 8 3 5 1 0 5 6 1 2 9 5 x 5 SRTTU – A. Akhavan 3 x 3 x 2 9 9 5 8 6 9 f = 3 S = 1 Lecture 10 - 14 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

Average pooling 1 3 2 1 2 9 1 1 3. 75 1. 25

Average pooling 1 3 2 1 2 9 1 1 3. 75 1. 25 1 4 2 3 4 2 5 6 1 2 SRTTU – A. Akhavan Lecture 10 - 15 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

Summary of pooling Hyperparameters: f : filter size s : stride Max or average

Summary of pooling Hyperparameters: f : filter size s : stride Max or average pooling SRTTU – A. Akhavan Lecture 10 - 16 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

Neural network example SRTTU – A. Akhavan Lecture 10 - 17 ﻓﺮﻭﺭﺩیﻦ ۲۹ -

Neural network example SRTTU – A. Akhavan Lecture 10 - 17 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

Neural network example Activation shape Activation Size # parameters Input: (32, 3) 3, 072

Neural network example Activation shape Activation Size # parameters Input: (32, 3) 3, 072 0 CONV 1 (f=5, s=1) (28, 8) 6, 272 208 POOL 1 (14, 8) 1, 568 0 CONV 2 (f=5, s=1) (10, 16) 1, 600 416 POOL 2 (5, 5, 16) 400 0 FC 3 (120, 1) 120 48, 001 FC 4 (84, 1) 84 10, 081 Softmax (10, 1) 10 841 SRTTU – A. Akhavan Lecture 10 - 18 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

10 10 10 0 0 0 10 10 10 0 0 0 1 0

10 10 10 0 0 0 10 10 10 0 0 0 1 0 -1 0 30 30 0 Parameter sharing: A feature detector (such as a vertical edge detector) that’s useful in one part of the image is probably useful in another part of the image. Sparsity of connections: In each layer, each output value depends only on a small number of inputs. SRTTU – A. Akhavan Lecture 10 - 20 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷

ﻣﻨﺎﺑﻊ • https: //www. coursera. org/specializations/dee p-learning • http: //cs 231 n. stanford.

ﻣﻨﺎﺑﻊ • https: //www. coursera. org/specializations/dee p-learning • http: //cs 231 n. stanford. edu/ SRTTU – A. Akhavan Lecture 10 - 21 ﻓﺮﻭﺭﺩیﻦ ۲۹ - چﻬﺎﺭﺷﻨﺒﻪ ۱۳۹۷