Introduction to Neural Networks Media IC System Lab
Introduction to Neural Networks 柯揚 Media IC & System Lab
Agenda • Origin of Neural Networks • Math of Neural Networks • Machine Learning • Deep Learning • Convolutional Neural Networks • Recurrent Neural Networks Media IC & System Lab 2
Origin Media IC & System Lab Observation Theory Verification Prediction 3
Origin Media IC & System Lab 4
Origin • Media IC & System Lab 5
Origin Take a look at nature Media IC & System Lab 6
Origin • Hubel and Wiesel Media IC & System Lab 7
Origin Math • Media IC & System Lab 8
Math • Media IC & System Lab 9
T @ July 17 th, 2018 T @ July 17 th, 2017 Math T @ July 17 th, 2016 July 17 th , 2019 T @ July 17 th, 2015 T @ July 17 th, 2014 • Media IC & System Lab 10
Math Machine Learning • Media IC & System Lab 11
Machine Learning • Media IC & System Lab 12
Machine Learning • 1. Collect Data • Most algorithms assume that data is IID • Independent & Identically distributed • An example: • Tell dogs apart from cats • Give your algorithm 990 images of dogs, 10 images of cats • If your algorithm always answers “dog”, what’s the error? Data is not IID Media IC & System Lab 13
Machine Learning • Media IC & System Lab 14
Machine Learning • Media IC & System Lab 15
Machine Learning • Media IC & System Lab 16
Machine Learning Deep Learning • Media IC & System Lab 17
Deep Learning • Media IC & System Lab 18
Deep Learning • Media IC & System Lab 19
Deep Learning • Media IC & System Lab 20
Deep Learning • Media IC & System Lab 21
Deep Learning • Media IC & System Lab 22
Deep Learning Convolutional Neural Networks • Media IC & System Lab 23
Convolutional Neural Networks • In a lot of data, we have direct neighborhood relations • • • Images Audio Video Natural Language … • All data that is sampled according to a single (or multiple) changing variable(s) • Time series: sampled at different times • Images: sampled at different locations Media IC & System Lab 24
Convolutional Neural Networks • Convolution: neighborhood operator Media IC & System Lab 25
Convolutional Neural Networks • Media IC & System Lab 26
Convolutional Neural Networks • Media IC & System Lab 27
Convolutional Neural Networks • Media IC & System Lab 28
Convolutional Neural Networks • Media IC & System Lab 29
Convolutional Neural Networks • Media IC & System Lab 30
Convolutional Neural Networks • Media IC & System Lab 31
Convolutional Neural Networks • Media IC & System Lab 32
Convolutional Neural Networks • Media IC & System Lab 33
Classification Loss Functions • Media IC & System Lab 34
Recurrent Neural Networks • Media IC & System Lab 35
Recurrent Neural Networks • Media IC & System Lab 36
Summary • Neural Networks: • Layers of linear functions + non-linearities • Convolutional neural networks: • Layers of convolutions + non-linearities • Recurrent neural networks: • Time steps of linear functions + non-linearities • Machine Learning • Learning from observations • DFLO= Data + Function + Loss + Optimisation Media IC & System Lab 37
- Slides: 37