The Relationship between Deep Learning and Brain Function

The Relationship between Deep Learning and Brain Function Sukhjinder Nahal, Jamal Wilson, Abel Renteria, Nusseir Moath

What is Deep Learning? ▰ Deep Learning is a branch of the broader field known as machine learning. ▰ It uses algorithms that mimic the structure and function of the human brain as artificial neural networks. ▰ E. g. - CNN (Convolutional Neural Networks) 2

Scalability of Deep Learning ▰ Deep learning can come up with better results when it is given larger sets of data as an input. ▰ Older learning algorithms don’t scale as well. ▰ We have access to large datasets now due to the decreasing cost of storage. ▰ This is one of the reasons that deep learning has become a feasible technology. 3

Supervised vs Unsupervised Learning ▰ Supervised training requires a large amount of data that is labelled for training ▰ Unsupervised training on the other hand does not require labeled data. It can sort and classify the data it is given without human intervention. ▰ CNN (Convolutional Neural Networks) are trained using the supervised method. ▰ The brain’s architecture of neurons is comparable to convolutional neural networks but they differ in the way they learn. 4

CNN: Convolutional Layer 5

CNN: Pooling Layer ▰ Responsible for down sampling the inputted image. ▰ Placed in between convolutional layers ▰ Decreasing the resolution of input reduces the amount of computation. 6

CNN: Rectified Linear Unit (Re. LU) ▰ Helps in computational costs when processing images. ▰ Turns all negative values into zero. ▰ Output of a Re. LU layer is the same size as what is put into it, but with all negative values removed. 7

CNN: Fully Connected Layer ▰ After a few repetitions of Convolution and pooling a fully connected layer is created to connect the neurons between the current layer and the previous layer. 8

Backpropagation ▰ Every image that is processed receives a weight and an error ▰ Error is calculated by subtracting the right answer which would be a 1 by the generated weight. ▰ This process adjusts the features and weights up and down to see how the error changes. 9

Visual Cortex ▰ The Visual Cortex functions similarly to how a CNN works ▰ Comprised of 6 Layers ▰ Each layer passes the information from one layer to next and extracts features 10

Differences Between the Brain and CNN ▰ Section A- Demonstrates how the supervised learning occurs using labeled data. ▰ Section B- Demonstrates how the brain uses supervised and unsupervised learning. ▰ Section C- shows how information enters the brain from the sensory inputs and the outputs that are generated as a result. 11

Memory ▰ Memory or storage is a vital part in the learning process of both the human brain and in deep learning ▰ Three types of memory are need for learning ▻ Content Addressable Memory ▻ Working Memory ▻ Implicit Memory 12

Image. Net ▰ Image. Net is a project that contains a large set of visual data which has been hand classified. It is used to train and test deep learning technology. ▰ Organized according to the Word. Net hierarchy ▰ Images are described using a synset (synonym set) ▰ According to Tomaso Poggio “Deep networks trained with Image. Net seem to mimic not only the recognition performance but also the tuning properties of neurons in cortical areas of the visual cortex of monkeys. ” 13

ILSVRC (Image. Net Large Scale Visual Recognition Challenge) ▰ Annually Held Contest ▰ Participants train their algorithm using images from a dataset and then automatically label a set of test images. ▰ The challenge consists of three different tasks ▻ Image Classification ▻ Single Object Localization ▻ Object Detection 14

Goog. Le. Net ▰ Deep learning algorithm that was developed by google ▰ In 2014 it was entered into ILSVRC it placed 1 st in the image classification and object identification tasks. ▻ Image Classification- Error Percentage 6. 66% ▻ Object Detection- Average Precision- 43. 93% ▻ Single Object Localization- Error Percentage- 26. 44% 15

Goog. Le. Net Vs Human ▰ Goog. Le. Net trained using 100, 000 images ▰ Annotator 1 (A 1) trained using 500 images and then annotated 1500 images ▰ Annotator 2 (A 2) trained using 100 images and then annotated 258 images 16

Conclusion ▰ A better understanding of how the brain functions will provide valuable information to help further develop deep learning. ▰ Improvements in unsupervised learning will make deep learning more efficient as it removes the process of inputting labelled data ▰ Deep learning has to evolve much like the brain to be as efficient and effective as it. 17