Image Inpainting Using PreTrained Classification CNN By Yaniv

Image Inpainting Using Pre-Trained Classification CNN By - Yaniv Kerzhner & Adar Elad Supervisor - Dr. Yaniv Romano

The Problem - Image Inpainting The process of reconstructing lost parts in a given image M This project aims to apply convolutional neural networks (CNN) for solving the inpainting problem Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion Our prime goal - Investigate whether a pre -trained CNN classifier (for digits/faces) has memorized the data in order to fill it in 2

CNN – Convolutional Neural Networks Input Output Le-Cun 1% 1% 2% 2% Abbel Hinton Bengio 90% Lena 2% Lin Adar Elad Michael Elad & Yaniv Kerzhner The Computer-Science Department The Technion 3

The straight-forward solution Given a set of training pairs (original and corrupted images), train a network in a supervised fashion Neural Network Adar Elad & Yaniv Kerzhner Michael Elad The Computer-Science Department The Technion 4 ℒ ,

The straight-forward solution Given a set of training pairs (original and corrupted images), train a network in a supervised fashion an Cai. et. al. (2015) Adar Elad & Yaniv Kerzhner Michael Elad The Computer-Science Department The Technion 5

The Proposed Method Pre-trained CNN car house lizard tree Adar Elad & Yaniv Kerzhner Michael Elad The Computer-Science Department The Technion 6

The Proposed Method Pre-trained CNN car house lizard tree Adar Elad & Yaniv Kerzhner Michael Elad The Computer-Science Department The Technion 6

The Proposed Method Pre-trained CNN car house lizard tree Adar Elad & Yaniv Kerzhner Michael Elad The Computer-Science Department The Technion 6

The Proposed Method Pre-trained CNN car house lizard tree Adar Elad & Yaniv Kerzhner Michael Elad The Computer-Science Department The Technion 6

The Proposed Method Pre-trained CNN car house lizard tree Adar Elad & Yaniv Kerzhner Michael Elad The Computer-Science Department The Technion 6

The Proposed Method Adar Elad & Yaniv Kerzhner Michael Elad The Computer-Science Department The Technion 6

Results – MNIST dataset original Masked Our Result Adar Elad & Yaniv Kerzhner Michael Elad The Computer-Science Department The Technion 7

Results – Yale Extended B dataset original Masked Our Result Adar Elad & Yaniv Kerzhner Michael Elad The Computer-Science Department The Technion 8

Results – Image. Net dataset age m i e h t f 90% o g !! n i s s i m s i Masked Our Result Adar Elad & Yaniv Kerzhner Michael Elad The Computer-Science Department The Technion 10

Results – Special Cases • Hallucination - The specific case of image inpainting where all the data is missing 0 3 5 8 • Illumination - Many face images in the dataset were taken in the dark, the question then arises: could these images be illuminated with the assistant of our algorithm? It is possible to distinguish that all the details of the original image still remains after the inpainting. Meaning, all we did is to add information to the dark parts. Michael Elad The Computer-Science Department The Technion input our result

Time to Conclude Michael Elad The Computer-Science Department The Technion

Time to Conclude Michael Elad The Computer-Science Department The Technion