Image SuperResolution Using Deep Convolutional Networks Chao Dong

Image Super-Resolution Using Deep Convolutional Networks Chao Dong, Chen Change Loy, Kaiming He and Xiaoou Tang

Abstract • A deep learning method for single image superresolution (SR) End-to-end mapping between the low/high-resolution images a deep convolutional neural network (CNN) that takes the low-resolution image as the input and outputs the high-resolution – Unlike traditional methods that handle each component separately, our method jointly optimizes all layers. – • A lightweight structure, yet demonstrates state-ofthe-art restoration quality, and fast speed for practical on-line usage. • Extend our network to cope with three color channels simultaneously for better overall reconstruction quality

Introduction • Single image super resolution(SR) is a ill-posed problem – Underdetermined(many HR for a LR) need priors to constrain solution space • Example-based strategy – Exploit internal similarities of the same image – External LR and HR exemplar pairs • Sparse-coding-based (SC) method: one of the representative external example-based SR methods – Pipeline • Overlapping patches are densely cropped from the input image and preprocessed (e. g. , subtracting mean and normalization) • These patches are then encoded by a LR dictionary. • The sparse coefficients are passed into a HR dictionary for reconstructing high resolution patches. • The overlapping reconstructed patches are aggregated (e. g. , by weighted averaging) to produce the final output. – Pay particular attention to learning and optimizing the dictionaries or building efficient mapping functions. However, the rest of the steps in the pipeline have been rarely optimized or considered in an unified optimization framework

Introduction • SRCNN – Entire SR pipeline is fully obtained through learning with little pre/post-processing – The aforementioned pipeline is equivalent to a deep CNN • Learns an end-to-end mapping between low- and high-resolution images • Does not explicitly learn the or manifolds for modeling the patch space, which are implicitly achieved via hidden layers – The patch extraction and aggregation are also formulated as convolutional layers, so are involved in the optimization • Properties – Accurate – Fast – Improved when (i) larger and more diverse datasets are available, and/or (ii) a larger and deeper model is used. – can cope with three channels of color images simultaneously to achieve improved super-resolution performance.

CNN for SR •

Relationship to Sparse-Coding. Based Methods •

Training •

Experiment

The first layer contain different structures (e. g. , edges at different directions), while that of the second layer are mainly different on intensities

Multi-Frame Video Super. Resolution Using Convolutional Neural Networks Alex Greaves and Hanna Winter

• Propose a HR for multi-frame super-resolution that outputs a higher resolution version of a given frame using pixel information from adjacent frames in the video – SRCNN-based SISR(Single Image Super Resolution) for SFSR(Single Frame Super Resolution) – VSR(Video Super Resolution) based on SFSR • Multi-frame CNN is initialized with pre-trained weights from the single-frame CNN. • Receives a set of video frames with 3 color channels and outputs a set of corresponding HR frames. • For each frame of the input, multi-frame model uses information from the frames before and after to construct a HR version • Multi-frame CNN out-performs other modern multiframe SR methods while requiring little preprocessing.

• A class of networks with up to 9 convolutional layers, each with a Re. Lu activation function, and employing dropout for training • Both Momentum and Adam updates, which are two common algorithms to perform parameter updates based on the gradient of the loss function

Experimental Result

• Accurate Image Super-Resolution Using Very Deep Convolutional Networks https: //www. youtube. com/watch? v=OR 1 a. BLL 0 Ev. Q • Deeply-Recursive Convolutional Network for Image Super-Resolution https: //www. youtube. com/watch? v=g. Wt 4 Ckw. Xl. Cc • In Korean, survey on recent SR tech. based on deep learning https: //www. youtube. com/watch? v=1 j. Gr_OFyfa 0