Single Image Dehazing using Ranking Convolutional Neural Network

Single Image Dehazing using Ranking Convolutional Neural Network Source: IEEE Transactions on Multimedia, Vol. 20, No. 6, pp. 1548 -1560, June 2018. Authors: Yafei Song, Jia Li, Xiaogang Wang, and Xiaowu Chen Speaker: Hsin-Yu Lee Date: 2018. 8. 30

Outline • Introduction • Preliminaries • Proposed scheme • Experiment results • Conclusions 2

Introduction(1/2) = Original image (Haze image) + Result (Haze-free image) Transmission map Atmospheric light 3

Introduction(2/2) • Statistical attributes • Structural attributes (CNN) Hazy image (Ranking-CNN) Dehazed image 4

Preliminaries (1/5) – CNN(Convolutional Neural Network) Input Image Dog Cat Bird 0. 94% 0. 05% 0. 01% Convolution Pooling Fully Connected*2 Output Predictions Can repeat many times (features/ properties extractor) (subsampling) (classification) 5

Preliminaries (2/5) – CNN(Convolutional Neural Network) • Convolution Layer stride=1 0 0 0 0 0 0 0 1 0 0 0 1 1 1 0 0 0 0 0 Input Image (7*7) 0 0 1 1 0 0 0 1 Filter (3*3) 1 0 0 0 0 1 1 1 0 1 2 1 1 4 2 1 0 0 0 1 2 1 Feature Map (5*5) 6

Preliminaries (3/5) – CNN(Convolutional Neural Network) • Pooling Layer (Max Pooling) stride=2 0 1 0 0 1 1 1 0 1 0 1 2 1 4 2 1 0 0 1 2 1 Feature Map (5*5) Pooled Feature Map (3*3) 7

Preliminaries (4/5) – CNN(Convolutional Neural Network) • Fully Connected Layer 1 1 0 4 2 1 0 2 1 Flattening Dog 0 4 Cat 2 1 Bird 0 Pooled Feature Map (3*3) 2 1 Fully Connected Feedforward network • Activation function (Re. LU) 8

Preliminaries (5/5) – Random forest Temperature Bad Good ? >7% <5% Bad Good Decision tree Bad Random forest 9

Proposed scheme(1/5) Ranking-CNN Estimated transmission Haze-relevant features Random forest regression Atmosphere light estimation Hazy image Refined transmission Guided filter Single image dehazing Atmosphere light Dehazed image 10

Proposed scheme(2/5) - the structure of the Ranking-CNN • Synthesized C P R C C P F F F C : convolution P : max pooling R : ranking F : fully-connected 11

Proposed scheme(3/5) - Ranking-CNN Filter (3*3) 0 0 -1 0 0 0 1 0 -1 0 0 1 4 5 1 6 8 6 2 3 8 7 5 2 7 9 3 1 3 4 8 7 Ranking 1 2 3 3 4 5 6 7 8 9 6 7 8 8 7 12

Proposed scheme(4/5) Optimization • Training Input X Output f(X) • Label Loss(Label, Output) • Loss function (0, 0. 1] (0. 1, 0. 2] (0. 2, 0. 3] (Label, Output) … (0. 9, 1] . . 13

Proposed scheme(5/5) • Predict transmission t • Image dehazing Haze-relevant feature • Adjust luminance • Random forest (200 trees) 14

Experiment results (1/6) • Datatset-Syn Input He et al. Tang et al. Zhu et al. Our Ground truth 15

Experiment results (2/6) • Datatset-Cap Input Berman et al. [2] Ren et al. [32] Cai et al. [4] Our Ground truth [2] D. Berman, T. Treibitz, and S. Avidan, “Non-local image dehazing, ” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , pp. 1674– 1682, 2016. [32] W. Ren et al. , “Single image dehazing via multi-scale convolutional neural networks, ” in Proc. Eur. Conf. Comput. Vis. , Part II, pp. 154– 169, 2016. [4]B. Cai, X. Xu, K. Jia, C. Qing, and D. Tao, “Dehazenet: An end-to-end system for single image haze removal, ” IEEE Trans. Image Process. , vol. 25, no. 11, pp. 5187– 5198, 2016. 16

Experiment results (3/6) • 17

Experiment results (4/6) 18

Experiment results (5/6) • Performance analysis • Feature comparison 325 D features 64 D features (CNN) 325 D features 64 D features (Regression/Random forest) (Ranking-CNN) • The location of the ranking layer 19

Experiment results (6/6) • The size of training data • The convergence speed 20

Conclusions • Structural and statistical features • Accuracy • Efficiency • Feature extracting process Ranking-CNN 283 s CNN 247 s 21