Dual SuperResolution Learning for Semantic Segmentation CVPR 2020

Dual Super-Resolution Learning for Semantic Segmentation CVPR 2020 Poster 2020/8/21 1

Motivation （1）For semantic segmentation task, applying highresolution input could help with a better performance. （2）This strategy brings large computation budgets and limits their applications on resource-constrained devices. 2

Main Contribution （1）A dual super-resolution learning framework is proposed to keep highresolution representation, which can improve the performance while keeping the inference speed. （2）The generality of the proposed framework is validated, which can be readily extended to other tasks requiring high-resolution representation, like human pose estimation. （3）The effectiveness of the method has been demonstrated both on semantic segmentation and human pose estimation. 3

Network Architecture 4

Semantic Segmentation Super-Resolution Output size is 2× than the input image, Instead of re-scale in the post -processing stage 5

Single Image Super-Resolution SISR aims to build a high-resolution image from the low-resolution input 6

Single Image Super-Resolution 7

Feature Affinity Learning 8

Optimization 9

Experiments Task 1: Semantic segmentation 1. Datasets: （1）City. Scapes: 19 classes （2）Cam. Vid: 11 classes 2. Evaluation Metric Mean Intersection over Union (m. Io. U) 10

Ablation Study 1. Effect of algorithmic components 11

Ablation Study 2. Effect of various input resolutions 12

Ablation Study 13

Results on City. Scapes Dataset 14

Results on Cam. Vid testset 15

Experiments Task 2: Human pose estimation 1. Datasets: MS COCO 2. Evaluation Metric Average precision (AP) Average Recall (AR) 16

Results on MS COCO validation set 17

Conclusion （1）Utilize the knowledge of image super-resolution to facilitate semantic segmentation and pose estimation. （2）It would be better to evaluate the performance of image superresolution. 18