Domain Adaptive Faster RCNN for Object Detection in

Domain Adaptive Faster R-CNN for Object Detection in the Wild CVPR 2018 李一帆 2019年 1月16日

Introduction • Object detection in the real world faces challenges from the large variance in viewpoints, object appearance, backgrounds, illumination, image quality, etc. , which may cause a considerable domain shift between the training and test data. • We tackle the domain shift on two levels: 1) the image-level shift, such as image style, illumination, etc. , and 2) the instance-level shift, such as object appearance, size, etc.

Introduction Contribution : • 1) We provide a theoretical analysis of the domain shift problem for cross-domain object detection from a probabilistic perspective. • 2) We design two domain adaptation components to alleviate the domain discrepancy at the image and instance levels, resp. • 3) We further propose a consistency regularization to encourage the RPN to be domain-invariant. • 4)We integrate the proposed components into the Faster R-CNN model, and the resulting system can be trained in an end-to-end manner.

Preliminaries • the baseline model: Faster R-CNN • a two-stage detector mainly consisting of three major components: shared bottom convolutional layers, a region proposal network (RPN) and a region-of-interest (ROI) based classifier.

Preliminaries • The H-divergence is designed to measure the divergence between two sets of samples with different distributions. • Let us denote by x a feature vector. A source domain sample can be denoted as x. S and a target domain sample as x. T. We also denote by h : x→｛ 0, 1｝a domain classifier, which aims to predict the source samples x. S to be 0, and target domain sample x. T to be 1.

Domain Adaptation • Image-Level Adaptation: P(C, B, I) = P(C, B|I)P(I) • I : feature map • B: the bounding-box of an object • C: the category of the object • 根据 Covariate Shift Assumption，条件概率P(C, B|I) 在目标域和源 域应该是一样的，所以只需要对齐 P(I) 即PS(I) = PT (I)。

Domain Adaptation • Instance-Level Adaptation: P(C, B, I) = P(C|B, I)P(B, I) • 同样根据 Covariate Shift Assumption，条件概率P(C|B, I) 在目标域和 源域应该是一样的，所以只需要对齐 P(B, I) 即PS(B, I) = PT (B, I) 。

Domain Adaptation • Joint Adaptation: P(B, I) = P(B|I)P(I) P(B|I)对应的是bounding boxes预测器，即 Faster RCNN中的 RPN(region proposal network) • the bounding box annotation is only available for source domain training data, therefore P(B|I) is learned using the source domain data only, which is easily biased toward the source domain.

Loss Function • image-level adaptation loss： • instance-level adaptation loss：

Loss Function • consistency regularization : • final training loss:

Model • An overview of our Domain Adaptive Faster R-CNN model:

Experiments • From synthetic data to real data • Datasets: SIM 10 k(source domain) Cityscapes(target domain) • Results:

Experiments • From Cityscapes data to Foggy Cityscapes • Datasets: Cityscapes(source domain) Foggy Cityscapes(target domain) • Results:

Experiments • Between two real datasets • Datasets: KITTI Cityscapes • Results:

Thanks!