Enhancedalignment Measure for Binary Foreground Map Evaluation DengPing
Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, Cheng Gong, Yang Cao, Bo Ren, Ming-Ming Cheng, Ali Borji Nankai University http: //mmcheng. net/e-measure
Binary Foreground Map p The binary foreground map consists of values of either 0 or 1. p 1 denotes foreground, 0 for background. 0 (a) Image 1 Enhanced-alignment Measure for Binary Foreground Map Evaluation (b) Binary foreground map Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Application p Object Segmentation, Foreground/background detection, Saliency, etc. (a) Image (b) GT Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
How to Evaluate p Object Segmentation (a) Image (b) GT VS (d) MDF (CVPR’ 15) Enhanced-alignment Measure for Binary Foreground Map Evaluation (f) Noise Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Previous Work p Intersection-over-Union (Io. U) Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Previous Work p Intersection-over-Union (Io. U) A Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Previous Work p Intersection-over-Union (Io. U) B Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Previous Work p Intersection-over-Union (Io. U) A B Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Previous Work p Intersection-over-Union (Io. U) A∩B Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
![Previous Work p Contour Mapping (CM)[1] Movahedi and Elder. Design and perceptual validation of](http://slidetodoc.com/presentation_image_h2/9c079618731cda296b1af6d7972fcc85/image-10.jpg "Previous Work p Contour Mapping (CM)[1] Movahedi and Elder. Design and perceptual validation of")
Previous Work p Contour Mapping (CM)[1] Movahedi and Elder. Design and perceptual validation of performance measures for salient object segmentation. CVPRW, 2010. Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
![Previous Work p Contour Mapping (CM)[1] Movahedi and Elder. Design and perceptual validation of](http://slidetodoc.com/presentation_image_h2/9c079618731cda296b1af6d7972fcc85/image-11.jpg "Previous Work p Contour Mapping (CM)[1] Movahedi and Elder. Design and perceptual validation of")
Previous Work p Contour Mapping (CM)[1] Movahedi and Elder. Design and perceptual validation of performance measures for salient object segmentation. CVPRW, 2010. Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
![Previous Work [1] Movahedi and Elder. Design and perceptual validation of performance measures for](http://slidetodoc.com/presentation_image_h2/9c079618731cda296b1af6d7972fcc85/image-12.jpg "Previous Work [1] Movahedi and Elder. Design and perceptual validation of performance measures for")
Previous Work [1] Movahedi and Elder. Design and perceptual validation of performance measures for salient object segmentation. CVPRW, 2010. [2] Margolin et al. How to evaluate foreground maps? CVPR, 2014. Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
![Previous Work [1] Movahedi and Elder. Design and perceptual validation of performance measures for](http://slidetodoc.com/presentation_image_h2/9c079618731cda296b1af6d7972fcc85/image-13.jpg "Previous Work [1] Movahedi and Elder. Design and perceptual validation of performance measures for")
Previous Work [1] Movahedi and Elder. Design and perceptual validation of performance measures for salient object segmentation. CVPRW, 2010. [2] Margolin et al. How to evaluate foreground maps? CVPR, 2014. [3] Shi et al. Visual quality evaluation of image object segmentation: Subjective assessment and objective measure. TIP, 2015. Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
![Previous Work [1] Movahedi and Elder. Design and perceptual validation of performance measures for](http://slidetodoc.com/presentation_image_h2/9c079618731cda296b1af6d7972fcc85/image-14.jpg "Previous Work [1] Movahedi and Elder. Design and perceptual validation of performance measures for")
Previous Work [1] Movahedi and Elder. Design and perceptual validation of performance measures for salient object segmentation. CVPRW, 2010. [2] Margolin et al. How to evaluate foreground maps? CVPR, 2014. [3] Shi et al. Visual quality evaluation of image object segmentation: Subjective assessment and objective measure. TIP, 2015. [4] Fan et al. Structure-measure: A New Way to Evaluate Foreground Maps. ICCV, 2017. Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Problem (a) Image (b) GT (c) Foreground map Enhanced-alignment Measure for Binary Foreground Map Evaluation (d) Noise Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Problem > (a) GT (b) Noise (c) Foreground map p Almost all of current measure (e. g. , Io. U, CM, Fbw, VQ) prefer the Noise map. Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Problem > (a) GT (b) Noise (c) Foreground map p Almost all of current measure (e. g. , Io. U, CM, Fbw, VQ) prefer the Noise map. p They are either edge-based(local details) or region-based (global information). Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Problem > (a) GT (b) Noise (c) Foreground map p Almost all of current measure (e. g. , Io. U, CM, Fbw, VQ) prefer the Noise map. p They are either edge-based(local details) or region-based (global information). p None of them consider both local and global simultaneously. Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Motivation 1. Global information can be captured by the eye movement. COGNIVISION 2. Local details recorded by focusing the special image region. http: //knowledgelotus. info/human-brain-facts/ Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Example 1. Global information (a) Image (b) GT (c) Noise Enhanced-alignment Measure for Binary Foreground Map Evaluation (d) Map 1 Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Example 1. Global information (a) Image (b) GT (c) Noise Enhanced-alignment Measure for Binary Foreground Map Evaluation (d) Map 1 Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Example 1. Global information (a) Image (b) GT > (d) Map 1 (c) Noise Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Example 1. Global information & 2. Local Details (a) Image (b) GT (c) Map 1 Enhanced-alignment Measure for Binary Foreground Map Evaluation (d) Map 2 Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Example 1. Global information & 2. Local Details (a) Image (b) GT (c) Map 1 Enhanced-alignment Measure for Binary Foreground Map Evaluation (d) Map 2 Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Example 1. Global information & 2. Local Details (a) Image (b) GT (c) Map 1 Enhanced-alignment Measure for Binary Foreground Map Evaluation (d) Map 2 Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Example 1. Global information & 2. Local Details (a) Image (b) GT > (c) Map 2 (d) Map 1 Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Enhanced-alignment measure p Enhanced-alignment measure = alignment term with enhanced function = Alignment term Enhanced-alignment Measure for Binary Foreground Map Evaluation + Enhanced function Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
E-measure: Alignment term 1. Global information Firstly, we compute the global mean of the input map to capture global information. Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
E-measure: Alignment term 1. Global information p Easy, fast to implement and use (b) Global mean Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
E-measure: Alignment term 1. Global information Firstly, we compute the global mean of the input map to capture global information. (b) Global mean Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
E-measure: Alignment term 1. Global information Firstly, we compute the global mean of the input map to capture global information. (b) Global mean Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
E-measure: Alignment term 1. Global information Firstly, we compute the global mean of the input map to capture global information. (b) Global mean 3. Combine global information with local details Introducing a bias matrix which can be treated as the signal centering by removing the mean from the signal. Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
E-measure: Alignment term 1. Global information Firstly, we compute the global mean of the input map to capture global information. (b) Global mean 3. Combine global information with local details Introducing a bias matrix which can be treated as the signal centering by removing the mean from the signal. (c) Bias matrix Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
E-measure: Alignment term 1. Global information Firstly, we compute the global mean of the input map to capture global information. (b) Global mean 3. Combine global information with local details Introducing a bias matrix which can be treated as the signal centering by removing the mean from the signal. (c) Bias matrix Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
E-measure: Alignment term Bias matrix Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
![E-measure: Alignment term 4. Alignment matrix [1][2] [1] Fan et al. Structure-measure: A New](http://slidetodoc.com/presentation_image_h2/9c079618731cda296b1af6d7972fcc85/image-36.jpg "E-measure: Alignment term 4. Alignment matrix [1][2] [1] Fan et al. Structure-measure: A New")
E-measure: Alignment term 4. Alignment matrix [1][2] [1] Fan et al. Structure-measure: A New Way to Evaluate Foreground Maps. ICCV, 2017. [2] Wang et al. Image quality assessment: from error visibility to structural similarity. TIP, 2004 Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
![E-measure: Alignment term 4. Alignment matrix [1][2] [1] Fan et al. Structure-measure: A New](http://slidetodoc.com/presentation_image_h2/9c079618731cda296b1af6d7972fcc85/image-37.jpg "E-measure: Alignment term 4. Alignment matrix [1][2] [1] Fan et al. Structure-measure: A New")
E-measure: Alignment term 4. Alignment matrix [1][2] [1] Fan et al. Structure-measure: A New Way to Evaluate Foreground Maps. ICCV, 2017. [2] Wang et al. Image quality assessment: from error visibility to structural similarity. TIP, 2004 Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
![E-measure: Alignment term 4. Alignment matrix [1] Fan et al. Structure-measure: A New Way](http://slidetodoc.com/presentation_image_h2/9c079618731cda296b1af6d7972fcc85/image-38.jpg "E-measure: Alignment term 4. Alignment matrix [1] Fan et al. Structure-measure: A New Way")
E-measure: Alignment term 4. Alignment matrix [1] Fan et al. Structure-measure: A New Way to Evaluate Foreground Maps. ICCV, 2017. [2] Wang et al. Image quality assessment: from error visibility to structural similarity. TIP, 2004 Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
![E-measure: Alignment term 4. Alignment matrix [1] Fan et al. Structure-measure: A New Way](http://slidetodoc.com/presentation_image_h2/9c079618731cda296b1af6d7972fcc85/image-39.jpg "E-measure: Alignment term 4. Alignment matrix [1] Fan et al. Structure-measure: A New Way")
E-measure: Alignment term 4. Alignment matrix [1] Fan et al. Structure-measure: A New Way to Evaluate Foreground Maps. ICCV, 2017. [2] Wang et al. Image quality assessment: from error visibility to structural similarity. TIP, 2004 Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
![E-measure: Alignment term 4. Alignment matrix [1] Fan et al. Structure-measure: A New Way](http://slidetodoc.com/presentation_image_h2/9c079618731cda296b1af6d7972fcc85/image-40.jpg "E-measure: Alignment term 4. Alignment matrix [1] Fan et al. Structure-measure: A New Way")
E-measure: Alignment term 4. Alignment matrix [1] Fan et al. Structure-measure: A New Way to Evaluate Foreground Maps. ICCV, 2017. [2] Wang et al. Image quality assessment: from error visibility to structural similarity. TIP, 2004 Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
E-measure: Enhanced function 5. Enhanced alignment matrix Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Experiments Meta-Measure 1: Application Ranking VS Measure ranking Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Experiments Meta-Measure 2: SOTA vs. Generic Maps Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Experiments Meta-Measure 2: SOTA vs. Generic Maps Meta-Measure 3: SOTA vs. Random Noise Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Experiments Meta-Measure 4: Human Ranking Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Results Salient Object Segmentation Results on 4 popular datasets 9. 08%-19. 65% improvement. Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Example Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
Conclusion p Evaluation measure p Reliable p Good speed p Intuitive p Easy to use Code & dataset: http: //dpfan. net/e-measure Enhanced-alignment Measure for Binary Foreground Map Evaluation Deng-Ping Fan, http: //dpfan. net/, 2018/7/7
- Slides: 48