Structuremeasure A New Way to Evaluate Foreground Maps

Structure-measure: A New Way to Evaluate Foreground Maps 1 1 2 Deng-Ping Fan Ming-Ming Cheng Yun Liu Tao Li Ali Borji 1 Nankai University 2 UCF Framework Introduction Current foreground map evaluation measures are based on pixel-wise errors and often ignore the structural similarities. Behavioral vision studies, however, have shown that the human visual system is highly sensitive to structures in scenes. We propose a novel, efficient, and easy to calculate measure known as structural similarity measure (S-measure) to evaluate non-binary foreground maps. Our new measure simultaneously evaluates region-aware and object-aware structural similarity between a foreground map (FM) and a ground-truth (GT) map. We demonstrate superiority of our measure over existing ones using 5 meta-measures on 5 benchmark datasets. International Conference on Computer Vision 2017 Ø Meta-measure 3: Ground-truth Switch. (a) Image (b) FM (c) GT (d) wrong GT + Pixel-wise based measures (a) MM 2 results (b) MM 3 results Ø Meta-measure 4: Human Judgments. = (a) Toy example (b) Realistic example Motivation Experiments Ø Region-aware: object-parts. : Ø Meta-measure 1: Application Ranking. Viewer preferred the map chosen by our measure. Conclusion In summary, our measure offers new insights into foreground maps evaluation where current measures fail to truly examine the strengths and weaknesses of the models. Establishing that our S-measure offers a better way to evaluate foreground map, we believe that this new measure will benefit the researchers. (a) object-parts (b) region similarity [1] Ø Object-aware: contrast and uniform. : contrast uniform (a) MM 1 results Ø Meta-measure 2: State-of-the-art VS. Generic. (a) FM 1 [1] Image quality assessment from error visibility to structural similarity. IEEE TIP, 2004 Code & You. Tube Video is available at the Project Page: https: //dpfan. net/smeasure/ (b) FM 2 (c) foreground similarity (a) Image (b) GT (c) FM (d) Generic