All images are compressed Computer Graphics Laboratory Hiroshima

All images are compressed. Computer Graphics Laboratory, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Interactive shadow removal from a single image using hierarchical graph cut Daisuke Miyazaki Yasuyuki Matsushita Katsushi Ikeuchi Computer Graphics Laboratory, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Motivation CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Prior work multiple Weiss 01; Matsushita et al. 04 single automatic Finlayson et al. 04; Fredembach & Finlayson 06; Tappen et al. 05; Baba et al. 04; Arbel & Hel-Or 07; Nielsen & Madsen 07 interactive Wu et al. 07 CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Contribution Robust Smoothness Prior Graph cut Alpha expansion Fast Intuitive User satisfaction CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Algorithm flow 1 2 Input 3 Over-segmentation Lazy snapping Shadow removal Initial 4 Interaction 5 Output 6 CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Shadow matte Solve Prior of shadowless image Prior of shadow image Smoothness of shadowless image Smoothness of shadow image CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Prior of shadowless image Likelihood P small appropriate large CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Prior of shadow image L 2 -Norm small L 1 -Norm appropriate L 0. 7 -Norm large CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Smoothness of shadowless image small appropriate large CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Smoothness of shadow image small appropriate large CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Interactive parameter optimization Initial 1 st 2 nd 3 rd 4 th 5 th 6 th Init 0. 25 0. 7 0 0. 4 Prior shadowless 0. 25 Prior shadow 0 Smoothness shadowless 1 0 Smoothness shadow CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Approach based on alpha expansion Alpha expansion 1 5 Hierarchical graph cut 1 5 3 1 5 2 alpha=2 3 2 1 1 alpha=4 3 2 4 1 5 2 4 3 5 1 2 alpha=2, 4, . . . 4 5 3 4 5 CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Hierarchical structure 20 32 16 16 48 24 40 48 8 16 24 32 40 48 56 16 8 32 32 48 56 8 16 24 32 40 48 56 CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Graph construction example a p a q b CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Algorithm Alpha expansion 1. A = {0, 1, 2, . . . , 63} 2. for i = 0 to 63 do a. for all nodes&edges do I. II. add nodes&edges using & b. solve max-flow/min-cut problem 3. Iterate 2 until convergence Hierarchical graph cut 1. A = {{0}, {32}, {16}, {48}, {8, 40}, {24, 56}, {4, 20, 36, 52}, {12, 28, 44, 60}, . . . , {3, 7, 11, . . . , 63}} 2. for i = 0 to 11 do a. for all nodes&edges do I. II. add nodes&edges using & b. solve max-flow/min-cut problem 3. Iterate 2 until convergence CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Output of Hierarchical graph cut Stereo matching [Art] Stereo matching [Laundry] Image restoration Shadow removal Input Ground truth Ishikawa 2003 a expansion Hierarchical cut CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Speed of hierarchical graph cut Stereo matching [Art] Stereo matching [Laundry] Image restoration Shadow removal Speed-up 6. 8 11. 4 16. 6 3. 4 Error difference +4. 6% +3. 0% -0. 4% +0. 0% Hierarchical graph cut Alpha expansion 1. 1 107 1. 7 106 Energy 0 Energy [sec] 250 1. 5 106 Iteration 7 CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Incremental improvement Input Truth Finlayson 2004 Wu 2007 Initial Interactive [Ours] Finlayson 2004 Error Wu 2007 Ours Number of strokes for parameter optimization 50 CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Natural images 0 strokes (param. opt. ) 1. 4 sec 1 strokes 1. 2 sec 4 strokes 2. 1 sec 6 strokes CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Natural images 1. 5 sec 8 strokes 2. 0 sec 12 strokes 1. 9 sec 15 strokes 1. 5 sec 2 strokes CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Natural images 2. 0 sec 26 strokes 5. 8 sec 27 strokes 6. 3 sec 32 strokes 2. 8 sec 40 strokes CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Aerial images CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Conclusions Cost function User interaction Hierarchical graph cut CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Discussions Cost function User interaction Hierarchical graph cut CG Lab, Hiroshima City University http: //www. hiroshima-cu. ac. jp/

Daisuke Miyazaki 2009 Creative Commons Attribution 4. 0 International License. http: //www. cg. info. hiroshima-cu. ac. jp/ Daisuke Miyazaki, Yasuyuki Matsushita, Katsushi Ikeuchi, "Interactive shadow removal from a single image using hierarchical graph cut, " Lecture Notes in Computer Science (Proc. Asian Conference on Computer Vision), vol. 5994, part 1, 2009. Computer Graphics Laboratory, Hiroshima City University http: //www. hiroshima-cu. ac. jp/