Polarizationbased Inverse Rendering from Single View Daisuke Miyazaki

Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

Modeling cultural assets Geometrical Photometrical Environmental Integrated framework for obtaining 3 types of information 2

Related work Geometry Photometry Environment Tominaga et. al. 2000 Zheng et. al. 1991 Nayar et. al. 1996 Sato et. al. 1999 Ramamoorthi et. al. 2001 Nishino et. al. 2001 Hara et. al. 2002 Proposed method 3

Outline 1. Reflection components separation 2. Shape from polarization using diffuse light 3. Light source estimation from intensity peak 2 4. Reflection parameters estimation by l. s. m. Minimize Ks, σ rendered image real image 4

1. Reflection components separation

Dichromatic reflection model Incident light Surface normal Specularly reflected light Diffusely reflected light Air Object 6

Reflection components separation [Tan 2002] Input Diffuse • Shape Specular • Illumination • Reflection parameters 7

2. Shape from polarization

Related work Object Koshikawa 1979 Opaque Wolff 1990 Opaque Rahmann et. al. 2001 Opaque Miyazaki et. al. 2002 Proposed method Reflection Specular Diffuse Transparent Specular Opaque Diffuse View 1 2 2~5 2 1 9

Polarization Incident light Specularly reflected light Diffusely reflected light Air Object 10

Surface normal Su ce rfa Camera al Zen rm no Polarizer ngle ith a q Azimuth angle Object 11

Azimuth angleφ and intensity difference 255 Imax Intensity Imin 0 1 2 Rotation angle 360 of polarizer Azimuth angle -ambiguity 12

Propagation object Determination of azimuth angle Propagate φ from occluding boundary to inner part of object area (Assumption: smooth surface) [Ikeuchi&Horn 1981] Cannot apply to “dimples”(=perfect concave) 13

Zenith angleθ and DOPρ 1 DOP ρ Degree Of Polarization ρ 0 θ Zenith angle θ 90° 14

Modification 0. 5 Definition of DOP: Modified DOP: DOP ρ Degree Of Polarization 0 u: modification factor • Raises DOP • Assumption • Closed smooth object • “u” is constant u Zenith angle 90° θ 15

Surface normal φ θ Surface normal 16

Height • Relaxation method [Ikeuchi 1984] Minimize: Surface normal where, Gradient Height H Iteratively update: 17

3. Illumination estimation

Illumination sphere Light source is represented in polar coordinate system (θ, φ) θ=0° L 2=(θ 2, φ2) L 1=(θ 1, φ1) L 3=(θ 3, φ3) φ=90° θ=90° φ=180° θ=90° φ=0° Object φ=270° θ=180° 19

Illumination estimation Detect position of intensity peak Determine light source orientation from the peak 1. Project to (θ, φ)-space 2. Thresholding 3. Detect intensity peak 20

4. Reflection parameters estimation

Torrance-Sparrow reflection model Surface Incident normal light Bisector α θi θr View Diffuse reflection Object surface Known: θi, θr, α Specular reflection Unknown: • Diffuse reflection scale; Kd • Specular reflection scale; Ks • Surface roughness; σ 22

Reflection parameters estimation Solve the following least-square problem by steepest-descent method 2 Minimize K s, σ rendered image real image 23

Experimental result

Input Azimuth angleφ Intensity I DOPρ 25

Result of shape estimation 26

Result of illumination estimation Actual illumination distribution Estimated illumination distribution 27

Rendering result Input Synthesized image Rendered image under different illumination & view 28

Result for another object Input Synthesized image Estimated shape Rendered image under different illumination & view 29

Conclusions • Estimated geometrical, photometrical, environmental information in one integrated framework – Shape from polarization – Surface reflection parameters from iterative computation – Illumination from intensity peak 30

Application to digital archiving project • Multiple View • Modeling a statue in a room – IBR with • surface normal • reflection parameters Photorealistic preservation 31

Fin

Daisuke Miyazaki 2003 Creative Commons Attribution 4. 0 International License. http: //www. cvl. iis. u-tokyo. ac. jp/ D. Miyazaki, R. T. Tan, K. Hara, K. Ikeuchi, "Polarization-based Inverse Rendering from Single View, " in Proceedings of International Symposium on the CREST Digital Archiving Project, pp. 51 -65, Tokyo, Japan, 2003. 05