Texture Synthesis from Multiple Sources LiYi Wei Stanford

Texture Synthesis from Multiple Sources Li-Yi Wei Stanford University (was) NVIDIA Corporation (now)

Texture Synthesis (Single-Source) Tong et. al. 2002 Soler et. al. 2002 Hertzmann et. al. 2001 Efros & Freeman 2001 Yin et. al. 2001 Turk 2001 Wei & Levoy 2000 Portilla & Simoncelli 1999 Efros & Leung 1999 Heeger & Bergen 1995 De Bonet 1997 Input Synthesis Output

Limitations of Single-Source Synthesis • Non-uniform, varying patterns – Junction of 2 textures – Varying scale, orientation, color, shape – Creating new textures • Textures of different dimensions – Solid textures from 2 D views

Multiple-Source Texture Synthesis • Texture mixtures from multiple sources • Solid textures from multiple 2 D views View 1 View 3 + Source 1 = ? Source 2 View 2

Previous Work • Solid texture from 2 D views – Heeger & Bergen 1995 – Ghazanfarpour & Dischler 1999 2 D Source 3 D result Images from [Heeger&Bergen 1995]

Previous Work • Texture varying, morphing, and mixture – – Portilla & Simoncelli 1999 Bar-Joseph, El-Yaniv, Lichinski, Werman 2001 Z. Liu, C. Liu, Shum, Yu 2003 PVT paper in SIGGRAPH 2003 Source 1 morphing Source 2 Image from [Portilla&Simoncelli 1999]

Algorithm Single-source algorithm [ICCV 99, SIGGRAPH 2000] N(pi) search copy N(p) p Source Synthesis Result

Algorithm User weights Source 1 L 2 distance E(p, {pi}) = Σ wi ×(|p-pi|2 + |Ni(p)-Ni(pi)|2) N 1(p 1) i= 1, 2, 3 search Fix p, search {pi} Fix {pi}, set p = Σ wi×pi search Source 2 N 2(p 2) N 3(p 3) Average search Source 3 Synthesis Result

Algorithm User weights Source 1 E(p, {pi}) = Σ wi ×(|p-pi|2 + |Ni(p)-Ni(pi)|2) i= 1, 2, 3 search Source 2 L 2 distance Fix p, search {pi} Fix {pi}, set p = Σ wi×pi search Average search Source 3 Synthesis Result

Texture Mixture Example Use weights wi to control the result + Source 1 = Uniform Source 2 Transition

Texture Mixture Results Source 1 Source 2 Mixture-uniform Mixture-transition

Texture Mixture Results Source 1 Source 2 Mixture-uniform Mixture-transition

Solid Texture Synthesis View 1 N 3 View 2 View 3 N 2 Use {Ni} with different orientations

Specifying Views × Inconsistent О Input О Views Result

Solid Texture Results

Solid Texture Results

Comparison : Surface/Volume Synthesis Surface Volume [SIGGRAPH 2001] [this work]

Conclusion • Single-source synthesis – mostly mature and well done – limited capability • Multiple-source synthesis – hard (patch-copying won’t work!) – more useful – requires more work!

Future Work • Combine the texton mask idea in SIGGRAPH 2003 PVT paper with this algorithm?

END

Algorithm Goal : Minimize E(p, {pi}) = Σ wi ×(|p-pi|2 + |Ni(p)-Ni(pi)|2) Step 2 Step 1 How : iterate 1. Fix p, search {pi} to minimize |Ni(p)-Ni(pi)|2 2. Fix {pi}, set p = Σ wi×pi

Comparison Reusable Distortion Surface Volume better Quality Efficiency Tunable better