Fast Texture Synthesis using Treestructured Vector Quantization LiYi

Fast Texture Synthesis using Tree-structured Vector Quantization Li-Yi Wei Marc Levoy Computer Graphics Group Stanford University

Introduction Texture Synthesis Input Result

Desirable Properties • • • Result looks like the input Efficient General Easy to use Extensible

Previous Work • Procedural Synthesis – Perlin 85, Witkin 91, Worley 96 • Statistical Feature Matching – Heeger 95, De Bonet 97, Simoncelli 98 • Markov Random Fields – Popat 93, Efros 99

Outline • • Basic algorithm Multi-resolution algorithm Acceleration Applications

Texture Model • Textures are – local – stationary • Model textures by – local spatial neighborhoods

Basic Algorithm • Exhaustively search neighborhoods

Neighborhood • Use causal neighborhoods Noise Input Causal Non-causal

Neighborhood • Neighborhood size determines the quality & cost 3 3 5 5 7 7 423 s 528 s 739 s 9 9 1020 s 11 11 41 41 1445 s 24350 s

Multi-resolution Pyramid High resolution Low resolution

Multi-resolution Algorithm

Benefit • Better image quality & faster computation 1 level 5 5 1 level 11 11 3 levels 5 5

Results Random Regular Oriented Semi-regular

Failures • Non-planar structures • Global information

Comparison Input 12 secs Heeger 95 De Bonet 97 Efros 99 1941 secs Our method 503 secs

Acceleration • Computation bottleneck: neighborhood search

Nearest Point Search • Treat neighborhoods as high dimensional points Neighborhood 1 2 3 4 5 6 7 8 9 10 11 12 High dimensional point/vector 1 2 3 4 5 6 7 8 9 10 11 12

Acceleration • Nearest point search in high dimensions – [Nene 97] • Cluster-based model for textures – [Popat 93] • Tree-structured Vector Quantization – [Gersho 92]

Tree-structured Vector Quantization

Timing • Time complexity : O(log N) instead of O(N) – 2 orders of magnitude speedup for non-trivial images Efros 99 Full searching 1941 secs 503 secs TSVQ 12 secs

Results: Brodatz Textures D 103 D 20 Input Exhaustive: 360 secs TSVQ: 7. 5 secs

Application 1: Constrained Synthesis ?

Possible Solution • Multi-resolution blending [Burt & Adelson 83] – produce visible boundaries

Possible Solution • Original raster-scan algorithm – discontinuities at right and bottom boundaries

Possible Solution • Adaptive neighborhoods [Efros 99] – Hard to accelerate

Modifications • Need to use a single symmetric neighborhood • 2 pass algorithm with extrapolation • Spiral order synthesis

Result

Result • Extrapolation ? ?

Result • Image editing by texture replacement

Application 2: Temporal Texture • Indeterminate motions both in space and time – fire, smoke, ocean waves • How to synthesize? – extend our 2 D algorithm to 3 D

Temporal Texture Input Result Fire Smoke Waves

Future Work • More general “textures” – light fields, solid textures – motion signals – displacement maps • Real time texture synthesis

Acknowledgment • • Kris Popat Alyosha Efros Stanford Graphics Group Intel, Interval, Sony More information http: //graphics. stanford. edu/projects/texture/