Towards RealTime Texture Synthesis With the Jump Map

Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics Workshop on Rendering (2002)

Texture Synthesis

Neighbourhood-based • Compare local causal neighbourhoods Efros and Leung (ICCV ’ 99) Wei and Levoy (SIGGRAPH 2000) Ashikhmin (I 3 D 2001) Input Output

Fast Texture Synthesis • Goal: Interactivity • Want synthesis that is: • Fast • Simple • High quality

Patch-based Methods • Copy patches of pixels rather than single pixels

Patch-based Methods • Copy patches of pixels rather than single pixels • Chaos Mosaic, Xu et al, 1997

Patch-based Methods • Copy patches of pixels rather than single pixels • Chaos Mosaic, Xu et al, 1997 • Patch-Based Sampling, Liang et al, 2002

Patch-based Methods • Copy patches of pixels rather than single pixels • Chaos Mosaic, Xu et al, 1997 • Patch-Based Sampling, Liang et al, 2002 • Image Quilting, Efros and Williams, 2001

Our Inspiration • Video Textures • . . Schodl et al, SIGGRAPH 2000 • Given a sample video, generate endless video without looping • Generate links between similar frames • Play video, randomly following links

Our Approach • Divide task into two phases: • Analysis • Once per input texture (need not be fast) • Generates jump map • Synthesis • Uses jump map • Fast enough for interactive applications

What is a Jump Map? • Same size as input

What is a Jump Map? • Same size as input • Set of jumps per pixel

What is a Jump Map? • Same size as input • Set of jumps per pixel • Jumps are weighted according to similarity 0. 4 0. 2 • Need not sum to 1 0. 3

Synthesis with Jump Maps Input Output

Synthesis With Jump Maps Input Output

Synthesis Order • Synthesis order influences patch shapes • Not likely to extend in directions where there aren’t alreadysynthesized neighbours

Synthesis Orders • Serpentine • Reverse direction at end of scan-line • Better than scan-line, just as fast • Hilbert curve • Maximizes locality • Much higher quality • Adds some overhead

Synthesis Issues • Artifacts may occur if a patch hits an input image boundary • Modify probability of taking a jump • Increase for jumps from input boundary • Decrease for jumps to input boundary • Blend patch boundaries

Texture Analysis • Need best matches for each input pixel • Pose as high-dimensional ANN problem PCA … … Input Neighbourhood Vectors ANN Vectors

Multi-resolution Analysis • Use image pyramid and multi-resolution neighbourhood vectors • Smaller neighbourhood required • Improves PCA reduction

Jump Map Diversity • Undesirable repitition may occur if jumps cluster spatially • L 2 norm is particularly susceptible

Poisson Disc Sampling 1 6 2 4 3 5 • Find extra matches • Iteratively accept matches satisfying Poisson disc criterion • Include a Poisson disc at the source

Analysis Summary • Need best matches for each input pixel • Use multi-resolution neighbourhoods • Pose as high-dimensional ANN problem • Reduce dimension with PCA • Filter matches with Poisson discs • Normalize similarity values across the jump map

Results • Current implementation: 2. 1 million pixels/second scan-line 0. 8 million pixels/second Hilbert • Good quality on stochastic textures • Not so good on structured textures • TBD: demo

Future Work • Analysis phase: • Use perceptual metrics • Clustering instead of ANN • Synthesis phase • Multi-resolution synthesis • Output control mechanisms

Future Work • Generalization to patches • Reduce storage used • Sample size required?

Contact Information Steve Zelinka zelinka@uiuc. edu Michael Garland garland@uiuc. edu http: //graphics. uiuc. edu/~zelinka/jumpmaps