Stylization and Abstraction of Photographs Doug Decarlo and

Stylization and Abstraction of Photographs Doug Decarlo and Anthony Santella

Outline • • Introduction Goals / Motivation Image Structure and Analysis Hierarchical Image Representation Results Video Summary Conclusion

Introduction • The human visual system is very powerful • It masks the complex perceptual and cognitive processing that is needed to understand images • Good information design makes it easer for us to understand images – Keep detail light in unimportant regions – Have fine detail only in important areas

Example • Invert these heuristics and…

Goals • Paper goal: Present a method for stylizing and abstracting photographs to clarify the meaningful information in them with no artistic ability required

Motivation and Contributions • Artists have known about abstraction for years Henri de Toulouse-Lautrec Moulin Rouge-La Goulue

Motivation and Contributions (Cont) • Many different heuristics have been created for emphasizing certain parts of images • But it is very hard to automatically detect meaningful elements in a photograph • Paper’s solution: Use eye movements to help detect the important areas of a photograph

Algorithm Summary • To transform an image: – Instruct a user to look at the image for a short time, obtaining a record of eye movements. – Disassemble the image into its constituents of visual form using visual analysis (image segmentation and edge detection) – Render the image, preserving the form predicted to be meaningful by applying a model of human visual perception to the eye-movement data

Image Structure and Analysis • Edge Detection: The process of extracting out location of high contrast in an image that are likely to form the boundary of objects

Image Structure and Analysis • Image Segmentation: Partitioning an image into contiguous regions of pixels that have similar appearance.

Image Structure and Analysis • For image segmentation, colors were represented in L*u*v space, to produce region boundaries that were more meaningful for human observers

Image Structure and Analysis • Scale-space Theory: Provides a description of images in terms of how content across different resolutions is related • This theory serves as the basis for their hierarchical representation of the image • It uses segmentation algorithms applied at a variety of scales, and finds containment relationships between their results

Visual Perception • Eye movements give a strong indication of the important elements in an image • People can examine only a small area of an image at a time, and therefore scan them in a series of “fixations”

Eye tracker • Use an eye tracking device to figure out the important areas of a photograph

Hierarchical Image Representation • Create an image pyramid of the input image – The bottom image is the original image – Each layer up is a downsampled by a constant factor – A segmentation algorithm is computed at each later • Edges are detected using the original image, a process called Edge Tracking is used to smooth them

Building the Hierarchy • Construct a tree, a leaf is created for each segmentation in the tree’s bottom image

Rendering with a Perceptual Model • To create the line drawings, the hierarchal tree that was created is “pruned” of leaf nodes in areas where it is determined the user didn’t see • Pruning is done based on the fixations that were recorded and perceptual cues such contrast sensitivity

Region Smoothing • The detail level of boundaries is uniformly high, since all boundaries derive from the lowest segmentation

Drawing Lines • After the segments have been drawn, we draw the lines • When drawing the lines, we ignore lines that were not important (using factors like how close they were to a fixation, how long they are, etc)

Results

Results

Video • Video time…

Future Work • Have the segmenter use a model of shading • Find new ways to represent textures

Conclusion • This paper presented a new visual style using bold edges and constant color and a method of interaction using eye tracking that helps find important areas of a photograph for image rendering

Questions? • ? ? ?