NPR Today Comprehensible Rendering of 3 D Shapes

NPR Today • “Comprehensible Rendering of 3 -D Shapes”, Takafumi Saito and Tokiichiro Takahashi, SIGGRAPH 1990 • “Painterly Rendering with Curved Brush Strokes of Multiple Sizes”, Aaron Hertzmann, SIGGRAPH 98 • “An Algorithm For Automatic Painterly Rendering Based On Local Source Image Approximation”, Michio Shiraishi and Yasushi Yamaguchi, NPAR 2000 03/28/03 © 2005 University of Wisconsin

Comprehensible Rendering of 3 -D Shapes Takafumi Saito and Tokiichiro Takahashi SIGGRAPH 1990 03/15/03

Comprehensible Rendering • Focus on conveying information, rather than realism • Goal: A range of enhancements to “standard” shaded models – Edge enhancement, contour lines, hatching, … 03/15/03 © 2005 University of Wisconsin

G-buffers • Extend color and depth buffers with additional images – – – Objects

Getting G-buffers • Almost any algorithm can be modified to produce G-buffers – Ray-tracing – get info from hit points – Hardware – use programmable shaders • Common technique now, but done here first • PBRT has almost all the required information available 03/15/03 © 2005 University of Wisconsin

Algorithms • Shading • Edges/Silhouettes – Find discontinuities in depth • Hatching – Trace

Finding Edges • Use edge detection filters on the depth image – Look for places with rapid change in depth – Tuning the filters provides control of extracted edges • Then just draw edges, or add back into result 03/15/03 © 2005 University of Wisconsin

Contours and Hatching • Trace lines of constant z to get screen -space contours • Or, use world coordinate image to get world-space contours • Control spacing to get hatching z=xy 03/15/03 © 2005 University of Wisconsin

Shading • World coordinates and normals enable shading • Or, use any procedural shading algorithm • Note meta-information is required – e. g. Materials associated with polygon IDs Shading & contours 03/15/03 © 2005 University of Wisconsin

More Results 03/15/03 © 2005 University of Wisconsin

Limitations • All results to image precision – May lose contour lines, or they may be hard to track – Use floating point images to help resolve this – Use layered images (multiple surfaces at each pixel) • Not all algorithms are amenable to fast hardware implementation 03/15/03 © 2005 University of Wisconsin

Future Work • The ideas from this paper re-emerged in many different algorithms – Generating an image of visible surface IDs was most common (could be done on earliest hardware) • Hardware fragment shaders have made much of this redundant – For example, can easily get contour images or false-color 03/15/03 © 2005 University of Wisconsin

Painterly Rendering with Curved Brush Strokes of Multiple Sizes : Aaron Hertzmann (SIGGRAPH 98

Painterly rendering • Objective: Creating an image with a hand-painted appearance from a photograph

Key Concept • Painters use varying brush sizes to capture detail: © 2005 University

Algorithm equivalent • Procedure: – Build painting one layer at a time, from biggest to smallest radius brushes – At each layer, add detail missing from previous layer • Source image= Original photograph • Reference image= Gaussian blurred image • 3 functions cover this algorithm © 2005 University of Wisconsin

function paint(source. Image, R 1…Rn) © 2005 University of Wisconsin

procedure paint. Layer(canvas, reference. Image, R) • canvas is the image we are updating layer by layer. reference. Image is the Gaussian blurred form of source image Gaussian blurred reference image Eucledean difference of r, g, b distances • Determine grid size for given radius • For each pixel of reference. Image calculate area error for grid size. If it is above a certain value, find pixel within grid with largest difference between canvas and reference. Image – Apply make. Stroke() algorithm for this point and store the stroke returned. • After exiting above ‘for loop’, paint all strokes on the canvas. © 2005 University of Wisconsin

function make. Stroke(x 0, y 0, R, ref. Image) • Generic procedure for placing stroke on canvas • Places a circle of radius R at the location x 0, y 0 • Disadvantage: Circles very visible © 2005 University of Wisconsin

function make. Spline. Stroke(x 0, y 0, R, ref. Image) • make. Spline. Stroke(x 0, y 0, R, ref. Image): Makes image closer to what a typical painting would look like, by: – Placing long, curved brush strokes, thus eliminating a regular uniform stroke appearance – Placing many small brush strokes in high frequency information region, and placing large brush strokes over areas with little detail. © 2005 University of Wisconsin

function make. Spline. Stroke(x 0, y 0, R, ref. Image) • Brush strokes of constant color • Stroke placement normal to image gradient • Stroke stops when – color deviates from control point by more than specified threshold OR – predetermined stroke length is reached • Stroke points are separated by brush radius distance R. • E. g. Stroke direction Image gradient © 2005 University of Wisconsin

function make. Spline. Stroke(x 0, y 0, R, ref. Image) © 2005 University of

Examples • Style can be changed by changing parameters of the algorithm such as

Future Work • No one ‘right’ algorithm. Future work should continue to extend strategies to rendering algorithms. • Brush strokes could be further developed to convey physical as well as semantic elements • Real time rendering where rendering style changes with mood or action © 2005 University of Wisconsin

Questions? ? ? Source: UW Madison- CS 559 Spring ’ 04 Painterly Rendering Assignment

An Algorithm For Automatic Painterly Rendering Based On Local Source Image Approximation Authors Michio

Goals: § Automatic Painterly Rendering § Synthesize an image with a handcrafted look from a source image like a photograph. § Intensive use of local region of source image. § High emphasis on color information.

Previous Work: § Paul Haeberli developed interactive system where users could place strokes on a canvas. § Barbara Meier introduced 3 D model support, painting order is based on depth. § Aaron Hertzmann used brush strokes represented by spline curves. Composite image with several layers of strokes.

Process: § Vary stroke size over the canvas. Details are represented by small strokes, flat areas with larger strokes. § Determine location, orientation, width and length of each stroke to approximate the local region of source image. § Paint strokes on order of size.

Color Difference Image: § One value set by user is s which is the length and width of the local region. § Grayscale image based on how far local region image is from stroke color. § Intensity is larger the closer the image color is to the stroke color. Color image Resulting Gray-scale image Stroke color is white in this example.

Brush Stroke Properties: § Color: The color C in RGB of stroke. § Location: The location of the stroke center (xc, yc). § Orientation: The angle between the longest axis of stroke and x-axis of canvas. § Size: The size of stroke, length l and width w.

How to find these values? : § Image moments are used. Image moments are defined on a gray-scale image. § Image moments of 0 th, 1 st, and 2 nd degree are used. § Use these values to find the length, width, location, and orientation of the stroke. Gray-scale image Resulting Stroke The result is two images with the same image moments.

Stroke Painting Process § Scale texture image to fit length and width of stroke. § Rotate scaled image by θ § Translate scaled and rotated image by (xc, yc). § Paint the stroke with color C by alpha blending.

Algorithm: § Stroke Area Estimation: § Calculate M 00 for each pixel with-in local region s.

Algorithm: § Sampling Points Generation: § Murray space filling polygon to dither image. § For each pixel, 1/M 00 is summed up until it reaches a certain number. § Then that pixel is a sample pixel.

Algorithm: § For each sampling point, § § calculate the color difference image. From color difference image, calculate the stroke values. Paint strokes on white canvas, largest ones first. Larger meaning greater area.

Results: 2500 Strokes Original 5000 Strokes 7500 Strokes s=25, 11048 Strokes 10000 Strokes s=15, 9673 Strokes

Future Work: § Make the algorithm calculate the s value of the local region size. § Calculate the minimum number of strokes to cover entire canvas.