NPR Today RealTime Hatching Emil Praun Hugues Hoppe

NPR Today • “Real–Time Hatching”, Emil Praun, Hugues Hoppe, Matthew Webb and Adam Finkelstein, SIGGRAPH 2001 • “Hijack. GL: Reconstructing Streams for Stylized Rendering”, Alex Mohr and Michael Gleicher, NPAR 2002 • “WYSIWYG NPR: Drawing Strokes Directly on 3 D Models”, Robert D. Kalnins, Lee Markosian, Barbara J. Meier, Michael A. Kowalski, Joseph C. Lee, Phillip L. Davidson Matthew Webb, John F. Hughes and Adam Finkelstein, SIGGRAPH 2002 04/06/05 © 2005 University of Wisconsin

Real–Time Hatching Emil Praun, Hugues Hoppe, Matthew Webb Adam Finkelstein Presented by Kael Greco 04/06/05 © 2005 University of Wisconsin

Hatching Groups of strokes with spatially coherent direction and quality Hatching conveys: • Lighting • Material Properties/Surface Texture • Shape/Volume 04/06/05 © 2005 University of Wisconsin

Goal Rendering of 3 D models in a real-time nonphoto realistic style based on hatching 04/06/05 © 2005 University of Wisconsin

Previous Work Real-Time Hatching – Screen-space “filter” [Lake et al. 2000] – Fixed density strokes [Elber ’ 99] 04/06/05 © 2005 University of Wisconsin

Previous Work tone 04/06/05 • Art Maps [Klein et al. 2000] scale • Prioritized Stroke Textures [Salisbury et al. ’ 94] [Winkenbach et al. ’ 94] © 2005 University of Wisconsin

Overall Approach Set of textures Example stroke Result Mesh 04/06/05 Preprocess Real-Time © 2005 University of Wisconsin

Tonal Art Maps scale • Collection of stroke images • Will blend (design with high coherence) • Stroke nesting property tone 04/06/05 © 2005 University of Wisconsin

Generating Tonal Art Maps Set of textures Example stroke Result Mesh 04/06/05 Preprocess Real-Time © 2005 University of Wisconsin

Generating Tonal Art Maps • Draw or import bitmap for one stroke • Automatically fill TAM with strokes – When placing stroke in an image, add it to all finer & darker images – Fill table column by column, coarse to fine – Space strokes evenly 04/06/05 © 2005 University of Wisconsin

Even Spacing of Strokes • Choose best stroke from large candidate pool • Fitness = uniformity & progress towards tone 1 TAM column (same tone) • Keep Gaussian pyramid forof all TAM images © 2005 University Wisconsin 04/06/05

Even Spacing of Strokes For each candidate stroke si: For the hatch image at each level l of the TAM that would receive si Find cumulative effect of adding si at all levels p of the pyramid. Tpl = average tone of hatch image drawn with all previous strokes Tipl=average tone with si added Normalize “goodness” by stroke length Darkness that this stroke would add to all the hatch images in this column 04/06/05 © 2005 University of Wisconsin

Rendering Set of textures Example stroke Result Mesh 04/06/05 Preprocess Real-Time © 2005 University of Wisconsin

Continuity • Stroke size continuity maintained by mipmapping • Tone continuity maintained by blending textures – Spatial continuity: same contribution for a texture on both sides of an edge Spatial Discontinuity – Temporal continuity: no “popping” 04/06/05 © 2005 University of Wisconsin

Texture Blending Tone v 1 v 2 6 -Way Blend Final 04/06/05 © 2005 University of Wisconsin v 3

Texture Blending • Pack grayscale tones in R, G, B channels → 6 tones in 2 textures • Use multitexture engine → single-pass 6 -way blend • Vertex programs compute blend weights → static vertex data !!VP 1. 0 #Vertex Program for Real-Time Hatching. //output vertex homogeneous coordinates DP 4 R 2. x, c[0], v[OPOS]; DP 4 R 2. y, c[1], v[OPOS]; DP 4 R 2. z, c[2], v[OPOS]; DP 4 R 2. w, c[3], v[OPOS]; MOV o[HPOS], R 2; //stroke texture coordinates, transformed DP 3 o[TEX 0]. x, c[4], v[TEX 0]; DP 3 o[TEX 0]. y, c[5], v[TEX 0]; DP 3 o[TEX 1]. x, c[4], v[TEX 0]; DP 3 o[TEX 1]. y, c[5], v[TEX 0]; // splotch mask coordinates MOV o[TEX 2], v[TEX 0]; //get the Gouraud shade DP 3 R 1, c[8], v[NRML]; 04/06/05 //apply clamp-linear tone transfer function MUL R 1, c[9]. x; ADD R 1, c[9]. y; MAX R 1, c[9]. z; MIN R 1, c[9]. w; //now look up the weights for the TAMs blending EXP R 2. y, R 1. x; //frac(tone) ARL A 0. x, R 1. x; MOV R 3, c[A 0. x + 10]; MAD R 3, -R 2. y, R 3; MAD o[COL 1], R 2. y, c[A 0. x + 11], R 3; MOV R 4, c[A 0. x + 20]; MAD R 4, -R 2. y, R 4; MAD o[COL 0], R 2. y, c[A 0. x + 21], R 4; END © 2005 University of Wisconsin

Texturing Arbitrary Surfaces Tonal Art Map Example stroke Result Mesh 04/06/05 Lapped texture Preprocess Real-Time © 2005 University of Wisconsin

Texturing Arbitrary Surfaces • Lapped Textures [Praun et al. 2000] 04/06/05 © 2005 University of Wisconsin

Direction Field • Based on surface principal curvatures • Optimized to be smooth – [Hertzmann & Zorin 2000] – Symmetry: 180º instead of 90º – Sample on faces 04/06/05 © 2005 University of Wisconsin

Future Work • Stroke Growth • More general TAMs • View-dependent stroke direction • Automatic indication 04/06/05 © 2005 University of Wisconsin

Fine Tone Control in Hardware Hatching Matthew Webb, Emil Praun, Adam Finkelstein, Hugues Hoppe NPAR 2002 Proceedings Scheme 1 04/06/05 Scheme 2 © 2005 University of Wisconsin

04/06/05 © 2005 University of Wisconsin

Hijack. GL: Reconstructing Streams for Stylized Rendering Alex Mohr and Michael Gleicher 04/06/05 © 2005 University of Wisconsin

Motivation • NPR techniques are different then traditional • Applications must be architected to fit new NPR styles • Non-invasive methods • Altering a graphics application without modifying source code • Provides new functionality to a program • Explore potential for creating new and interesting visual styles 04/06/05 © 2005 University of Wisconsin

Previous Work • Concept of non-invasive stylization of interactive applications. • Alex Mohr and Michael Gleicher. Non-invasive, interactive, stylized rendering. 2001 ACM Symposium on Interactive 3 D Graphics, pages 175. 178, March 2001. • Performance analysis and Debugging • SGI and John Miles. gltrace. http: //reality. sgi. com/opengl/gltrace/, 1997. 04/06/05 © 2005 University of Wisconsin

Algorithm • Intercept calls to Open. GL library • Reconstruct Geometry • Run NPR rendering algorithm using reconstructed geometry • Process reconstructed data and call system Open. GL 04/06/05 © 2005 University of Wisconsin

Intercepting Open. GL • Change shared library • Can change behavior of application. • Must provide same interface as shared library. • Make dynamic linker link to your intercept library • Explicitly call system Open. GL from intercept library • New library acts as a filter for Open. GL calls 04/06/05 © 2005 University of Wisconsin

Reconstructing Geometry • Why? • Many NPR techniques need to know about silhouette edges, connectivity, etc. • Open. GL programs do not contain explicit representation of the information. • Challenge • Open. GL command stream does not directly correspond to a declarative representation of the geometry. • Need to process Open. GL command stream and create a declarative representation of the underlying geometry. 04/06/05 © 2005 University of Wisconsin

Reconstructing Geometry • Hash tables used to reconstruct geometry • Store attributes of the geometry. • Reduces memory size because only contains unique values. • Hash routine must be fast • Have to hash all vertex attributes to determine unique vertices. • Use eye coordinates to determine if vertices share same location and normal. • After attributes hashed, the vertex is stored in a hash table. 04/06/05 © 2005 University of Wisconsin

Reconstructing Geometry 04/06/05 © 2005 University of Wisconsin

Reconstructing Geometry • Computes connectivity info. • As edges added to hash table, if they come from triangles, two triangles are recorded as neighbors. • This encodes connectivity • Silhouette Edges and sharpness of edges recorded. • If no normal is given, an approximate normal is computed. • Computes screen space location of vertices when storing. 04/06/05 © 2005 University of Wisconsin

Reconstructing Geometry • Might want to know more? • Distinct objects • Foreground and background objects • Which objects should and should not be changed • This information not in Open. GL stream. • Make guesses and assumptions • i. e. Every time the transformation matrix changes a new object is being drawn. • Now the reconstructed geometry can be used by various rendering algorithms. 04/06/05 © 2005 University of Wisconsin

Examples • Selected to demonstrate the usefulness of geometry reconstruction and the potential and problems of the intercept based approach. • Authors do not claim that they are not the best, but are interesting. 04/06/05 © 2005 University of Wisconsin

Wireframe Rendering 04/06/05 © 2005 University of Wisconsin

Pencil Sketch Rendering 04/06/05 © 2005 University of Wisconsin

Blueprint Rendering 04/06/05 © 2005 University of Wisconsin

Cartoon Rendering 04/06/05 © 2005 University of Wisconsin

Model Capturing 04/06/05 © 2005 University of Wisconsin

Capabilities and Limitations • Implement a new rendering technique across a wide variety of applications. • Performance takes a hit. • • Application data has to go through interception library Geometry reconstruction occurs Data sent to renderer 20 fps • Don’t know why the program is drawing an object. • Apply the change to all objects • Can not tell the difference between a menu and a character 04/06/05 © 2005 University of Wisconsin

Future Work • Mildly Invasive Strategy • Create Open. GL extensions to provide hints on intent of program • Implement entire Open. GL library. • Make system more robust, complete, and efficient. 04/06/05 © 2005 University of Wisconsin

Questions 04/06/05 © 2005 University of Wisconsin

04/06/05 © 2005 University of Wisconsin

WYSIWYG NPR: Drawing Strokes Directly on 3 D Models Robert D. Kalnins, Lee Markosian, Barbara J. Meier, Michael A. Kowalski, Joseph C. Lee, Phillip L. Davidson Matthew Webb, John F. Hughes Adam Finkelstein Princeton University Brown University 04/06/05 © 2005 University of Wisconsin

Goals • Allow designer to directly annotate a 3 D model with strokes • Using “brush” styles draw strokes over a model from one or more viewpoints • Have the system adapt the number and placement of the strokes appropriately to maintain the artists original work 04/06/05 © 2005 University of Wisconsin

Previous Work • Systems for producing still images and systems for providing temporal coherence such as [Deussen and Strothotte 2000; Meier 1996] • Other systems of interactive rendering [Gooch et al. 1999; Kowalski et al. 2001]) • WYSIWYG painting on 3 D surfaces by Janrahan and Haeberli [1990] 04/06/05 © 2005 University of Wisconsin

How It Works • Three editing modes • Base coat: The artist can position the objects as well as set the “base coat” of each object • Outline: The artist can draw decals on the objects and also stylize silhouettes and creases. • Hatching: The artist can draw hatching strokes on the objects • In any mode the artist can modify the current “brush style” which affects stroke properties 04/06/05 © 2005 University of Wisconsin

An Example • Step 1: Model of a can is loaded. • Step 2: Toon base coat is chosen and background color is set. Lighting direction is also chosen. • Step 3: A black pencil style brush is chosen. Silhouette is chosen and it is propagated to all silhouettes. A crease is also created at the top of the can. • Step 4: Decal is added directly on the surface. • Step 5: Roughly parallel lines are drawn where shading is wanted to appear. Second set created at an angle to the first. 04/06/05 © 2005 University of Wisconsin

Rendering • Models are triangle meshes • Mesh is divided into patches • Each patch is rendered by 4 shaders • • 04/06/05 Base coat Creases Silhouettes Hatching © 2005 University of Wisconsin

Strokes • Stroke base path represented as a Catmull -Rom spline. • User can control color, alpha, width, taper and “halo” • Controlled using sliders • 1 D texture to specify cross-section of stroke 04/06/05 © 2005 University of Wisconsin

Stroke Visibility • Z buffer not enough • An ID reference image is used • Off-screen buffer where each mesh face, crease edge, and silhouette polyline is rendered in a unique color • Strokes found by sampling the ID reference image along stroke path 04/06/05 © 2005 University of Wisconsin

Paper Textures • User can apply a paper effect to any semitransparent scene primitive, background image, toon shader, or stroke • Basically encodes peaks and valley of paper • At peaks paper easily catches pigment • At valleys pigment is resisted 04/06/05 © 2005 University of Wisconsin

Base Coat • First, user can choose a background color or an image that fills the rendering window. • Second, user can select for each patch a “base coat” Solid Color Toon Shader 04/06/05 © 2005 University of Wisconsin

Decals and Creases • Decals drawn directly on surface • Stored as spline curves – Benefits • Decals do not require a parameterization of the surface • No sampling artifacts at magnified or strange angles • Screen-space width of strokes consistent under all views • Crease strokes are represented as offsets from the original mesh – When the artist oversketches a crease system saves this “detail” in offset lists – Offset lists store small scale wiggles relative to the base path 04/06/05 © 2005 University of Wisconsin

Stroke Synthesizing • Rubber Stamp – Repeats a sequence of example offsets along each crease • Stroke synthesis – 3 or 4 example strokes given – Each stroke separated into an unstyled “base patch” 04/06/05 © 2005 University of Wisconsin

Examples 04/06/05 © 2005 University of Wisconsin

Silhouettes • • View dependent Detected using function Silhouettes taken to be the zero-set of f Silhouettes assigned consistent parameterization for temporal coherence • Style is taken as the arc-length period for repeating the offset pattern using a scaling factor as shown here 04/06/05 © 2005 University of Wisconsin

Hatching • Structured – Parallel strokes – Able to obtain automatic level of detail control • Free – Artist draws arbitrary strokes – Must explicitly build LODs • Mobile – Used to represent light cast from behind camera – Can be structured or free – Limited to smooth surface regions, motion limited to straight lines in either u or v 04/06/05 © 2005 University of Wisconsin

Picture Examples 04/06/05 © 2005 University of Wisconsin

Future Work • Extension of the system – Stippling – Pointillism • Automatic synthesis of creases could be extended to hatching strokes • More object to object interactions – Shadows – Smudging 04/06/05 © 2005 University of Wisconsin

04/06/05 © 2005 University of Wisconsin