Painterly Rendering with Curved Brush Strokes of Multiple

Painterly Rendering with Curved Brush Strokes of Multiple Sizes Aaron Hertzman, NYU http: //mrl. nyu. edu/publications/painterly 98/ (SIGGRAPH ’ 98) Presented by Marc Flury 3/3/2021 1

What is Painterly Rendering? An oxymoron? It’s a method of reproducing the artistic style and/or expression of a painting using a source image. An image space technique Sort of a glorified Photoshop™ filter 3/3/2021 2

An Example 3/3/2021 3

An Example 3/3/2021 4

In This Presentation… The rationale of painterly rendering using “curved brush strokes of multiple styles” Overview of implementation Examples of applying different parameters to achieve different styles 3/3/2021 5

Rationale Real painting requires n n n Time Skill Artistic talent Typical filtering methods n n 3/3/2021 Use only one size brush stroke Can’t refine style with multiple passes Only support one style Image is “flattened” 6

Typical Filtering - Photoshop 3/3/2021 7

More detail needs smaller strokes 3/3/2021 8

Improved Quality and Detail 3/3/2021 9

Advantages Much faster than painting n Can be used for video (or even interactive rendering) Multiple brush sizes allow for varying detail and continuous color regions Multi-pass method is somewhat analogous to true artistic techniques (better results? ) The system can be parameterized to create different “styles” 3/3/2021 10

General Painting Algorithm function paint (source. Image, R 1 … Rn) { canvas : = a new constant color image // paint the canvas for each brush radius Ri, from largest to smallest do { // apply Gaussian blur reference. Image = source. Image * G(fσRi) // paint a layer paint. Layer(canvas, reference. Image, Ri) } return canvas } 3/3/2021 List of brush radii Original picture Approximation of image we want to paint Gaussian kernel – based on brush size Locates areas of image that differ and paints with brush explained later 11

Gauss source image Gauss reference image Paint canvas 3/3/2021 12

General Painting Algorithm function paint (source. Image, R 1 … Rn) { canvas : = a new constant color image // paint the canvas for each brush radius Ri, from largest to smallest do { // apply Gaussian blur reference. Image = source. Image * G(fσRi) // paint a layer paint. Layer(canvas, reference. Image, Ri) } return canvas } 3/3/2021 13

Layer Painting Algorithm function paint. Layer(canvas, reference. Image, R) { S : = a new set of strokes, initially empty D : = difference(canvas, reference. Image) grid : = f g R for x=0 to image. Width stepsize grid do { creates a pointwise difference image using: |(r 1, g 1, b 1) – (r 2, g 2, b 2)| = ((r 1 -r 2)2 + (g 1 -g 2)2 + (b 1 -b 2)2)1/2 for y=0 to image. Height stepsize grid do { M : = the region(x-grid/2…x+grid/2, y-grid/2…y+grid/2) area. Error : = sum. Of. Error(M, D) / Sums the error near (x, y) grid 2 if (area. Error > T) then { Finds largest error point (x 1, y 1) : = max. Point(area. Error) stroke : = make. Stroke(R, x 1, y 1, reference. Image) add stroke to S } Calculates stroke to be placed on canvas – explained later } } paint all strokes S on canvas – random order } 3/3/2021 Prevents regularity – counter intuitive? 14

Curved Brush Strokes Anti-aliased cubic B-splines Each stroke models the color gradient of the reference image Stroke Representation n 3/3/2021 List of control points Color Brush Size 15

Curved Stroke Algorithm function make. Spline. Stroke(x 0, y 0, R, ref. Image) { stroke. Color = ref. Image. color(x 0, y 0) K : = new stroke, radius R, color stroke. Color add point (x 0, y 0) to K Prevents infinite loop Checks that approximation is still good enough (x, y) : = (x 0, y 0) (last. Dx, last. Dy) : = (0, 0) for i=1 to max. Stroke. Length do { if (i > min. Stroke. Length and (|ref. Image. color(x, y) – canvas. color(x, y)| < |ref. Image. color(x, y)stroke. Color)) then return K if (ref. Image. gradient. Mag(x, y) ==0) then return K (gx, gy) : = ref. Image. gradient. Direction(x, y) (dx, dy) : = (-gy, gx) if (last. Dx * dx + last. Dy * dy < 0) then (dx, dy) = (-dx, -dy) (dx, dy) : = fc * (dx, dy) + (1 -fc) * (last. Dx, last. Dy) (dx, dy) : = (dx, dy)/(dx 2 + dy 2)1/2 (x, y) : = (x + R*dx, y + R*dy) Checks that there is a gradient Calculates direction of greatest change (gradient) Calculates normal Reverses direction if necessary Filters brush stroke curvature based on fc (last. Dx, last. Dy) : = (dx, dy) add the point (x, y) to K } return K 3/3/2021 } 16

Finding Control Points D 2 G 1 (x , y ) D 1 2 2 G 2 θ 1 (x 1, y. D 1)0 G 0 θ 0 (x 0, y 0) 3/3/2021 17

Style Parameters Approximation Threshold (T) Brush Sizes – Smallest (Ri), Number (n), Size Ratio (Ri-1/Ri) Curvature Filter (fc) Blur Factor (fσ) Min and max stroke lengths (min. Length, max. Length) Opacity (α) Grid size (fg) Color Jitter (jh, js, jv, jr, jg, jb) 3/3/2021 18

Examples – Source Image 3/3/2021 19

Examples – “Impressionism” T = 100 R = (8, 4, 2) fc = 1 fσ =. 5 α=1 fg = 1 min. Length = 4 max. Length = 16 3/3/2021 20

Examples – “Expressionism” T = 50 R = (8, 4, 2) fc =. 25 fσ =. 5 α =. 7 fg = 1 min. Length = 10 max. Length = 16 3/3/2021 js =. 5 21

Examples – Water Color T = 200 R = (8, 4, 2) fc = 1 fσ =. 5 α =. 5 fg = 1 min. Length = 4 max. Length = 16 jr = jg = jh =. 3 3/3/2021 22

Examples – “Pointillism” T = 100 R = (4, 2) fc = 1 fσ =. 5 α=1 fg =. 5 min. Length = 0 max. Length = 0 jv = jh =. 3 3/3/2021 23

More Examples…. 3/3/2021 24

More Examples… 3/3/2021 25

More Examples… 3/3/2021 26