Ambient Occlusion and Edge Cueing for enhancing Real

Ambient Occlusion and Edge Cueing for enhancing Real Time Molecular Visualization Marco Tarini, Paolo Cignoni, Claudio Montani

Background: Molecular Graphics 4 The discipline and philosophy of studying molecules and their properties through graphical representations 4 Stereotypical Scientific Visualization task • 3 D shape Molecule functionality 4 Long story • Long bibliography • Many established techniques and principles Hemagglutinin image courtesy of Riumplus 4 [ Capstone after lunch by Jane Richardson! ] image courtesy of Accelrys 2

Background: Molecular Graphics 3 D molecular structure modelling 3 D representation(s) rendering images (animations, stereo images. . . ) insight, understanding Many 3 D Metaphors: 4 Ball and Sticks 4 "Licorice" modes 4 Space Fill (CPK) 4 Solvent Excluded Surfaces (Connolly's) 4 Various kind of isosurfaces • (electron density, potential. . . ) 4 Higher level abstractions: • e. g. ribbons 3

Background: Molecular Graphics 3 D molecular structure modelling 3 D representation(s) rendering images (animations, stereo images. . . ) insight, understanding Problems: 4 Readability: • Unfamiliar, awkward 3 D structures • Hard to understand 3 D shapes (spatial relationships, etc) 4 Efficiency: • Geometric complexity (need for real time) (e. g. for very large proteins) 4

Background: Molecular Graphics 3 D molecular structure modelling 3 D representation(s) rendering images (animations, stereo images. . . ) insight, understanding The content of this talk: 4 Advanced rendering techniques can be of great help here! 4 Advanced rendering techniques can be adapted to the specific case of molecular rendering 5

The rest of this talk: ad-hoc Advanced Rendering Techniques 4 Impostors for spheres, cylinders 4 Ambient occlusion (for impostors) 4 Depth aware borders (for impostors) 4 Localized depth cueing effect (for impostors) 6

Impostor based rendering For each primitive (sphere or cylinder): 4 render a screen-oriented impostor quad • per vertex: • displace vertex in screen space • preprocess all common sub-expressions • for each fragment: • test membership (discard outliers fragments) • compute –z – normal – lighting – texture coords (as we will see) 7

The rest of this talk: ad-hoc Advanced Rendering Techniques 4 Impostors for spheres, cylinders 4 Ambient occlusion (for impostors) 4 Depth aware borders (for impostors) 4 Localized depth cueing effect (for impostors) 8

Ambient Occlusion 4 Global lighting technique • Approximation of global illumination 4 Concept: • light each point p with normal np with the irradiance I(p) (quantity of light directly reaching p from any direction) 4 in our case, L is: k (a constant) if p visible from ω, 0 if p shadowed 4 Effect: • “lighted by daylight under a cloudy sky”: 9

Ambient Occlusion: why 4 Known to conceive realism Local lighting (but, we don’t care) Ambient Occlusion 10

Ambient Occlusion: why 4 Known to conceive 3 D shape understanding • perceptual study in [Langer & Bulthoff, Tech. Rep 99] 4 Especially in our case (molecules) • large scale structures composed by multitudes of tiny shapes → normals ≈ noise • see also: particle systems by [Bigler et. Al. VCGT 06] 11

Ambient Occlusion: why Local lighting Ambient Occlusion 12

Ambient Occlusion: how 4 Store computed Irradiance of each point on molecule surface • (fast precomputation) 4 use it during rendering 4 PROBLEM: where do we store it? • use a texture… • for impostors! 13

Texturing the molecule x, y, z object space u, v texture space E x, y, z gnomic projection unfolding [Praun and Hoppe. TG 03] y, z gnomic projection unfolding G A B C D F H A B C D 14

Texturing the molecule x, y, z object space u, v texture space E G A B C D F H A B C D 15

Texturing the molecule x, y, z object space u, v texture space E G bilinear filetering A B C D F H texel replications A B C D 16

Texturing the molecule Global Texture for the molecule: v ATOM 1 BOND 1 ATOM 2 ATOM 3 ATOM 4 BOND 2 ATOM 5 ATOM 6 ATOM 7 BOND 2 BOND 4 BOND 5 BOND 6 BOND 7 ATOM 8 BOND 8 ATOM 9 ATOM 10 ATOM 11 ATOM 12 BOND 9 BOND 10 BOND 11 (unused) 4 one texture patch for each: • atom (sphere) or • bond (cylinder) in the scene 4 pack together all texture patches (trivial) u 17

Texturing the molecule For example: v IRRADIANCE MAP 4 small molecule: • ~80 patches • ~40 atoms • ~40 bonds • texture size: 512 x 512 texels • patch size: 50 x 50 texels 4 large molecule: • 120. 000+ patches • 60. 000+ atoms • 60. 000+ bonds u • texture size: 2048 x 2048 texels • patch size: 6 x 6 texels 18

Computing irradiance map on the GPU 4 Get N directions w 1. . N sampling the unit sphere 4 For each i in (1. . N) : 1. render new depth-map from view direction wi 2. render over irradiance-map: for each patch: for each fragment (texel) f : 1. invert (u, v) map project depth-map wi get pos p, normal np 2. project p over depth-map & check visibility (compare depths) irradiance-map 3. if visibile, accumulate light K(np∙wi) (if >0) at f (with alpha blending) 19

Computing irradiance map on the GPU 4 Get N directions w 1. . N sampling the unit sphere 4 For each i in (1. . N) : 1. render new depth-map from view direction wi 2. render over irradiance-map: for each patch: for each fragment (texel) f : 1. invert (u, v) map depth-map wi Render flat impostors! - mid z (better result) - no z-displacing (much faster) get pos p, normal np 2. project p over depth-map & check visibility (compare depths) irradiance-map 3. if visibile, accumulate light K(np∙wi) (if >0) at f (with alpha blending) 20

Computing irradiance map on the GPU 4 Get N directions w 1. . N sampling the unit sphere 4 For each i in (1. . N) : 1. render 2 new depth-map from view directions wi & -wi 2. render over irradiance-map: for each patch: for each fragment (texel) f : 1. invert (u, v) map depth-map wi depth-map -wi get pos p, normal np 2. project p over depth-map & check visibility (compare depths) irradiance-map 3. if visibile, accumulate light K(np∙wi) (if >0) at f (with alpha blending) 21

The rest of this talk: ad-hoc Advanced Rendering Techniques 4 Impostors for spheres, cylinders 4 Ambient occlusion (for impostors) 4 Depth aware borders (for impostors) 4 Localized depth cueing effect (for impostors) 22

Depth aware borders 4 Example • See demo 23

Depth aware borders: how 4 Single pass technique! B depth impostor A C 24

Depth aware borders: how 4 Single pass technique! B depth impostor A C 25

Depth aware borders: how 4 Single pass technique! B depth impostor A C 26

Depth aware borders: how 4 Single pass technique! B depth impostor A C 27

The rest of this talk: ad-hoc Advanced Rendering Techniques 4 Impostors for spheres, cylinders 4 Ambient occlusion (for impostors) 4 Depth aware borders (for impostors) 4 Localized depth cueing effect (for impostors) 28

Localized depth cueing (“halos”) Depth cueing (“fog”) 4 common technique to visually separate far from near 4 BUT: 1. far parts lose contrast (global loss of contrast) 2. cannot discriminate between too many planes Our alternative: localized depth cueing (“halos”) 4 like fog, but concentrate effects near depth jumps! Similar to: [T. Luft, C. Colditz, O. Deussen. SIGGRAPH 2006] (parallel result) Matisse Seuret Picasso Dalì See Demo! 29

Localized depth cueing (“halos”) 4 First pass: • render geometry and depth buffer 4 Second pass: (alpha blended) • render coronas around each atom • choose alpha according to • screen distance to atom • distance of current fragment depth from destination depth 30

Putting all together 4 Combining: • • Impostors for spheres, cylinders. Ambient occlusion (for impostors) Depth aware borders (for impostors) Localized depth cueing effect (for impostors) 4 with standard techniques • • • Direct light (Lambertian + Phong) Cast shadow Standard depth cueing (fog) Toon shading (for illustrative renderings) Antialiasing (render to 2 x, resample to screen) 31

Implementation Open. Source project 4 win installer 4 unix, mac versions coming up 4 reads PDB files 4 exports hi res. images (e. g. for print publications) 4 …and small animations (e. g. for web sites, presentations) GIF animation Just Google for 32