Volume Graphics Technology to Tools David R Nadeau

Volume Graphics • Technology to Tools David R. Nadeau Principal Scientist San Diego Supercomputer Center University of California, San Diego, USA SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

Technology & Tools • Technology = a useful idea – Polygons, images, volumes, transforms, shading, … • Tool = an idea put to use – Visualization, art, story telling, games, … • As a technology matures, it becomes a tool – We focus more on its use, than its mechanism – Use really explodes when artists begin to use it SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

Technology & Tools • Written storytelling – dramatically advanced by… – Printing press, movable type, word processing, e-mail, Internet, … • Music composition – dramatically advanced by… – Standardized notation, equal-tempered tuning, piano, electric guitar, synthesizer, digital signal processing, … • Computer graphics – dramatically advanced by… – Hmmm…? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Topics of the Talk • History – What has happened so far? • Rendering – – What can we do now? What would we like to do? What might we be able to do in the future? How might this affect the way we render volumes? • Authoring – How might rendering issues affect how we create volumes? – What might future authoring tools look like? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring y r e V A Brief Look at Graphics History… • What are some major events? • What did they enable? • • Surface vs. Volume graphics Technology vs. Story telling vs. Games SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Evolution of surface graphics • Selected Technology events: 1963 “Sketchpad” I. E Sutherland, "Sketchpad: A Man-Machine Graphical Communication System, " Ph. D. Thesis, MIT, 1962. Digital line drawing J. E. Bresenham, “Algorithm for computer control of a digital plotter, ” IBM Systems Journal, 4(1), 1965. 1971 Gouraud shading H. Gouraud, “Continuous shading of curved surfaces, ” IEEE Transactions on Computers, C -20, 1971. Hidden surface removal 1974 Polygon clipping 1975 Phong shading M. E. Newell, R. G. Newell, T. L. Sancha, “A Solution to the Hidden Surface Problem, ” Proceedings of ACM National Meeting, 1972. 1965 1972 Scene graphs 1976 Texture mapping Shadows 1977 Standard lighting model 1978 Bump mapping I. E. Sutherland, G. W. Hodgman, “Reentrant Polygon Clipping, ” CACM, 17(1), 1974. B. T. Phong, “Illumination for Computer Generated Pictures, ” CACM, 18(6), 1975. J. H. Clark, “Hierarchical Geometric Models for Visible Surface Algorithms, ” CACM, 19(10), 1976. J. F. Blinn, M. E. Newell, “Texture and Reflection in Computer Generated Images, ” CACM, 19(10), 1976. F. C. Crow, “Shadow Algorithms for Computer Graphics, ” Computer Graphics, 11(2), 1977. J. F. Blinn, “Models of Light Reflection for Computer Synthesized Pictures, ” Computer Graphics, 11(2), 1977. J. F. Blinn, “Simulation of Wrinkled Surfaces, ” Computer Graphics, 12(3), 1978. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Evolution of surface graphics • Selected Technology events: 1980 Ray tracing Geometry VLSI 1982 Auto. CAD Particle systems 1983 Alias Radiosity Shaders 1984 SGI IRIS 1000 Wavefront 1986 Rendering eq. 1989 Render. Man 1992 Open. GL 1993 Direct 3 D 1994 GLINT 300 SX 1995 VRML Voodoo 1997 Riva 128 T. Whitted, “An improved illumination model for shaded display, ” CACM, 23(6), 1980. J. H. Clark, “The Geometry Engine: A VLSI geometry system for graphics, ” SIGGRAPH 82. Autodesk Inc. W. T. Reeves, “Particle Systems – A Technique for Modeling a Class of Fuzzy Objects, ” SIGGRAPH 83. Alias Research Inc. C. M. Goral, K. E. Torrance, D. P. Greenberg, B. Battaile, “Modeling the Interaction of Light Between Diffuse Surfaces, ” SIGGRAPH 84. R. L. Cook, “Shade trees, ” SIGGRAPH 84. Silicon Graphics Wavefront Inc. J. T. Kajiya, “The rendering equation, ” SIGGRAPH 86. PIXAR Open. GL ARB Microsoft 3 Dlabs VRML/Web 3 D Consortium 3 dfx n. Vidia SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Too many papers to list

History Rendering Authoring Evolution of surface graphics • Selected Story Telling events: 1982 1984 1985 1986 1987 1988 1989 1991 1992 Tron Wrath of Khan Andre & Wally B. The Last Starfighter Young Sherlock Holmes Luxo Jr. Red’s Dream Stanley & Stella Tin Toy Knickknack The Abyss Beauty and the Beast Terminator 2 Aladdin Lawnmower Man SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Disney Paramount PIXAR Universal Paramount PIXAR Symbolics PIXAR Fox Disney Tri-Star Disney New Line All images copyright by Disney, Fox, Paramount, New Line, PIXAR, and Tri-Star, as appropriate.

History Rendering Authoring Evolution of surface graphics • Selected Story Telling events: 1993 Jurassic Park The Mask 1994 Reboot True Lies 1995 Toy Story Dragonheart 1996 Twister Geri’s Game Lost World 1997 Star Wars, Special Edition Titanic A Bug’s Life 1998 Antz Toy Story 2 1999 Star Wars, Episode 1 2000 Dinosaur SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Universal New Line Main. Frame Fox PIXAR Universal Warner Bros. PIXAR Universal Lucasfilm Paramount PIXAR Dream. Works PIXAR Lucasfilm Disney All images copyright by Disney, Dream. Works, Paramount, Lucasfilm, Main. Frame, PIXAR, and Universal, as appropriate.

History Rendering Authoring Evolution of surface graphics Catacomb 3 -D • Selected Game events: 1991 Catacomb 3 -D Wolfenstein 3 -D 1992 7 th Guest DOOM 1993 Myst 1995 Dark Forces Nintendo 64 1996 Quake Tomb Raider Jedi Knight 1997 Quake II Riven Half-Life 1998 Thief Unreal 1999 Quake III: Arena 2000 Real. Myst Id Software Tilobyte Studios Quake Id Software Cyan Productions Tomb Raider Lucas. Arts Nintendo Id Software Eidos Interactive Jedi Knight Half-Life Lucas. Arts Id Software Cyan Productions Sierra Eidos Interactive Epic Mega. Games Unreal Quake III: Arena Id Software Cyan Productions SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego DOOM All images copyright by Id Software, Cyan Productions, Eidos Interactive, Epic Mega. Games, Lucas. Arts, and Sierra, as appropriate.

History Rendering Authoring Evolution of surface graphics 1960’s 1970’s 1980’s 1990’s Games Story Telling Technology SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 2000’s

History Rendering Authoring Evolution of volume graphics • Selected Technology events: Volume ray casting 3 D texture 1985 generation H. Tuy, L. Tuy, “Direct 2 D display of 3 D objects, ” Computer Graphics and Applications, 4(10), 1984. 1987 Marching cubes W. E. Lorensen, H. E. Cline, “Marching cubes: A high resolution 3 D surface construction algorithm, ” SIGGRAPH 87. 1989 Hypertexture 1990 Splatting K. Perlin, E. M. Hoffert, “Hypertexture, ” SIGGRAPH 89. 1991 Volume sculpting T. A. Galyean, J. F. Hughes, “Sculpting: An interactive volumetric modeling technique, ” SIGGRAPH 91. P. Lacroute, M. Levoy, “Fast volume rendering using a shear-warp factorization of the viewing transformation, ” SIGGRAPH 94. B. Cabral, N. Cam, J. Foran, “Accelerated volume rendering and tomographic reconstruction using texture mapping hardware, ” Volume Visualization Symposium, 1994. 1984 Shear-warp 1994 3 D texture mapping Volume. Pro 1999 Ge. Force 256 Ge. Force 2 2000 Direct. X 8. 0 K. Perlin, “An image synthesizer, ” SIGGRAPH 85. L. Westover, “Footprint evaluation for volume rendering, ” SIGGRAPH 90. Real-Time Visualization, Mitsubishi n. Vidia Microsoft SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Too many papers to list

History Rendering Authoring Evolution of volume graphics • Selected Story Telling events: 1997 Contact 1998 Sphere Contact Warner Bros. Sphere SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego All images copyright by Warner Bros. , San Diego Supercomputer Center, and the American Museum of Natural History, as appropriate.

History Rendering Authoring Evolution of volume graphics • Selected Game events: 2001 ? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego ?

History Rendering Authoring Evolution of volume graphics 1960’s 1970’s 1980’s 1990’s Games Story Telling Technology SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 2000’s

History Rendering Authoring Evolution of volume graphics • Volume graphics today… …is where surface graphics was 15 years ago – We are at the start of a transition from technology to tool • What enabled story telling and games for surface graphics? • What might do the same for volume graphics? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Enabling events Surface graphics 1960’s 1970’s 1980’s 1990’s 2000’s Games Story Telling Technology • 1 st business-level (expensive): 1982 -4 – Attention-grabbing event: TRON – Interactive rendering hardware: SGI – Authoring tools: Alias, Wavefront • 1 st consumer-level (cheap): – Attention-grabbing event: Catacomb 3 D, DOOM – Interactive rendering hardware: Voodoo SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 1991 -3 1997

History Rendering Authoring Enabling events Volume graphics ? 1960’s 1970’s 1980’s Story Telling Technology – Attention-grabbing event: ? – Interactive rendering hardware: Volume. Pro, SGI? – Authoring tools: ? • 1 st consumer-level (cheap): – Attention-grabbing event: ? – Interactive rendering hardware: n. Vidia? University of California, San Diego 2000’s Games • 1 st business-level (expensive): SAN DIEGO SUPERCOMPUTER CENTER 1990’s 1999 2000

History Rendering Authoring Interactive Volume Rendering… • What can we do now? • What would we like to do? • What might we be able to do in the future? • How might this affect the way we render volumes? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring What can we do now? • Canonical volume visualizations… • What can we do interactively? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring What can we do now? n. Vidia Ge. Force 2 Ultra 64 MB 1 Gpixels/s SGI IR 2 Memory Fill rate RTviz Volume. Pro 256 MB N/A Texture rate Max stored volume N/A 16 x 2563 (scalar) 2 Gtexels/s 2563 (RGBα) 768 Mtexels/s 2563 (RGBα) Max volume 2563 @ 30 fps 2563 @ 15 fps 2563 @ 12 fps (1 pipe, 4 RMs) 64 MB 896 Mpixels/s • But 2563 is a small volume… SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: Product literature

History Rendering Authoring What resolution can we see? • Eye’s lens focuses light onto retina – Fovea = focus area = center 20 • More receptors at the fovea – Mostly “Cones” (color) at fovea – Mostly “Rods” (intensity) in surrounding area Fovea Periphery SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: Perception, Sekuler & Blake, 2 nd ed, 1990, Mc. Graw-Hill.

History Rendering Authoring What resolution can we see? • Measure visual acuity by viewing “gratings” of parallel lines Gratings • How fine a grating can you see? • Normal vision: • 120 lines/degree in center 20 = 2, 400 lines • Legally blind: • 1/10 th normal vision • 12 lines/degree in center 20 = 240 lines SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: Visual Perception, Spillmann & Werner, 1990, Academic Press.

History Rendering Authoring What resolution can we see? • A 2563 volume legally blind! – Very low resolution • Normal vision requires 2, 4003! – If it only covers central 20 of visual field • 180 visual field requires 21, 6003! – 120 lines/degree in 180 = 21, 600 lines! SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring What resolution can we show? • We’re constrained by screen resolution – 1280 x 1024 on a large monitor – 30 pixels/degree at normal distance – ¼ normal vision = visually impaired • For now… – 10243 is a minimum for effective screen use • Smaller volumes are blurry (1 voxel covers multiple pixels) – But we want much higher volume resolutions… SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring What resolution do we want? • Medical imaging (cryosections) – State of the art: 2048 x 2048 images, 1/10 mm apart – 2048 x 18000 for full body (180 cm person) – Visible Human Male: 2048 x 1216 x 1871 • Re-digitization of images: 4096 x 2700 x 1871 – Visible Human Female: 2048 x 1216 x 5189 – Recent brain only: 1789 x 1472 x 1152 SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Head data from the Visible Human Project, www. nlm. nih. gov/research/visible Brain data from Arthur Toga’s lab, LONI, UCLA, www. loni. ucla. edu

History Rendering Authoring What resolution do we want? • Medical imaging (CT scan) – State of the art: 2048 x 2048 images, ½mm apart – 2048 x 3600 for full body (180 cm person) – Visible Human Male: 512 x 1871 SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Head data from the Visible Human Project, www. nlm. nih. gov/research/visible

History Rendering Authoring What resolution do we want? • Microscopy imaging (fluorescence) – State of the art: 1024 x 1024, 0. 15 microns apart – 1024 x 75 for a 10 micron cell – Cell membranes: SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 768 x 72 Cell data from Hiro Tsukada, Eric Elenko, and Maria Pinhal, UCSD Cancer Center

History Rendering Authoring What resolution do we want? • Simulations – Tend to use all available memory on a supercomputer – SDSC IBM SP “Blue Horizon” • 1152 processors (144 nodes, 8 cpus/node, +others), 1. 7 teraflops • 576 Gbytes total memory (4 Gbytes/node) • Largest possible volume is 50003 – Ocean simulation: SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 2160 x 960 x 30 Ocean data from Detlef Stammer, UCSD, Scripps Institute of Oceanography

History Rendering Authoring What resolution do we want? • Combine to build volumetric scenes – Composite overlapping volumes – Mosaic volumes to fill space – Combine procedural elements SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring What resolution do we want? Full eye Fovea Full screen 2563 • We’re a long way from what we want… why? It’s a logarithmic scale! SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Memory and bandwidth • Memory needed grows with cube of volume’s width – – – 643 RGBα = 1 Mbyte 1283 RGBα 2563 RGBα 5123 RGBα 10243 RGBα = 8 Mbytes = 64 Mbytes = 512 Mbytes = 4096 Mbytes • Bandwidth needed also grows with frame rate – 10 fps minimum – 30 fps better – 70 fps best SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring What will volumes require? 2563 Memory 10 fps 30 fps 64 MB 640 MB/s 1. 9 GB/s 70 fps 4. 5 GB/s (legally blind) 10243 4 GB 40 GB/s (full screen) 24003 120 GB/s 180 GB/s (30 Gpixl/s) 55 GB 550 GB/s 1. 6 TB/s 3. 8 TB/s 40 TB 400 TB/s 280 TB/s (fovea) 216003 120 TB/s (full eye) • 120 GB/s is > 30 times current bandwidths! SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Growth of pixel fill rates SGI PC cards * Not counting custom hardware or special configurations • Fill rates recently growing by x 2 every year SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 1996 30 Mpixels/s 3 Dlabs Permedia 1997 100 Mpixels/s n. Vidia RIVA 128 1998 366 Mpixels/s 3 dfx Voodoo 3 1999 540 Mpixels/s ATI Rage Fury MAXX 2000 1000 Mpixels/s n. Vidia Ge. Force 2 Ultra Data source: Product literature

History Rendering Authoring How long to get there? Bandwidth @ 30 fps Years 2563 1. 9 GB/s (legally blind) (0. 5 Gpixels/s) 10243 120 GB/s (full screen) (30 Gpixels/s) 24003 1600 GB/s (fovea) (400 Gpixels/s) 216003 120000 GB/s (full eye) (30, 000 Gpixels/s) • But, this is not very realistic… SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Last year 5 years 8 years 14 years

History Rendering Authoring How long to get there? About trends… “A frequent criticism of predictions of the future is that they rely on mindless extrapolation of current trends without consideration of forces that may terminate or alter that trend. ” – Ray Kurzweil, The Age of Spiritual Machines SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring How long to get there? About the computer industry… “If the automobile industry had made as much progress in the past fifty years, a car today would cost a hundredth of a cent and go faster than the speed of light. ” – Ray Kurzweil, The Age of Spiritual Machines SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring CPU performance growth • Floating point power grows by x 2 every 2 years SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: SPEC benchmark database, www. spec. org

History Rendering Authoring CPU performance growth • At this rate, by 2020 transistor insulators will be just a few atoms thick – A possible end to growth using this technology – What technology will arrive next? • Meanwhile, back to the present… SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Memory speed growth • PC processors only • Memory bandwidth only grows by x 2 every 5 years SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: Kingston Technology, www. kingston. com

History Rendering Authoring Comparing growth rates • Highly unlikely that fill rates can continue to grow faster than processor speed and memory bandwidth SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring What should we do? • Don’t bet on rapid fill rate growth to satisfy our volume rendering needs – If fill rate growth drops to track memory bandwidth growth, full screen volume rendering may arrive in 25 years, not 5 years • And full eye in 70 years! • Faster hardware is no substitute for a better algorithm – How can we change the way we work? – Lots of ways… but I’ll focus on a few… SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Change how we render • Object-space rendering traversals are in common use – Scan through all voxels and project towards screen • Splatting, shear-warp, 3 D texture mapping, point clouds – Memory streaming is possible • If the viewpoint is outside the volume • O(n 3) time & space n = volume width SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Change how we render • Image-space traversals are more time/space friendly – Scan through all screen pixels and project towards voxels • Ray casting, … – Memory streaming not possible • Nearly random access • O(k r n) time & space n = volume width r = image resolution (width x height) k = additional computation factor (for comparisons) SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Change how we render • Is volume ray casting really O(k r n)? – Worst case: each ray takes longest path = n 3 – Normal case: early ray termination reduces it – Rays diverge, skipping voxels, introducing aliasing – For accuracy: supersample – For speed: use volume MIP-mapping • Upfront cost to build MIP-map volumes • Increases memory needed (but not bandwidth) – Memory is cheap, bandwidth is not SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

fovea full screen Rendering time growth legally blind History Rendering Authoring • To interactively render larger volumes, we need to get on the better curves – such as ray casting SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring What can we do? • Changing our rendering algorithm can help – For “large data, ” image-space traversals are better than object-space traversals – We have “large data” now Cross-over • What about changing our data? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Volume Authoring… • How might rendering issues affect how we create volumes? • What might future authoring tools look like? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring CPU vs. memory speeds • Main memory access (cache miss) • PC processors, excluding RDRAM & DDR SDRAM • Slower memory speed growth means memory references are getting more costly, in terms of cycles SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: SPEC benchmark database, www. spec. org

History Rendering Authoring CPUs per Supercomputer Average of top 100 Average of top 500 • Adapt to slow memory by adding CPUs w/own memory • CPUs/Supercomputer grows by x 2 every 3 years SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: Top 500 Supercomputers, www. top 500. org

History Rendering Authoring Cycles & bandwidth • Cycles are more available than memory bandwidth – “Easier” to add CPUs than increase bandwidth – Parallel computing is an obvious industry trend • But CPU coordination and data sharing is still bandwidth limited • Can we trade computation for memory accesses? – Data compression is clearly needed • Texture compression nearly standard in graphics hardware • Geometry compression becoming available – Additional techniques to reduce memory access costs • Interleaved frame buffers, Z-buffer cache • View frustum culling, occlusion culling SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring with cycles • “Shaders” compute parts of the scene as needed – Already common-place in software rendering • “Data amplification” – small number of parameters generates large amount of scene content – Programmable graphics pipelines arriving now – For surface graphics: splines, procedural textures, particles – For volume graphics: voxelization, procedural volumes SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring with cycles • Shaders aren’t just for shading any more – Procedural content creation • Noise and turbulence functions – Voxelization based upon geometry “parameters” • Voxelize on demand – don’t prevoxelize • Authoring uses a mix of data and shaders – Import data sets (CT, MRI, cryosection, etc. ) – Sculpt & 3 D paint volumes – Program “shaders” SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring with shaders • Procedural turbulence creates “clouds” – Defines a color & opacity at each point in space – Animate parameters to evolve the cloud – Voxelize during ray-casting • No volumes created • Very low memory use SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring with shaders • Manipulate shapes – Compute effects during ray-casting • No additional volumes created • For each point in space: – Compute shortest distance to surface – Perturb the distance with turbulence – Map distances to opacity Compute distances Add turbulence SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Map to opacity Do at high resolution

History Rendering Authoring with shaders • Constructing shapes – Constructive Solid Geometry (CSG) – Cut using primitive shapes • (or anything) – Transform and “cut” during ray-casting • No pre-cut volumes created SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring with shaders • Repeat to build volumetric scenes – Multiple data sets, geometry, shaders, … – Evaluate some or all during ray-casting • Rendering is an active part of authoring SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring scenes • Several existing scene structure metaphors – – – CSG trees = combine primitives (most CAD apps. ) Composite trees = combine images (Houdini, Shake) Shade trees = combine shaders (Render. Man) Scene graphs = lay out data sets (VRML, Java 3 D) Expression trees = compute scene (any programming lang. ) • “Compilation” prevoxelizes some, all, or none of the scene – Tune to minimize error, memory use, rendering time SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Visualizing the Orion nebula • Fluorescing clouds of hydrogen, helium, oxygen, . . . – Complex structure with pillars, swirls, ripples, and cavities Eagle Nebula SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Lagoon Nebula Orion Nebula

History Rendering Authoring Visualizing the Orion nebula • Build a surface for the ionization front – Derived from visual and infrared data • Make it fuzzy – Perturb distance field with turbulence • Project a color-corrected Hubble image through it – Jitter to reduce streaking SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Visualizing the Orion nebula • Add 85 proplyds (protostars), shock fronts, … – Each built in a similar way • Fly around in it all SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Visualizing the Orion nebula SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Visualizing the Orion nebula • 2112 shader nodes in the full scene – Surfaces, turbulence, image projection, transforms, … • Prevoxelized as 86 volumes – 2 Gbytes of multi-resolution data – 40 Gbytes if we didn’t voxelized separately • Ray-caster composited volumes and stars SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

History Rendering Authoring Concluding remarks • What are some directions to explore? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

Data directions At WSCG 2001 • Volume registration “Multimodal Medical Volume Registration Based on Spherical Markers” – Align volumes for compositing • Volume compression – Make it take less space • Volume decimation – Express it well at a smaller size SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego “Geometric Processing of Volumetric Objects” “Towards Continuous Image Representations” “Exploiting Eigenvalues of the Hessian Matrix for Volume Decimation”

Rendering directions At WSCG 2001 • Rendering from compressed data – Smaller data = less bandwidth needed • Large (out-of-core) data rendering – I/O during rendering to get data • Parallel rendering “A New Parallel Volume Rendering Algorithm” “Parallel Ray Tracing with 5 D Adaptive Subdivision” – More processors to get more bandwidth • Hybrid volume + geometry rendering – Keep shapes in their smallest format SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

Authoring directions At WSCG 2001 • Content creation “shaders” “Hypertexturing Complex Volume Objects” – Create using procedures • Volume sculpting / 3 D painting – Create from scratch in an artist-natural way • Volume scene construction – Compose, composite, cut, … • Volume scene compilation – Prevoxelize for best memory/CPU use SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

Tool directions • Work with users! – Put technology into user’s hands … make it a tool – Help create that attention-grabbing event ? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego

Acknowledgements • USA National Science Foundation • USA Department of Energy • San Diego Supercomputer Center – – – – Mike Bailey Steve Cutchin Alex De. Castro Eric Enquist Jon Genetti Mike Houston John Moreland SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego