Interactive Ray Tracing of Pointbased Models Ingo Wald

Interactive Ray Tracing of Point-based Models Ingo Wald SCI Institute, University of Utah wald@sci. utah. edu June 21, 2005 Interactive Point Based Ray Tracing 1

Motivation • Why point-based rendering ? – Many advantages, but no details here … • But: Why point-based ray tracing ? – Actually, two different questions: • Q 1: Why ray tracing for point-based models ? • Q 2: Why use point-based techniques in ray tracing ? June 21, 2005 Interactive Point Based Ray Tracing 2

Q 1: Why Ray Tracing ? • Reason 1: Logarithmic scalability to large models – Very handy for today‘s huge models – But: For PBR not that important (multiresolution)… • Reason 2 : Image quality / shading quality – Ray traced image quality not yet standard in PBG… • But would be nice to have June 21, 2005 Interactive Point Based Ray Tracing 3

Q 2: Why Point-based techniques in Ray Tracing? • Native point-based data (acquisition) – Direct ray tracing avoids tesselation • Much simpler • And for native triangular models ? – Triangular model Point-based model PBRT • Does this make sense ? – Highly controversial issue amongst ray tracing researchers • RT is log. in #tris, anyway. Don‘t need multiresolution… • Many new problems with points as primitives – Consistency, efficiency, loss of information… June 21, 2005 Interactive Point Based Ray Tracing 4

Q 2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples June 21, 2005 Interactive Point Based Ray Tracing 5

Q 2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples – Example 1: Boeing 777 (350 million triangles, 40 GB data) June 21, 2005 Interactive Point Based Ray Tracing 6

Q 2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples – Example 2: Lawrence Livermore isosurface (8 billion voxels) June 21, 2005 Interactive Point Based Ray Tracing 7

Q 2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples – Example 3: Forest landscape (1. 5 billion triangles) June 21, 2005 Interactive Point Based Ray Tracing 8

Q 2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples – Yes, can render all three already today without PB techniques – But: Doesn‘t make sense (>1 G prim. for <1 M pixels ? ) • Too much data: Tiny camera move totally different data – Even though image doesn‘t change at all… • Sample only every 1000 th pixel Severe aliasing – Aliasing: THE most important problem of RTRT today Need multiresolution approach also for RT – Probably best via PB approach • Current results only intermediate step… • … towards long-term goal of multiresolution (PB)RT June 21, 2005 Interactive Point Based Ray Tracing 9

How to ray trace PB models ? • Problem: Zero ray-point hit probability – Rays infinitely thin – Points infitnitely thin, too • Two possible solutions – Use „thick“ rays that have volume (cones, beams, …) • Reconstruct from all points overlapping ray‘s volume – „Grow“ points to have a surface • can then be found by „thin“ ray – Long history of debate on which is better… June 21, 2005 Interactive Point Based Ray Tracing 10

How to ray trace PB models ? • Alternative 1: Growing rays – Experience: Tracing beams very costly (much more than rays) June 21, 2005 Interactive Point Based Ray Tracing 11

How to ray trace PB models ? • Alternative 1: Growing rays – Many un-answered questions • Where exactly is the hit point ? – E. g. , where to start the shadow ray from ? Position A points gathered by beam Hitpoint position ? June 21, 2005 Interactive Point Based Ray Tracing 12

How to ray trace PB models ? • Alternative 1: Growing rays – Many un-answered questions • Where exactly is the hit point ? • How stable is the decision ? Position A points gathered by beam Hitpoint position ? June 21, 2005 Interactive Point Based Ray Tracing 13

How to ray trace PB models ? • Alternative 1: Growing rays – Many un-answered questions • Where exactly is the hit point ? • How stable is the decision ? • Is it consistent ? – E. g. , when viewed from different angle (shadow ray) Position A surface seen from A Position B (Light ? ) Same surface, different viewpoint: Hitpoint position ? Same hitpoint ? Is it in shadow ? June 21, 2005 Interactive Point Based Ray Tracing 14

How to ray trace PB models ? • Alternative 1: Growing rays – Many un-answered questions • Where exactly is the hit point ? • How stable is the decision? • Is it consistent ? Too many open questions June 21, 2005 Interactive Point Based Ray Tracing 15

How to ray trace PB models ? • Alternative 2: Growing points – Define unique surface through points • Always consistent – Most trivial solution: Use disks/ellipses/… • Trivial June 21, 2005 Interactive Point Based Ray Tracing 16

How to ray trace PB models ? • Alternative 2: Growing points – Define unique surface through points • Always consistent – Most trivial solution: Use disks/ellipses/… • Trivial, but shading discontinuities (worse w/ shadows) June 21, 2005 Interactive Point Based Ray Tracing 17

How to ray trace PB models ? • Better: Define smooth surface – Use Adamson/Alexa‘s model • Each point xi has normal ni and radius ri of influence • Influence wi falls off with distance to xi – With wi(xi)=1 and wi(x)=0 for all |x-xi|>ri • Define weighted average position X(x) and N(x) • Define „distance“ function F(x) : = ((x-X(x)). N(x) Implicit fct F(x)=0 is smooth surface – Need: • Useful ri ‘s (not given) • Fast ray/surface intersection framework June 21, 2005 Interactive Point Based Ray Tracing 18

Choosing Splat Radii • Step 1: Choosing the splat radii – Too small Holes in the model – Too large Many regions of influence will overlap (slow) • Use Wu et al. ‘s optimal subsampling technique – Wu [EG 04]: Determine ri such as to • Optimally cover surface • Achieve minimal overlap Optimal solution: Use exactly that technique • Note: Can also use ‚other‘ radii – e. g. , user-specified… – Might just be slower or contain holes June 21, 2005 Interactive Point Based Ray Tracing 19

Intersecting the Surface • Step 2: Determining the splats to intersect with – Each point x always influenced by only few xi C B A June 21, 2005 D Interactive Point Based Ray Tracing 20

Intersecting the Surface • Step 2: Determining the splats to intersect with • Enclose each splat‘s ROI with tight fitting box C B A June 21, 2005 D Interactive Point Based Ray Tracing 21

Intersecting the Surface • Step 2: Determining the splats to intersect with • Enclose each splat‘s ROI with tight fitting box • Build kd-tree over those AA boxes C B A June 21, 2005 D Interactive Point Based Ray Tracing 22

Intersecting the Surface • Step 2: Determining the splats to intersect with • Enclose each splat‘s ROI with tight fitting box • Build kd-tree over those AA boxes C B A June 21, 2005 D Interactive Point Based Ray Tracing 23

Intersecting the Surface • Step 2: Determining the splats to intersect with • Enclose each splat‘s ROI with tight fitting box • Build kd-tree over those AA boxes C B A June 21, 2005 D Interactive Point Based Ray Tracing 24

Intersecting the Surface • Step 2: Determining the splats to intersect with • Enclose each splat‘s ROI with tight fitting box • Build kd-tree over those AA boxes C B A June 21, 2005 D Interactive Point Based Ray Tracing 25

Intersecting the Surface • Step 2: Determining the splats to intersect with • Enclose each splat‘s ROI with tight fitting box • Build kd-tree over those AA boxes • Leaves: Store list of splats whose ROI overlaps cell C - - A A June 21, 2005 B - C B B, C A, B, C D A (right side incomplete) Interactive Point Based Ray Tracing 26

Intersecting the Surface • Step 2: Determining the splats to intersect with • Enclose each splat‘s ROI with tight fitting box • Build kd-tree over those AA boxes • Leaves: Store list of splats whose ROI overlaps cell – KD-Tree: Inherit advantages of fast triangular ray tracing • Fast traversal, occlusion culling, good kd-trees (SAH)… C - - A A June 21, 2005 B - C B B, C A, B, C D A Interactive Point Based Ray Tracing 27

Intersecting the Surface • Step 3: Ray/surface intersection in each voxel – Find distance t : F(t)=0 – Equidistant sampling along t • Have intersection if sgn(F(ti+1)) != sign(F(ti)) • But: Too coarse sampling can miss the surface – Optimizations: Fast SSE implementation (no details) • Quite fast, but: Still main cost factor Further optimize kd-tree to minimize F(x) calls – Need to take closer look at kd-tree June 21, 2005 Interactive Point Based Ray Tracing 28

KD-Tree Post-optimizations • Observation 1: Many full cells don‘t contain surf. – Reason: KD-tree was built over ROI‘s • ROI‘s bound surface, but not closely June 21, 2005 Interactive Point Based Ray Tracing 29

KD-Tree Post-optimizations • Observation 1: Many full cells don‘t contain surf. – Example from before C - - A A June 21, 2005 B - C B B, C A, B, C A A, B, C Interactive Point Based Ray Tracing 30

KD-Tree Post-optimizations • Observation 1: Many full cells don‘t contain surf. – Actual surface only touches few voxels C - - A A June 21, 2005 B - C B B, C A, B, C A A, B, C Interactive Point Based Ray Tracing 31

KD-Tree Post-optimizations • Observation 1: Many full cells don‘t contain surf. – Many non-empty cells that are not necessary C - - A A June 21, 2005 B - C B B, C A, B, C A A, B, C Interactive Point Based Ray Tracing 32

KD-Tree Post-optimizations • Observation 1: Many full cells don‘t contain surf. – Many non-empty cells that are not necessary • Can make quite a difference C - - A A June 21, 2005 B - C B B, C A, B, C A A, B, C Interactive Point Based Ray Tracing 33

KD-Tree Post-optimizations • Observation 1: Many full cells don‘t contain surf. • Fix: Determine min(F()) and max(F()) in each cell Cull cells with min > 0 or max < 0 – Currently: Sample F with random x Can produce false culling (holes) – Better: Interval arithmetic (Not implemented yet) June 21, 2005 Interactive Point Based Ray Tracing 34

KD-Tree Post-optimizations • Observation 2: Most cells too large – Reason: Only split planes at ROI borders – Example from earlier-on: C - - A A June 21, 2005 B - C B B, C A, B, C A Interactive Point Based Ray Tracing 35

KD-Tree Post-optimizations • Observation 2: Most cells too large – Reason: Only split planes at ROI borders – Example from earlier-on: Green regions would suffice… C - A June 21, 2005 - C B B, C A A, B, C A A B Interactive Point Based Ray Tracing 36

KD-Tree Post-optimizations • Observation 2: Most cells too large – Reason: Only split planes at ROI borders – Example from earlier-on: Green regions would suffice… • Fix: Post-shrinking of cells – Cut voxel into slices, cull slices • Same problem as before: False culling… • Important: Not only fewer cell intersections… – Also: Cells much thinner: Finer sampling, fewer artifacts June 21, 2005 Interactive Point Based Ray Tracing 37

KD-Tree Post-optimizations • Observation 3: After shrink/cull, many cells empty – Example from earlier-on: Most cells empty after cull/shrink - - A - C B B, C A, B, C A June 21, 2005 Interactive Point Based Ray Tracing 38

KD-Tree Post-optimizations • Observation 3: After shrink/cull, many cells empty – Example from earlier-on: Most cells empty after cull/shrink B, C C A, B A June 21, 2005 Interactive Point Based Ray Tracing 39

KD-Tree Post-optimizations • Observation 3: After shrink/cull, many cells empty – Example from earlier-on: Most cells empty after cull/shrink B, C C A, B A – Optimization: Post-collapse • Undo splits that don‘t make sense any more June 21, 2005 Interactive Point Based Ray Tracing 40

Results • Overall Framerate: 1 Dual-Opteron 2. 4 GHz, 512 x 512 Model Size Diffuse Phong w/Shadows Iphigenia Head 1 M 6. 8 Not measured Iphigenia Full 1 M 15. 9 Not measured Octopus 465 K 8. 9 7. 5 Dragon 1. 3 M 7. 4 7. 1 David 1. 5 M 11. 2 7. 9 June 21, 2005 Interactive Point Based Ray Tracing 41

Results • Overall Framerate: ~5 -15 fps @ 1 PC • Nice scalability to very complex models – „Many. Iffis“: 24 M points w/ shadows ~2 fps@1 PC June 21, 2005 Interactive Point Based Ray Tracing 42

Results • Overall Framerate: ~5 -15 fps @ 1 PC • Nice scalability to very complex models • High-quality shading – Global Illumination June 21, 2005 Interactive Point Based Ray Tracing 43

Results • Overall Framerate: ~5 -15 fps @ 1 PC • Nice scalability to very complex models • High-quality shading • Problems and limitations: – Sometimes small holes (false culling, undersampling, …) – All is static: Can‘t move points, can‘t change radii – Assumes continuous surface: • Not applicable to random point cloud (forest leaves…) June 21, 2005 Interactive Point Based Ray Tracing 44

Summary & Future Work • Summary – Motivated/Argued for Point-based Ray Tracing – Discussed „thin rays“ vs „thick rays“ issue – Outlined framework for Interactive PBRT • Future work – Performance & Stability • Bounding the surface, interval arithmetic, … – Multiresolution PBRT • In particular, for massively complex models June 21, 2005 Interactive Point Based Ray Tracing 45

Questions ? June 21, 2005 Interactive Point Based Ray Tracing 46

Questions ? June 21, 2005 Interactive Point Based Ray Tracing 47

Q 1: Why Ray Tracing ? • Impact of Shadows and Reflections: Example June 21, 2005 Interactive Point Based Ray Tracing 48

Q 2: Why Point-based techniques in Ray Tracing? • Need multiresolution approach also for RT – Trivial for „nice“ (manifold) meshes (Progressive meshes, …) – But: „problematic“ for real-world models (Forest, Boeing, …) • Should be much easier for point-based representation Eventual goal of (our) PBRT activities – Not ray tracing (nice) point-based models – Rather: Multiresolution ray tracing using a PB approach • Reduce (in-core) data • Fight aliasing – Current activities only first step in that direction… June 21, 2005 Interactive Point Based Ray Tracing 49

Adamson&Alexa‘s model • Observations – Need position and normal for each point assume as given – Need radius of influence for each point – For each x, most wi(x) will be zero • Fast computation: Need to know which. . . – Evaluating F(x) will be costly • Need to minimize F(x) calls • Need to quickly determine where surface can NOT be • Need to minimize number of points contributing to x Need to minimize regions of influence (minimize ri !) June 21, 2005 Interactive Point Based Ray Tracing 50

So: Why Interactive Pointbased Ray Tracing ? Advantages of PBG in particular for ray tracers: • Native point-based data (acquisition) – Avoid tesselation … sure, makes sense. • And for native triangular models ? – Triangular model Point-based model PBRT • Does this make sense ? June 21, 2005 Interactive Point Based Ray Tracing 51

So: Why Interactive Pointbased Ray Tracing ? Advantages of PBG in particular for ray tracers: Highly controversial issue amongst ray tracers – Ray tracing is logarithmic in scene size, anyway • Don‘t need point-based multiresolution capabilities – Triangles very well suited for fast ray tracing • How to (efficiently) intersect points ? – Triangles: well-defined surface • How to consistently intersect points ? • In particular for secondary rays – shadow ray has to see exactly the same surface • Sampling: Loss of information ? Many new problems, PBG advantages over rast. don‘t apply… Where are the advantages ? June 21, 2005 Interactive Point Based Ray Tracing 52

Q 1: Why Ray Tracing • Impact of Shadows and Reflections: Example June 21, 2005 Interactive Point Based Ray Tracing 53

Motivation: Why Point-based Rendering ? • Point-based models become commonplace – Acquisition: Almost all models are point clouds • Direct PBR simplifies 3 D processing pipeline – Today: Acquisition triangulation rendering – Direct PBR: Get rid of (non-trivial) triangulation – Plus: Increasing use in rendering/modelling • Multiresolution, no connectivity/topology, much simpler, … June 21, 2005 Interactive Point Based Ray Tracing 54

Motivation: Why Point-based Rendering ? • Point-based models become commonplace – Acquisition: Almost all models are point clouds • Direct PBR simplifies 3 D processing pipeline – Today: Acquisition triangulation rendering – Direct PBR: Get rid of (non-trivial) triangulation – Plus: Increasing use in rendering/modelling • Multiresolution, no connectivity/topology, much simpler, … • But – Why point-based ray tracing ? – Actually, two different questions: • Q 1: Why ray tracing for point-based models ? • Q 2: Why point-based approach to ray tracing ? June 21, 2005 Interactive Point Based Ray Tracing 55

Outline • Motivation – Why point based ray tracing • Our approach – Thin rays vs thick rays – Fast traversal and intersection • Results • Summary and Future Work June 21, 2005 Interactive Point Based Ray Tracing 56

How to ray trace PB models ? • Alternative 1: Growing rays – Many un-answered questions • Where exactly is the hit point ? • How sensitive is the procedure to exact form of beam ? – E. g. , what if the beam were traced a little bit further ? Position A Hitpoint position ? June 21, 2005 points gathered by beam Same surface ? Interactive Point Based Ray Tracing 57