Exploiting Temporal Coherence in Ray Casted Walkthrougs Vlastimil

Exploiting Temporal Coherence in Ray Casted Walkthrougs Vlastimil Havran, Jiří Bittner and Hans-Peter Seidel AG 4, MPI Informatik, Saarbruecken, Germany Institute for Computer Graphics, Vienna University of Technology, Austria

Talk Outline Introduction – ray shooting and ray casting – temporal coherence New Algorithm Results and Demo Conclusions and Future Work

Introduction: Ray Shooting Task: Given a ray, find out the first object intersected. A Input: a scene and a ray B C ray D Output: the object C and signed distance

Ray Casting Image pixel rays

Temporal Coherence “Weak Definition”: a similarity between subsequent images in the animation. Exploiting temporal coherence: speed up computation. Algorithm Categories A – static camera, moving objects B – moving camera, static objects (walkthrough) C – moving camera, moving objects Many coherence classes exist. – ray coherence, spatial coherence, temporal coherence, traversal coherence, etc. (see Eduard Groeller’s Ph. D thesis, TU Vienna)

Previous and Related Work Glassner, 88: Spacetime ray tracing for animation. Badt Jr, 88: Two algorithms for taking advantage of temporal coherence in ray tracing. Groeller and Purgathofer, 91: Using Temporal and spatial coherence for acceleration of animation sequences. Sudarsky, 93: Exploiting Temporal Coherence in Animation Rendering. A Survey. Adelson, 95: Generating exact-raytraced animation frames by reprojection. Walter et al. , 99: Interactive rendering using Render Cache, EGWR’ 99. Reinhard et al. , 01: Parallel Point Reprojection. Lin. Q et al. , 00: Frame Coherent Volume Rendering.

New Algorithm: Overview Main Idea: - Compute a single intersection of the ray with an object, if possible - If not possible -> ordinary ray shooting algorithm First frame: use ordinary ray shooting, remember intersection points in 3 D space Next frames: reproject points, check if we can decide on intersection objects Properties: intersection points and surface normals compute correctly (unlike in Render. Cache etc. )

New Algorithm: Data Structures POS = array in 2 D {point in 3 D, object ID, distance} (Points in Object Space) AIP = array in 2 D {object, distance, count} (Auxiliary Image Plane) Size of AIP and POS = width * height First frame: use an ordinary ray shooting algorithm, remember the points where ray hit objects (store it to an array POS)

Reprojection phase “Next frame(s) – using reprojection”: for each pixel (xx, yy) do – reproject a point (POS(xx, yy)) from previous frame to AIP at pixel (x, y), compute approximate distance tp. – for each reprojected point from POS store distance tp into neighbourhood 3 x 3 of (x, y) at AIP. Overwrite farther already stored points by closer ones. If the reprojected points are from the same object, increment AIP(X, Y). count, for all 9 pixels X=x-1, x, x+1, Y=y-1, y, y+1. end-for

Ray Casting based on Reprojection “Next frame(s) – ray casting”: for each pixel (x, y) (x=0. . . width), (y=0. . . height) do if AIP(x, y). count > THRESHOLD then ray-cast ray(x, y) to AIP(x, y). OBJ end-if if (intersection was not found) then use an ordinary ray shooting algorithm. end-if end-for THRESHOLD = 5, 6, 7, 8, or 9

Dilatation of Objects (footprint) Properties: reprojection using 3 x 3 neighbourhood corresponds to dilatation on the image plane.

Scene Example RED - reprojection successfull GREEN - reprojection fails WHITE - regular resampling

Reprojection Failures Question: When reprojection cannot be used ? Answer: appearance of new objects occluding previously visible objects a) that were outside viewing frustum (view frustum errorr) b) that were occluded (occlusion error) c) that were too small (undersampling error)

View Frustum Error Moving backward. View Frustum Error

Occlusion Error Occluded objects that appear. Occlusion Error

Solution to Occlusion Error: Epipolar Geometry

Correct Reprojection Algorithm Check possible occlusion on the fly.

Small Approaching Objects Problem: Let us have a scene with polygon A that in reprojection has big footprint. Some small objects of current subpixel size are placed in front of A and are not currently visible. Question: What happens when we enclose the polygon and use only and only the reprojection ? Answer: Small objects are missing!

Resampling Order – maximize the probability that small object of subpixel size will be detected when moving camera straight forward. Goal: minimize discrepancy of resampling pattern in spatio-temporal domain

Further Improvements Shifting Ray Origin - when reprojection fails, we can move the ray origin along the ray path to avoid traversing empty space. Handling background - sphere enclosing the whole scene, use as any other object (always overwritten). Double Reprojection - edges between connected objects in 3 D must be recomputed. Use two AIP arrays, keep two closest objects information. Collision detection - between camera and the environments based on signed distance is quite simple to implement.

Note on Reprojection Efficiency How many arithmetical operations to reproject one point from 3 D space to image plane ? 8 x (+), 3 x (-), 12 x (multiplication), 1 x (division), 1 x. SQRT, and 2 x. IF. Note: Image-based incremental reprojection techniques by Mc. Millan and Mark are not applicable, since we need signed distance!

Walkthrough Demo

Results: HW independent profiling ORSA: ordinary ray shooting algorithm REPR: ray casting with reprojection IRSA: ideal ray shooting algorithm N_IT N_TS T_REN FPS ORSA 9. 23 52. 0 701 sec 1. 17 REPR 3. 37 12. 2 478 sec 1. 72 IRSA 0. 98 0. 0 256 sec 3. 22 ORSA 3. 02 44. 5 892 sec 1. 65 REPR 1. 89 21. 9 789 sec 1. 89 IRSA 0. 98 0. 0 355 sec 4. 19 ORSA 3. 79 58. 1 979 sec 1. 19 REPR 2. 73 26. 0 838 sec 1. 39 IRSA 0. 999 0. 0 374 sec 3. 11 Scene A, 85 k objects, 825 frames Scene B, 626 k objects, 1489 frames Scene C, 112 k objects, 1165 frames

Results Summary – tested on three different scenes. – increased speedup (with shading, 512 x 512): scene A – 1. 17 fps to 1. 72 fps (85 x 10^3 objects) scene B – 1. 65 fps to 1. 89 fps (626 x 10^3 objects) scene C – 1. 19 fps to 1. 39 fps (112 x 10^3 objects) – 11. 1% pixels computed by regular resampling – 78. 9% pixels computed by reprojection – 10% pixels computed by ordinary ray shooting algorithm Question: is this interesting and valuable speedup ?

Results: Profiling (scene A) ORSA REPR IRSA Ray. Shooting Function 63. 4% 25. 5% 18. 7% Compute Color 28. 7% 41. 4% 63. 6% Ray Initialization 1. 1% 1. 7% 2. 9% Extra ray-object inters. ------ 3. 2% ------ Reprojection phase ------ 14. 7% ------ Rest of Computation 6. 7% 13. 5% 15. 3% Profiler timing [sec] 1001 676 442 281 sec REPR/ORSA: Visibility speedup = 1. 81 (45% savings in time) IRSA/ORSA: Visibility speedup = 4. 44 (77% savings in time) !!!

Conclusions – conservative ray casting algorithm based on temporal coherence using reprojection aimed at solving hidden surface removal (visibility). (previous algorithms based on reprojection interpolate colour information from 3 D space). – possible use in online mode (many reprojection techniques for ray-casting/tracing are offline). – good use of temporal coherence, where temporal coherence exists (80% pixels computed by single ray-object intersection).

Future Work – use in context of global illumination methods. – generalisation to scenes with moving objects. – instead of using objects ID, use a pointer to the cells of spatial subdivision, thus decreasing the dependency on visual complexity. – extension to visibility for direct illumination is possible (point light source exactly and area light sources with good quality).