INFORMATIK An Efficient SpatioTemporal Architecture for Animation Rendering

INFORMATIK An Efficient Spatio-Temporal Architecture for Animation Rendering Vlastimil Havran, Cyrille Damez, Karol Myszkowski, and Hans-Peter Seidel Max-Planck-Institut für Informatik, Saarbrücken, Germany

Motivation INFORMATIK In the traditional approach to rendering of high quality animation sequences every frame is considered separately. • The temporal coherence is poorly exploited: Ø redundant computations. • The visual sensitivity to temporal detail cannot be properly accounted for: Ø too conservative stopping conditions, Ø temporal aliasing.

Goal INFORMATIK Developing an architecture for efficient rendering of high-quality animations which better exploits the spatio-temporal coherence between frames: • Visibility: Multi-frame ray tracing • Global illumination: Bi-directional Path Tracing extended to re-using samples between frames • Texturing and shading: sharing information between frames • Motion blur: conservative computation in 2 D image space • Cache use: coherent patterns of access to data structures in memory (along motion compensation trajectories) • Memory requirements: unlimited image resolution

Previous Work INFORMATIK • Industrial solutions such as Photorealistic. Renderman, Maya Rendering System, or Softimage – frame–by–frame computation – postprocessing of animation to decrease temporal aliasing

Previous Work INFORMATIK Temporal Coherence in Ray Tracing • Coherence of shaded pixels – Interpolation between frames [Maisel and Hegron 92] – Reprojection [Adelson and Hughes 95] – 4 D radiance interpolants [Bala et al. 99] • Coherence in acceleration data structures – Space-time solutions [Glassner 88, Groeller 91] – Static vs. dynamic objects [Besuievsky and Pueyo 01, Lext and Moller 01, and Wald et al. 02]

Previous Work INFORMATIK Temporal Coherence in Global Illumination • Coherence of illumination samples – Render cache [Walter et al. 99] – Shading cache [Tole et al. 02] – Reusing photon hit points [Myszkowski et al. 01] – Selective Photon Tracing [Dmitriev 02] • Coherence in visibility computation – Global lines [Besuievsky and Pueyo 01] • Coherence in generation of random numbers – Random number sequence associated with each light transport path [Lafortune 96, Jensen 01, Wald et al. 02]

Space-Time Architecture: Principle INFORMATIK • Compute samples using a variant of Path-Tracing • Pixels color = mean of sample values • 2 types of samples: – Native samples: 8 Expensive 8 Computed from scratch – Recycled samples: 8 Cheap 8 Based on previous computations of native samples (using reprojections)

Motion Compensation • Camera and object motion compensation • Memory access coherence …. . INFORMATIK

Bi-directional Path Tracing INFORMATIK

Camera Motion INFORMATIK

Occluded Connection INFORMATIK

Path Change INFORMATIK

Multi-Frame Ray Tracing INFORMATIK • Aggregate queries – Single ray geometry – Results possible for all frames in [f(i-R-S)’f(i+R)] • Two types of visibility queries – Ray shooting – Visibility between two points • Three settings of visibility queries – Exact result for single frame f(i) – Exact for single frame f(i) + validity for all other frames – Exact results for all frames in [f(i-R)’f(i+R)]

Multi-Frame Ray Tracing INFORMATIK Construction of Spatial Data Structure • • Instantiation of dynamic objects for range of frames Static versus dynamic objects separation Construction of global kd-tree over static objects Hierarchical clustering over dynamic objects Construction of kd-trees for clusters of objects Insertion of kd-trees for clusters into global kd-tree leaves Refinement of global kd-tree leaves + empty space cutting off (efficiency improvement methods) Additional techniques used • Mail boxes (cache) for ray transforms, objects, and kd-trees • Frame masks for inserted kd-trees and shadow cache

The Animation Buffer INFORMATIK • Iterates over all pixels in S consecutive frames • If more samples are required – Compute a native sample for frame fi – Reproject it and recycle it for all frames in [f(i-R)’ f(i+R)] • S+2 R frames are kept in the buffer R S R

The Animation Buffer INFORMATIK • Iterates over all pixels in S consecutive frames • If more samples are required – Compute a native sample for frame fi – Reproject it and recycle it for all frames in [f(i-R)’ f(i+R)] • S+2 R frames are kept in the buffer S Saving to a disk R S R

Shading Computation INFORMATIK • A simplified version of Render. Man Shading Language • Each shader decomposed into – View-independent component 8 re-usable, shared between frames – View-dependent component 8 recomputed for each frame

Motion Blur INFORMATIK • Accuracy & Quality – The sample point is considered for multiple frames 8 In other frame-by-frame architectures the motion of objects must be computed explicitely by additional samples. – Temporal changes in shading are properly accounted for 8 Difficult in other architectures • Efficiency – 2 D computation

Results INFORMATIK Speedup for bi-directional path tracing • Moving camera, moving objects: 7. 7 • Moving objects only: 13. 3 • Moving camera only: 8. 8 Time per frame 240 – 415 sec. Proportion of native samples 2. 4 - 4. 7 % Cost of native samples (profiler) 44 - 64 % of the whole computation time. Disc caching overhead (for 1% of main memory used) 10 % slowdown

Motion blur renderings Time per frame 120 sec. INFORMATIK Time per frame 150 sec. Motion Blur - computational overhead 25%

Videos INFORMATIK

Conclusions INFORMATIK • We presented an efficient architecture for rendering of highquality animations • In our architecture path tracing, texturing and shading, motion blur can be efficiently computed. • Our architecture is efficient in particular for scenes in which a limited number of objects moves locally in the scene and camera motion is slow. – For more complex motion scenario the animation segment length can be reduced which in the limit may boil down to the traditional frame-by-frame computation. • Data structures handling dynamic objects require additional memory, which is acceptable on modern computers. • The memory overhead involved in storing multiple images is negligible due to efficient buffering.

Future Work INFORMATIK • Reusing samples for many pixels in the same frame as suggested in [Bekaert et al. 2002, Szirmay-Kalos 2002] • Improving the efficiency of visibility tests for reprojected samples using a shaft culling approach. • Skipping some visibility tests based on the spatiotemporal coherence of neighboring samples as implemented in the Maya Rendering System [Sung et al. 2002]

Thanks! INFORMATIK • Polina Kondratieva for rewriting Render. Man Shaders • Markus Weber for help with preparing the scenes • Partial support of IST-2001 -34744 project Real. Reflect • All anonymous reviewers for their comments

Space-Time Architecture: Reprojection INFORMATIK

Rendering animations – ray tracing with shaders INFORMATIK Speedup ray tracing • Moving camera, moving objects: 2. 6 Time per frame 770 sec. ( deterministic reflections must be recomputed! )