Photorealistic Animation Rendering with Energy Redistribution YuChi Lai

Photorealistic Animation Rendering with Energy Redistribution Yu-Chi Lai 賴祐吉 University of Wisconsin - Madison Yu-Chi Lai

Agenda • Introduction • Physically-based rendering methods • Population Monte Carlo energy redistribution • Future works Yu-Chi Lai

Goal and Applications • Goal: generate realistic animations • Applications – Movie – Interactive entertainments • Computer games • Virtual reality walk-throughs – Light Engineering – Etc. From Day After Tomorrow Yu-Chi Lai

Applications Animation: Kristensen et al. Grand Theft Auto 4 IGNEntertainment Yu-Chi Lai

Modeling and Simulating Appearance Yu-Chi Lai

Agenda • Introduction • Background • Population Monte Carlo energy redistribution • Future works Yu-Chi Lai

Rendering Equation • Reflection Equation Light reflected Sum BRDF Incoming light incoming light reflected at the point • Energy Balance Equation • Difficulty: Lr appear in both sides of equation => Fredholm equation of the second type Yu-Chi Lai

Physically-based Rendering • Render images according to physical principles – Radiosity: finite element – Ray-tracing: based on Monte Carlo integrations • Unbiased: path tracing, bidirectional path tracing, metropolis light transport, energy redistribution path tracing and so on. • Biased: irradiance caching, photon mapping, and so on. From Jenson et al. Yu-Chi Lai

Path Integral for General Integration • Measurement equation: • Path Integral: – : a path – Ω: path space – : area product measurement – : contribution of a path • Difficulty: high dimensions, computation grow exponentially when using deterministic integrations Yu-Chi Lai

Monte Carlo Algorithms • We can estimate the integral by generating a set of samples: • Different ways to generate samples: path tracing, light tracing, bidirectional, … Yu-Chi Lai

Issues with MC methods • Computationally expensive • Variance reduced slowly with number of samples • Reuse path samples From PBRT Book Yu-Chi Lai

Frame-by-Frame Photorealistic Animation Rendering • Takes a long time to generate – Several hours per frame is industry standard • Temporal noise – Flickering – Shimmering From Max-Planck Institute, German Yu-Chi Lai

Physically-based Animation Rendering • Right now the research is mainly from German’s MPI, USA’s UCSD, Standford, and Cornell. • Parallel independently rendering => temporal coherence – Use multiple processors to do independent ray tracing [Warld’ 01] – Coherently trace rays: Kdtree, Grids, and bounding volume hierarchy (BVH) [Wald’ 01 b, Wald’ 07 a] – Efficient update schemes of acceleration structure [Popov’ 06, Lauterbach’ 06, Yoon’ 07] – Implement global illumination in GPU architecture including radiosity, photonmapping, ray tracing, …[Purrel’ 02, Purell’ 03] • Copy the samples across the frames => validity – Irradiance Reuse. [Martin’ 99, Tawara’ 02, Tawara’ 04, Smyk’ 05] – Photon shooting (next slide) Yu-Chi Lai

Photon Shooting • Photon shooting algorithm [Myszkowski’ 01, Dmitriev’ 02, Weber’ 04] • Instant Radiosity [Keller’ 97, Wald 0’ 2 a, Laine’ 07] • Light cut algorithm: [Walter’ 05, Hansan’ 07, Hansan’ 08] – Create a set of point light sources in the entire animation – Cluster the light sources into a tree according to the perceptual metrics in temporal and spatial domain Animation by Hansan et al Yu-Chi Lai

Reuse Path Samples • Camera is allowed to move [Briere’ 96, Murakami’ 89, Jevans’ 92, Sequin’ 89, Bala’ 99, Fernandez’ 00, Havran’ 03, Mendez’ 06] • Light source is allowed to move or change properties: [Sbert’ 04 a, Sbert’ 04 b , Sbert’ 04 C, Ghosh’ 06] Animation by Havran et al Animation by Ghosh et al Yu-Chi Lai

Grand Challenge • Efficiently render entire animation – – Do parallel computation in multi-processors or GPU. Do coherent ray tracing. Efficiently update the acceleration structure. Reuse samples from previous computation • Reduce the temporal artifacts – Explore the temporal coherence among paths – Reuse the computation results. • Perceptual evaluation of the rendering results. – Evaluate the strength of each algorithm. – Allow us to distribute computation to perceptual important features. Yu-Chi Lai

Agenda • Introduction • Background • Population Monte Carlo energy redistribution • Future works Yu-Chi Lai

Challenges • How to adjust the sampling parameter according to path or scene properties? • How to concentrate more computation on paths that are important without introducing bias? • How to reuse paths temporally? • How to handle the huge computation for animation rendering? Yu-Chi Lai

Markov Chain Monte Carlo • Write the integrand as: – – – : the sensor measurement for pixel j of frame k : represents all other factors : represents the radiant energy passing through the image sweep – Create the distribution of paths in animation proportional to the contribution Yu-Chi Lai

Metropolis and Energy Redistribution Path Tracing • Generate a sequence of paths: where path is generated according to Yu-Chi Lai

Overview of Population Monte Carlo Energy Redistribution Animation Rendering System Yu-Chi Lai

PMC-ER in Each Frame Process Yu-Chi Lai

Detail • Preprocess: collect the information for the following computation • Energy redistribution: distribute the energy to similar paths by using spatial and temporal perturbations. • Resampling: – Eliminate paths from the population. – Generate replaced paths to determine the area of exploration. – Adjust the rendering parameters Yu-Chi Lai

Adjust Sampling Parameters (EGSR 07) • Different regions have different details • We would like to adjust sampling parameters accordingly Low detailed regions: large distribution radius High detailed regions: smaller distribution radius Yu-Chi Lai

Population Monte Carlo Algorithm To estimate integral At t iterations, a PMC estimator of the integral is given by Yu-Chi Lai

Yu-Chi Lai

Kernel Function and Adaptation • Kernel Function • Adapt values: to choose proper perturbation radius – Initialize them to constant values when a path is generated – After each successful perturbation, the acceptability is labeled with the perturbation radius, and the path, i – By using Yu-Chi Lai

Adaptation Results • Color represents perturbation radius – Red: 5, Green: 10, Blue: 50 Yu-Chi Lai

Cornell Box PMC-ER ERPT Yu-Chi Lai

Room Scene PMC-ER ERPT Yu-Chi Lai

Concentrate More Computation on Certain areas (ISVC) • Stratified exploration of the image plane – The importance of regions on the image are not perceptually the same. – Some types of paths are visually more important and harder to find. PMC-ER 4 SPPs PMC-ER 8 SPPs Yu-Chi Lai

Regeneration (I) • Perceptually distributed pixel positions according to which is the radiance sample variance in each pixel • Weighting Yu-Chi Lai

Regeneration (II) • Use light tracing to generate a valid light paths • Link each surface vertex to the camera to form a set of valid paths • Evaluate whether it is a caustics path • Weighting Yu-Chi Lai

Results 4 SPPs+Reg 4 SPPs 8 SPPs Yu-Chi Lai

Results 6 SPPs+Reg 6 SPPs 12 SPPs 18 SPPs Yu-Chi Lai

Problem in Frame-by-Frame Rendering • Each frame takes long time. – Parallel rendering with condor system • Temporal artifacts: temporal perturbation Yu-Chi Lai

Temporal Perturbation • Update the position of diffuse vertices • Reconstruct the specular sub-path • Check the validity of the path Yu-Chi Lai

Cornell Box Frame-By-Frame With Temporal Perturbation Yu-Chi Lai

Chess Body Frame-By-Frame With Temporal Perturbation Yu-Chi Lai

Room Scene Frame-By-Frame With Temporal Perturbation Yu-Chi Lai

Chess Board Frame-By-Frame With Temporal Perturbation Yu-Chi Lai

Basement Frame-By-Frame With Temporal Perturbation Yu-Chi Lai

Contributions • A new rendering algorithm based on PMC framework • • – Correlatedly explore important paths – Automatically adjust energy redistribution area according to the information collected in previous iterations – Elimination-regeneration to achieve ergocity and adjust the exploring area according to paths’ remaining energy New lens perturbation method – Increase the caustics perturbation success rate – Ease the control of caustics perturbation on the image plane New regeneration methods – Concentrate the computation on perceptual important regions – Concentrate the computation on perceptual important types of paths. Temporal perturbation method: exploration the temporal coherence among paths A algorithm allows us to render a scene in parallel Yu-Chi Lai

Limitations • Human observation is the evaluation tool for animation quality. • Dark regions are hard to get the chance to be explored and thus are relatively noisy. Although it is hard to notice in single image, this becomes an issues because human perception is very sensitive to this kind of temporal inconsistency. • Temporal perturbations in each condor process will create a large set of temporal files for related frames. Transferring and updating data in condor daemon process involves a large number of disk IOs. • Our variance-sample distribution criterion is based on variance of sample radiances but this did not represent the result after energy redistribution. • We separate perturbations into two types, temporal and spatial perturbations, and this makes control harder and the initial probing samples for each perturbation is relative low. • Limit the light to area light sources and the efficiency goes down when the number of lights goes up Yu-Chi Lai

Agenda • Introduction • Background • Population Monte Carlo Energy Redistribution • Future Works Yu-Chi Lai

Future Works • Animation quality evaluation algorithms – Current available perceptual animation quality evaluation algorithm is for video compression. – Adjust the quality perceptual evaluation algorithm for Monte Carlo algorithms • Construct a temporal filter based on result of the temporal perturbations: – The result of temporal perturbation can create the relations among pixels in different frames, if we can use this relation information to create a temporal filter accordingly, we should be able to reduce the temporal artifact and iterations to generate a smooth result • Develop an perturbation which perturb in spatial domain randomly but perturb in a fixed regions in temporal domain deterministically Yu-Chi Lai

Future Works • Apply Population Monte Carlo with path tracing into animation rendering using environment map lighting. – A path has the form of L(D|S)C. – For each path of current frame, temporally trace the path with a proper temporal perturbation algorithm to enhance the temporal coherence among frames. • Spread the photon collection positions in photon splatting with Metropolis: – Generate a set of collection positions – Use temporal perturbation to correlatedly generate new photon collection positions from the previous frame. • Use the light cut or light clusters algorithm to solve the many light problem. Yu-Chi Lai

Future Research • Explore the parallel ability in GPU – Energy redistribution path tracing are naturally parallel. => transform the ERPT algorithms onto GPU – Explore the research possibility in environment map lighting and shadow generation. • Construction and application of flock tiles: – If we can find a common set of temporal and spatial boundary conditions for setting up a tile. – We can use the constraint simulation to simulate the inner agents’ motion according to flock rules – The animation tiles can be used to construct a seamless animation such as a large crowd in a city, a school of fishes, a flock of birds or the traffics in a city. Yu-Chi Lai

Publications • Computer Science – – Yu-Chi Lai, Steven Chenney, Shaohua Fan “Group Motion Graphs”, Eurographics/SIGGRAPH Symposium on Computer Animation 2005, pp. 281– 290. Yu-Chi Lai, Shaohua Fan, Stephen Chenney, and Charles Dyer, “Photorealistic Image Rendering with Population Monte Carlo Energy Redistribution”, Eurographics Symposium on Rendering, 2007, pp. 287 -296. Yu-Chi Lai, Feng Liu, Li Zhang, and Charles Dyer, “Efficient Schemes for Monte Carlo Markov Chain Algorithms in Global Illumination”, Proc. 4 th International Symposium on Visual Computing, 2008. Yu-Chi Lai, Feng Liu, and Charles Dyer, “Physically-based Animation Rendering with Markov Chain Monte Carlo”, (submit to Eurographics Symposium on Rendering 2009) Yu-Chi Lai

Publications • Computer Science (Others) – – – Shaohua Fan, Stephen Chenney and Yu-Chi Lai “Metropolis Photon Sampling With Optical User Guidance”, Eurographics Symposium on Rendering, 2005, pp. 127 -138 Yu-Chi Lai, Stephen Chenney, Shaohua Fan, “Data-Driven Group Animation”, Technical Report, Department of Computer Sciences, University of Wisconsin. Madison, 2005 Yu-Chi Lai, Shaohua Fan, and Charles Dyer, “Population Monte Carlo Path Tracing”, Technical Report, Department of Computer Sciences, University of Wisconsin-Madison, 2006 Shaohua Fan, Stephen Chenney, Bo Hu, Kam-Wah Tsui and Yu-Chi Lai, “Optimum Control Variate”, Computer Graphics Forum, Vol. 25, No. 3, pp. 351358, 2006. Yu-Chi Lai, Shaohua Fan, Feng Liu, Brandom Smith, Stephen Chenney, Li Zhang and Charles Dyer, “Population Monte Carlo Sampler for Rendering”, Technical Report, Department of Computer Sciences, University of Wisconsin-Madison, 2009 Yu-Chi Lai

Publications • Electrical and Computer Engineering – – – Lai, Y-C, D. Haemmerich, et al (2003). “Lesion Size estimator at different common locations with different tip temperature during cardiac radio-frequency ablation”, IEEE Transactions on Biomedical Engineering. Lai, Y-C et al (2009). “Guidelines for predicting lesion size at different common locations with different temperature during in vitro radio-frequency ablation”, (Prepare) Lai, Y-C et al (2009). “The effects of insertion depth and meat dimension on the lesion formation during cardiac radio-frequency ablation”, (Prepare) Yu-Chi Lai

Publications • Chapters in books – “Biomedical Electrode”: John Webster ed. , Ch 8 Electrocardiogram – “Introduction to biometric identification”, Willians Tompkin ed. , Ch 12 Biometric standards, testing, and evaluation – “Tissue ablation: devices and procedures”, John G. Webster ed. , Ch. 28. Lesion size estimator Yu-Chi Lai

Work & Research Experience – Research associate, Department of Electrical Engineering, National Cheng-Kong (1998 – 2000) • Lead a group to write a program of satellite orbit simulation, and also participating in coding the drivers for CD-ROM and DVD, and GPS’s map display system. – Research Assistant, Department of Electrical and Computer Engineering, UW-Madison (2001 – 2004) – Research Assistant, Department of Computer Science, UW-Madison (2005 -2006) – Summer Internship, Raven Software (2007 summer) – Teaching Experience (1994 to 1996 in National Taiwan University, 2003 – 2008 in UW-Madision) Yu-Chi Lai

Thank You Question? More Details: www. cs. wisc. edu/~yu-chi Yu-Chi Lai

Reference • • [Debevec’ 96]: P. Debevec, C. Taylor, and J. Malik. Modeling and rendering architecture from photographs. SIGGRAPH 96, pages 11– 20. [Debevec’ 98]: Paul E. Debevec, Yizhou Yu, and George D. Borshukov. Efficient viewdependent image-based rendering with projective texture-mapping. Eurographics Rendering Workshop 1998. [Chen’ 95]: CHEN, S. E. Quicktime VR - an image-based approach to virtual environment navigation. In Computer Graphics, Annual. Conference Series, 1995, pp. 29– 38. [Gorler’ 96]: Steven J. Gortler, Radek Grzeszczuk, Richard Szeliski, and Michael F. Cohen. The lumigraph. SIGGRAPH 96, pages 43– 54. [Levoy’ 96]: M. Levoy and P. Hanrahan. Light field rendering. SIGGRAPH 96, pages 31– 42. [Adelson’ 91]: ADELSON, E. H. , AND BERGEN, J. R. The plenoptic function and the elements of early vision. In. Computational. Models of Visual Processing, Landy and. Movshon, Eds. MIT Press, Cambridge, Massachusetts, 1991, ch. 1. [Liu’ 08]: Feng Liu, Yuhen-Hu and Michael Gleicher. Discovering Panoramas in Web Videos. ACM Multimedia 2008. Oct. 2008. Yu-Chi Lai

Reference • • • [Szeliski’ 06]: R. Szeliski. Image alignment and stitching: a tutorial. Found. Trends. Comput. Graph. Vis. , 2(1): 1{104, 2006. [Diefenbach’ 94]: Diefenbach, P. J. & Badler, N. I. : Pipeline Rendering: Interactive refractions, reflections and shadows Displays: Special Issue on Interactive Computer Graphics, 1994, 15, 18 -0 [Diefenbach’ 96]: P. J. Diefenbach. Pipeline Rendering: Interaction and Realism through Hardware-Based Multi-Pass Rendering. Ph. D. thesis, University of Pennsylvania, 1996. 11 [Diefenbach’ 97]: P. J. Diefenbach and N. I. Badler. : Multi-pass pipeline rendering: Realism for dynamic environments. In 1997 Symposium on Interactive 3 D Graphics, pages 59– 70. ACM SIGGRAPH, 1997. [Wojdala’ 94]: Wojdala, A. ; Gruszewski, M. & Dudkiewicz, K. : Using harware texture mapping for efficient image synthesis and walkthrough with specular effects Machine Graphics and Vision, 1994, 137 -151 [Voorhies’ 94]: Voorhies, D. & Foran, J. Reflection vector shading hardware, SIGGRAPH '94: Proceedings of the 21 st annual conference on Computer graphics and interactive techniques, ACM, 1994, 163 -166 Yu-Chi Lai

Reference • • • [Greene’ 86]: Greene, N. : Environment mapping and other applications of world projections, IEEE Comput. Graph. Appl. , IEEE Computer Society Press, 1986, 6, 21 -29 [Haeberli’ 90]: Haeberli, P. & Akeley, K. : The accumulation buffer: hardware support for high-quality rendering, SIGGRAPH '90: Proceedings of the 17 th annual conference on Computer graphics and interactive techniques, ACM, 1990, 309 -318 [Heidrich’ 99]: Heidrich, W. & peter Seidel, H. : Realistic, hardware-accelerated shading and lighting, Proceedings of the 26 th annual conference on Computer graphics and interactive techniques, 1999, 171 -178 [Kautz’ 00]: Kautz, J. ; pau Vazquez, P. ; Heidrich, W. ; peter Seidel, H. & Udg, A. : A unified approach to prefiltered environment maps, Springer, 2000, 185196 [Sloan’ 02]: Sloan, P. -P. ; Kautz, J. & Snyder, J. : Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments, SIGGRAPH '02: Proceedings of the 29 th annual conference on Computer graphics and interactive techniques, ACM, 2002, 527 -536 [Kristensen’ 05]: Kristensen, A. W. ; Akenine-Möller, T. & Jensen, H. W. : Precomputed local radiance transfer for real-time lighting design, ACM Trans. Graph. , ACM, 2005, 24, 1208 -1215 Yu-Chi Lai

Reference • • • [Ng’ 03]: Ng, R. ; Ramamoorthi, R. & Hanrahan, P. : All-frequency shadows using non-linear wavelet lighting approximation, ACM Trans. Graph. , ACM, 2003, 22, 376 -381 [Ng’ 04]: Ng, R. ; Ramamoorthi, R. & Hanrahan, P. : Triple product wavelet integrals for all-frequency relighting, SIGGRAPH '04: ACM SIGGRAPH 2004 Papers, ACM, 2004, 477 -487 [Hasan’ 06]: Hašan, M. ; Pellacini, F. & Bala, K. : Direct-to-indirect transfer for cinematic relighting, ACM Trans. Graph. , ACM, 2006, 25, 1089 -1097 [Ben’ 06]: Ben-Artzi, A. ; Overbeck, R. & Ramamoorthi, R. : Real-time BRDF editing in complex lighting, ACM Trans. Graph. , ACM, 2006, 25, 945 -954 [Nimeroff’ 96]: Nimeroff, J. ; Dorsey, J. & Rushmeier, H. : Implementation and Analysis of an Image-Based Global Illumination Framework for Animated Environments, IEEE Transactions on Visualization and Computer Graphics, 1996, 2, 283 -298 [Myszkowski’ 99]: Myszkowski, K. ; Tawara, T. ; Akamine, H. & Seidel, H. -P. : Perception-informed accelerated rendering of high quality walkthrough sequences, Proc. of the 10 th Eurographics Workshop on Rendering, 1999, 5 -18 Yu-Chi Lai

Reference • • • [Chen; 90]: Chen, S. : Incremental Radiosity: An Extension of Progressive Radiosity to an Interactive image Synthesis System, SIGGRAPH '90: Proceedings of the 23 th annual conference on Computer graphics and interactive techniques, 1990, 135 -144 [George’ 90]: George, D. ; Sillion, F. & Greenberg, D. : Radiosity Redistribution for Dynamic Environments, IEEE Computer Graphics and Applications, 1990, 26 -34 [Muller’ 95]: Muller, S. ; Kresse, W. & Gatenby, N. : A radiosity approach for the simulation of daylight, In Eurographics Rendering Workshop), 1995, 251 -262 [Forsyth’ 94]: Forsyth, D. ; Yang, C. & Teo, K. : Efficient radiosity in dynamic environments, Proceedings of the Fifth Eurographics Workshop on Rendering, 1994, 221 -231 [Martin’ 99]: Martin, I. ; Pueyo, X. & Tosti, D. : A framework for animatio in global illumination, Univ. of Girona, 1999 [Drettakis’ 97]: Drettakis, G. & Sillion, F. X. : Interactive update of global illumination using a line-space hierarchy, SIGGRAPH '97: Proceedings of the 24 th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co. , 1997, 57 -64 Yu-Chi Lai

Reference • • • [Damez’ 99]: Damez, C. & Sillion, F. : Space-time hierarchical radiosity, In Rendering Techniques � 9, Springer, 1999, 235 -246 [Shaw’ 97]: Shaw, E. : Hierarchical Radiosity for Dynamic Environments, Computer Graphics Forum (Proceedings Eurographics 1997), 1997, 16, 107118 [Keller’ 97]: Keller, A. : Instant Radiosity, SIGGRAPH '97: Proceedings of the 21 st Annual Conference on Computer Graphics and Interactive Techniques, 1997, 49 -56 [Wald’ 02 b]: Wald, I. ; Kollig, T. ; Benthin, C. ; Keller, A. & Slusallek, P. : Interactive global illumination using fast ray tracing, EGRW '02: Proceedings of the 13 th Eurographics workshop on Rendering, Eurographics Association, 2002, 15 -24 [Wald’ 02 a]: Wald, I. ; Kollig, T. ; Benthin, C. ; Keller, A. & Slusallek, P. , Interactive Global Illumination, Proc. of the 13 th Eurographics Workshop on Rendering, 2002, 15 -24 Yu-Chi Lai

Reference • • • [Segovia’ 06]: Segovia, B. ; Iehl, J. -C. ; Mitanchey, R. & Péroche, B. : Bidirectional Instant Radiosity, Proceedings of the 17 th Eurographics Workshop on Rendering, to appear, 2006 [Segovia’ 07]: Segovia, B. ; Iehl, J. -C. & Péroche, B. : Metropolis Instant Radiosity, Proceedings of Eurographics 2007, to appear, 2007 [Myszkowski’ 01]: Myszkowski, K. ; Tawara, T. ; Akamine, H. & Seidel, H. -P. : Perception-guided global illumination solution for animation rendering, SIGGRAPH '01: Proceedings of the 28 th annual conference on Computer graphics and interactive techniques, ACM Press, 2001, 221 -230 [Weber’ 04]: Weber, M. ; Milch, M. ; Myszkowski, K. ; Dmitriev, K. ; Rokita, P. & Seidel, H. -P. : Spatio-Temporal Photon Density Estimation Using Bilateral Filtering, CGI '04: Proceedings of the Computer Graphics International (CGI'04), IEEE Computer Society, 2004, 120 -127 [Cammarano; 02]: Cammarano, M. & Jensen, H. W. : Time Dependent Photon Mapping, In Proceedings of the 13 th Eurographics Workshop on Rendering, 2002, 135 -144 Yu-Chi Lai

Reference • • • [Dmitriev’ 02]: Dmitriev, K. ; Brabec, S. ; Myszkowski, K. & Seidel, H. -P. : Interactive Global Illumination Using Selective Photon Tracing, Rendering Techniques '02 (Proceedings of the 13 th Eurographics Workshop on Rendering), 2002, 25 -36 [Walter’ 99]: Walter, B. ; Drettakis, G. & Parker, S. Lischinski, D. & Larson, G. (ed. ): Interactive Rendering using the Render Cache, Rendering techniques '99 (Proceedings of the 10 th Eurographics Workshop on Rendering), Springer. Verlag/Wien, 1999, 10, 235 -246 [Walter’ 02]: Walter, B. ; Drettakis, G. & Greenberg, D. : Debevec, P. & Gibson, S. (ed. ), Enhancing and Optimizing the Render Cache, Thirteenth Eurographics Workshop on Rendering (2002), ACM Press, 2002, June [Ward’ 99]: Ward, G. & Simmons, M. : The holodeck ray cache: an interactive rendering system for global illumination in nondiffuse environments, ACM Trans. Graph. , ACM, 1999, 18, 361 -368 [Larson’ 98]: Larson, G. W. & Simmons, M. : The Holodeck interactive ray cache, SIGGRAPH '99: ACM SIGGRAPH 99 Conference abstracts and applications, ACM, 1999, 246 Yu-Chi Lai

Reference • • • [Tawara’ 02]: Tawara, T. ; Myszkowski, K. & Seidel, H. -P. : Exploiting Temporal Coherence in Final Gathering for Dynamic Scenes, CGI '04: Proceedings of the Computer Graphics International (CGI'04), IEEE Computer Society, 2004, 110119 [Tawara’ 04]: Tawara, T. ; Myszkowski, K. & Seidel, H. -P. Greiner, G. ; Niemann, H. ; Ertl, T. ; Girod, B. & Seidel, H. -P. (ed. ): Localizing the Final Gathering for Dynamic Scenes using the Photon Map, Proceedings of Vision, Modeling, and Visualization VMV 2002, Akademische Verlagsgesellschaft Aka, 2002, 69 -76 [Smky’ 05]: Smky, M. ; Kinuwaki, S. -i. ; Durikovic, R. & Myszkowski, K. : Temporally Coherent Irradiance Caching for High Quality Animation Rendering, The European Association for Computer Graphics 26 th Annual Conference EUROGRAPHICS 2005, Blackwell, 2005, 24, xx-xx [Adelson’ 95]: Adelson, S. J. & Hodges, L. F. : Generating Exact Ray-Traced Animation Frames by Reprojection, IEEE Comput. Graph. Appl. , IEEE Computer Society Press, 1995, 15, 43 -52 [Badt’ 88]: Jr. , S. B. : Two algorithms for taking advantage of temporal coherence in ray tracing The Visual Computer, 1988, 123 -132 Yu-Chi Lai

Reference • • • [Stamminger’ 00]: Stamminger, M. ; Haber, J. ; Schirmacher, H. & peter Seidel, H. : Walkthroughs with corrective texturing, tal Signal Processing. California Technical Publishing, 2000, 377 -388 [Simmons’ 00]: Simmons, M. & Sequin, C. H. : Tapestry: A dynamic mesh-based display representation for interactive rendering, Proceedings of the 11 th Eurographics Workshop on Rendering, 2000, 329 -340 [Tole’ 02]: Tole, P. ; Pellacini, F. ; Walter, B. & Greenberg, D. P. : Interactive global illumination in dynamic scenes, SIGGRAPH '02: Proceedings of the 29 th Annual Conference on Computer Graphics and Interactive Techniques, ACM Press, 2002, 537 -546 [Mendez’ 06]: Mendez-Feliu, A. ; Sbert, M. & kalo, L. S. : Reusing Frames in Camera Animation, Journal of WSCG, 2006, 14 [Sbert’ 04 a]: Sbert, M. ; Szecsi, L. & Szirmay-Kalos, L. : Real-time Light Animation, Computer Graphics Forum (Proceedings Eurographics 2004), 2004, 23(3), 291 -300 Yu-Chi Lai

Reference • • • [Sbert’ 04 b]: Sbert, M. & Castro, F. : Reuse of Paths in Final Gathering Step with Moving Light Sources, International Conference on Computational Science 2004, 189 -196 [Sbert’ 04 c]: Sbert, M. ; Castro, F. & Halton, J. : Reuse of Paths in Light Source Animation, CGI '04: Proceedings of the Computer Graphics International, IEEE Computer Society, 2004, 532 -535 [Ghosh’ 06]: Ghosh, A. ; Doucet, A. & Heidrich, W. : Sequential Sampling for Dynamic Environment Map Illumination, Proc. Eurographics Symposium on Rendering, 2006, 115 -126 [Briere’ 96]: Briere, N. & Pierre, P. : Hierarchical view-dependent structures for interactive scene manipulation, SIGGRAPH '96, 1996, 83 -90 [Murakami’ 89]: Murakami, K. & Hirota, K. : Incremental ray tracing, Eurographics Workshop on Photosimulation, Realism and Physics in Computer Graphics, 1989 Yu-Chi Lai

Reference • [Murakami’ 89]: Murakami, K. & Hirota, K. : Incremental ray tracing, Eurographics Workshop on Photosimulation, Realism and Physics in Computer Graphics, 1989 • [Jevans’ 92]: Jevans, D. A. : Object space temporal coherence for ray tracing, Proceedings of the conference on Graphics interface '92, Morgan Kaufmann Publishers Inc. , 1992, 176 -183 • [Sequin’ 89]: Séquin, C. H. & Smyrl, E. K. : Parameterized Ray-tracing, SIGGRAPH '89: Proceedings of the 16 th annual conference on Computer graphics and interactive techniques, ACM, 1989, 307 -314 • [Bala’ 99]: Bala, K. ; Dorsey, J. & Teller, S. : Interactive ray-traced scene editing using ray segment trees, In Tenth Eurographics Workshop on Rendering, 1999, 39 -52 • [Fernadez]: Fernandez, S. ; Bala, K. ; Piccolotto, M. A. & Greenberg, D. P. : Interactive Direct Lighting in Dynamic Scenes, 2000 Yu-Chi Lai

Reference • • • [Muuss’ 95 a]: Muuss, M. J. : Toward real-time ray-tracing of combnatorial solid geometric models, Proceeding of BRL-CAD Symposium, 1995 [Muuss’ 95 b]: Muuss, M. J. & Lorenzo, M. : High-resolution interactive multispectral missile sensor simulation for atr and dis. , Proceeding of BRL-CAD Symposium, 1995 [Parker’ 99 a]: Parker, S. ; Martin, W. ; Livnat, Y. ; Sloan, P. ; Shirley, P. ; Smits, B. & Hansen, C. : Interactive Ray Tracing, Symposium on Interactive 3 D Computer Graphics, 1999 [Parker’ 99 b]: Parker, S. ; Martin, W. ; Livnat, Y. ; Sloan, P. ; Shirley, P. ; Smits, B. & Hansen, C. : Interactive Ray Tracing for volume visualization, IEEE transaction on COmputer Graphics and Visualization, 1999, 5 [Wald’ 01 a]: Wald, I. ; Slusallek, P. & Benthin, C. S. J. Gortler & K. Myszkowski (ed. ): Interactive Distributed Ray Tracing of Highly Complex Models, Rendering Techniques 2001 (Proceedings of the 12 th EUROGRAPHICS Workshop on Rendering, Springer, 2001, 277 -288 [Wald’ 01 b]: Wald, I. ; Benthin, C. ; Wagner, M. & Slusallek, P. Bhalmers, A. & Rhyne, T. M. (ed. ): Interactive Rendering with Coherent Ray Tracing, Computer Graphics Forum (Proceedings of EUROGRAPHICS 2001, Blackwell Publishers, Oxford, 2001, 20, 153 -164 Yu-Chi Lai

Reference • • [Reshetov’ 05]: Reshetov, A. ; Soupikov, A. & Hurley, J. : Multi-level ray tracing algorithm, SIGGRAPH '05: ACM SIGGRAPH 2005 Papers, ACM, 2005, 1176 -1185 [Woop’ 05]: Sven Woop, J. S. & Slusallek, P. : RPU: A Programmable Ray Processing Unit for Realtime Ray Tracing, Proceedings of ACM SIGGRAPH 2005, 434 -444 [Wald’ 06 a]: Wald, I. ; Thiago, I. ; Kensler, A. ; Knoll, A. & Parker, S. G. : Ray Tracing Animated Scenes Using Coherence Grid Traversal, SIGGRAPH '06: Proceedings of the 32 th annual conference on Computer graphics and interactive techniques, 2005 [Gribble’ 06]: Gribble, C. ; Ize, T. ; Kensler, A. ; Wald, I. & Parke, S. G. : A Coherent Grid Traversal Approach to Visualizing Particle-Based Simulation Data, Visualization and Computer Graphics, IEEE Transactions on, 2006, 758 -768 [Overbeck’ 08]: Overbeck, R. & andi Mark William, R. R. : Large ray packets for real-time Whitted ray tracing, 2008, 41 -48 [Wald’ 07 a]: Wald, I. ; Boulos, S. & Shirley, P. : Ray Tracing Deformable Scenes using Dynamic Bounding Volume Hierarchies, ACM Transactions on Graphics, 2007, 26, 2007 [Lauterbach’ 06], Lauterbach, C. ; eui Yoon, S. & Manocha, D. : RT-DEFORM: Interactive Ray Tracing of Dynamic Scenes using BVHs, In Proceedings of the 2006 IEEE Symposium on Interactive Ray Tracing, 2006, 39 -45 Yu-Chi Lai

Reference • • • [Wald’ 03 a]: Wald, I. ; Benthin, C. & Slusallek, P. : Distributed Interactive Ray Tracing of Dynamic Scenes, Proc. of the IEEE Symposium on Parallel and Large-Data Visualization and Graphics, 2003 [Gun’ 06]: Gunther, J. ; Friedrich, H. ; Wald, H. ; Seidel, H. -P. & Slusallek, P. : Ray Tracing Animated Scenes Using Motion Decomposition, Proc. Eurographics, 2006 [Hunt’ 06]: Hunt, W. ; Mark, W. R. & Stoll, G. : Fast kd-tree Construction with an Adaptive Error-Bounded Heuristic, 2006 IEEE Symposium on Interactive Ray Tracing, IEEE, 2006 [Zhou’ 08]: Zhou, K. ; Hou, Q. ; Wang, R. & Guo, B. : Real-time KD-tree construction on graphics hardware, SIGGRAPH Asia '08: ACM SIGGRAPH Asia 2008 papers, ACM, 2008, 1 -11 [Popov’ 06]: Popov, S. ; Günther, J. ; Seidel, H. -P. & Slusallek, P. : Experiences with Streaming Construction of SAH KD-Trees, Proceedings of the 2006 IEEE Symposium on Interactive Ray Tracing, 2006, 89 -94 Yu-Chi Lai

Reference • • • [Ize’ 06 a]: Ize, T. ; Wald, I. ; Robertson, C. & Parker, S. : An Evaluation of Parallel Grid Construction for Ray Tracing Dynamic Scenes, Interactive Ray Tracing 2006, IEEE Symposium on, 2006, 47 -55 [Ize’ 06 b]: Ize, T. ; Shirley, P. & Parker, S. : Grid Creation Strategies for Efficient Ray Tracing, Interactive Ray Tracing 2006, IEEE Symposium on, 2006, 27 -32 [Lagae’ 08 a]: Lagae, A. & Dutré, P. : Compact, fast and robust grids for ray tracing, SIGGRAPH '08: ACM SIGGRAPH 2008 talks, ACM, 2008, 1 -1 Lext’ 01: Lext, J. & muller, T. A. EUROGRAPHICS 2001 / Jonathan C. Roberts Short Presentations, Towards Rapid Reconstruction for Animated Ray Tracing, Eurographics, 2001, 311 -318 [Stoll’ 06]: Stoll, G. ; Mark, W. R. ; Djeu, P. ; Wang, R. & Elhassan, I. , Razor: An architecture for dynamic multiresolution ray tracing. : University of Texas at Austin, 2005 [Yoon’ 07]: Yoon, S. -E. ; Curtis, S. & Manocha, D. : Ray tracing dynamic scenes using selective restructuring, SIGGRAPH '07: ACM SIGGRAPH 2007 sketches, ACM, 2007, 55 Yu-Chi Lai

Reference • • • [Hasan’ 08]: Hašan, M. ; Edgar Velazquez-Armendariz and Pellacini, F. & Bala, K. : Tensor Clustering for Rendering Many-Light Animations, Proc. of the 19 th Eurographics Symposium on Rendering, Eurographics Association, 2008 [Hasan’ 07]: Hašan, M. F. & Bala, K. : Matrix row-column sampling for the many-light problem, SIGGRAPH '07: ACM SIGGRAPH 2007 papers, ACM, 2007, 26 [Walter’ 05]: Walter, B. ; Fernandez, S. ; Arbree, A. ; Bala: Lightcuts: a scalable approach to illumination, ACM Trans. Graph. , ACM, 2005, 24, 1098 -1107 [Clarberg’ 05]: Clarberg, P. ; Jarosz, W. ; Akenine-Möller, T. & Jensen, H. W. : Wavelet Importance Sampling: Efficiently Evaluating Products of Complex Functions, Proceedings of ACM SIGGRAPH 2005, ACM Press, 2005 [Clarberg’ 08]: Clarberg, P. & Akenine-Möller, T. : Practical Product Importance Sampling for Direct Illumination, Computer Graphics Forum (Proceedings of Eurographics 2008), 2008, 27, 681 -690 Yu-Chi Lai

Reference • [Dayal’ 05]: Dayal, A. ; Woolley, C. ; Watson, B. & Luebke, D. : Adaptive frameless rendering, SIGGRAPH '05: ACM SIGGRAPH 2005 Courses, ACM, 2005, 24 • [Herzog’ 08]: Herzog, R. ; Kinuwaki, S. ; Myszkowski, K. & Seidel, H. P. , Render 2 MPEG: A Perception-based Framework Towards Integrating Rendering and Video Compression, Computer Graphics Forum, 2008, 27, 183 -192 • [Purcell’ 03]: Timothy J. Purcell, Craig Donner, Mike Cammarano, Henrik Wann Jensen, and Pat Hanrahan: Photon Mapping on Programmable Graphics Hardware, Proceedings of the ACM SIGGRAPH/EUROGRAPHICS Conference on Graphics Hardware, pp. 41 -50, 2003. • [Purcell’ 02]: Timothy J. Purcell, Ian Buck, William R. Mark, and Pat Hanrahan. : Ray tracing on programmable graphics hardware. , ACM Transactions on Graphics, 21(3): 703. 712, July 2002. ISSN 0730 -0301 Yu-Chi Lai

Spatial Perturbation • Perturbed the pixel position and reconstruct the lens edge • Reconstruct the specular sub -paths • Connect to the rest of paths. Yu-Chi Lai

Temporal Perturbation • Update the position of diffuse vertices • Reconstruct the specular sub -path • Check the validity of the path Yu-Chi Lai

Photon Mapping • Two pass: light and eye pass – Light pass shoots out photons to create photon maps – Eye pass collects the radiance by splitting • Ldirect: CS*L ( direct lighting algorithms) • Lcaustics: CS*D+S*L (caustics map) • Lindirect: all others involve more than one diffuse surfaces (global map or final gathering) • Can generate good results but introduce bias and final gathering is still very time consuming Yu-Chi Lai

Physically-based Animation Rendering • Photon shooting algorithms – Photon shooting – Instant radiosity – Light cut • Path reusing algorithms • GPU-based algorithms – Two phase pre-computation rendering: separate the rendering into two phases: preprocess computation (CPU) and real-time rendering (GPU) such as environment map prefiltering, precomputed radiance transfer, relighting, … – Hybrid methods: separate the algorithm into two parts: one is computed by CPU and the other is by GPU to take advantage of both such as instant radiosity, photon splatting, and its extensions. Yu-Chi Lai

Grand Challenge • Efficiently render entire animation – – Do parallel computation in multi-processors or GPU. Do coherent ray tracing. Efficiently update the acceleration structure. Reuse samples from previous computation • Reduce the temporal artifacts – Exploration the temporal coherence among paths – Reuse the computation results. • Perceptual evaluation of the rendering results. – Evaluate the strength of each algorithm. – Allow us to distribute computation to perceptual important features. • Challenge in two phase pre-computation methods – – Efficiently generate accurate data Efficiently store and retrieve the data Handle the movement and change of objects and scenes Can we compute the data on fly? Yu-Chi Lai

Hybrid Algorithms • Based on instant radiosity[Laine’ 07]: – Create a set of virtual lights and then create a set of shadow maps – Use the depth in the shadow map to determine the visibility of rendering points • Based on photon splatting [Gautron’ 05] – Cpu generate a set of photon maps and then translate the gpu – GPU trace the collection and use the photon maps to deposit energy on each collection position • Simplify the global illumination Animation: Laine et al. Yu-Chi Lai

Physically-based Animation Rendering • Right now the research is mainly from German’s MPI, USA’s UCSD, Standford, and Cornell. • Image-based + global illumination => hard for dynamic scene [Nimeroff’ 96, Myszkowski’ 99] • Radiosity methods => simple objects and material – Progressive algorithms [Chen’ 90, George’ 90, Muller’ 95] – Hierarchical algorithms[Forsyth’ 94, Shaw’ 97, Drettakis’ 97, Martin’ 99, Damez’ 99 ] • Decouple render and display => not really solve global illumination problem [Walter’ 99, walter’ 02, Larson’ 98, Ward’ 99, Stamminger’ 00, Simmons’ 00, Tole’ 02] Yu-Chi Lai