Convolution FFT Bloom David Hill Principal Mathematician Epic

Convolution (FFT) Bloom David Hill Principal Mathematician Epic Games 5/25/2017

Bloom: Why Invest? A convincing bloom adds subtle but impactful physically motivated effect. Currently feasible on today’s high end hardware. . ● ● Adapt film / offline approach for use in real time Additional realism with highly detailed effects Provide ability to customize the camera / eye response Target Cinematics and High End hardware

Bloom As Weak Scattering: Some Causes ● ● ● Haze Car Windshield (mine is always a little dirty) Eyelash Diffraction Inner-Camera Effects Fraunhofer Diffraction ○ Physically-Based Glare Effects for Digital Images (1995, Spencer et al) ○ Glare Generation Based on Wave Optics (2004, Kakimoto et al, Computer Graphics Forum)

Bloom As Weak Scattering A single pin point remains mostly focused In Camera scattering of a point Model Assumptions: ● Every point of light is scattered the same ● For most sources the scattered amount is imperceptible ● The scattering from bright sources creates “bloom”

Bloom: Standard Game-Style Weak Scattering Sum of Gaussian Filter Standard Bloom in UE 4: Recommended for game use ● ● ● Image down-sampled multiple times and Gaussian Blur applied Resulting images are summed and added to original Every pixel scatters light in a symmetric way to its neighbors Very Fast! But. . .

Bloom As Weak Scattering The camera response to a single bright source: Close Up of center Camera scattering of point Appears to be a very small star Center pixel about 10, 000 times brighter than any other point in this. exr Full image, adjusted contrast Detail extents far across image!

Bloom Scatter As Gather: Convolution Original Value Filtered Value Filter View this as a weighted sum, here in one dimension : For Example, simple smoothing Each Pixel Scatters to all the others according to the filter.

Bloom Scatter As Gather: Convolution: ● Every point is weighted sum of all other points ● N points gather from all N points. O(N^2) ● We need a faster way! -> FFT

Bloom Scatter As Gather: Convolution With Fast Fourier Transform: 1. 2. 3. 4. Image_Frequencies Filter_Frequencies Convolved_Image = = FFT(Image) FFT(Filter) Image_Frequencies x Filter_Freqencies Inverse. FFT(Convolved_Frequencies)

Bloom: Why FFT Convolution? Better Scaling For our purposes, the Fast Fourier Transform (FFT) is an acceleration technique for convolution. ● ● FFT from signal to frequencies - O(N Log N) Convolution in frequency space - O(N) FFT from frequencies to signal - O(N Log N) Overall Scaling O(N Log N)

Bloom: What is Fourier Transform? Any finite length signal can be expresses as a sum of sines and cosines Signal Amplitude of each frequency Sine & Cos Inverse computes amplitudes of each frequency Low-Pass filter: only used first few V_k

Bloom: Fast Fourier Transform? Sines and Cosines have symmetries that can be exploited speeding the transform Split the sum into even and odd terms, and symmetry saves work! Now recurse, each small transform (Even, Odd) and be treated as new transforms And subdivided. Due to this splitting FFT does best with power_2 signals.

Bloom : Parallel FFT for GPU: Highly parallel, but with thread-communication limitations. Similar restriction are found in supercomputing architectures. ● Stockham Formulation: Parallel formulation ○ Used for Vector Computers (1987 D Baily, Journal of Supercomputer Applications) ○ Applied to GPUs (2008 Govindaraju, Proceedings of ACM/IEEE on Supercomputing) ■ Our implementation largely follow this ● Each Scanline ○ ○ Transformed independently Assigned to a thread group with group shared memory

Bloom: Compute Shaders FFT implemented as compute shaders ● Pass From GPUVisualizer ○ Forward Horizontal Transform ■ Uses 2 -for-1 trick to transform (r, g) and (b, a) ● Pass ○ Forward Vertical Transform ○ Convolution, i. e. Multiply, with cached pre-transformed Kernel ○ Inverse Vertical Transform ● Pass ○ Inverse Horizontal Transform ■ Inverts 2 -for-1 trick