Halide A compiler for HighPerformance Image Processing Jonathan

Halide: A compiler for High-Performance Image Processing Jonathan Ragan-Kelley, Andrew Adams, Sylvain Paris, Marc Levoy, Saman Amarasinghe, Frédo Durand MIT CSAIL, Adobe, Stanford

High performance image processing • • Hard to program - Requires parallelism, pipeline fusion, and cache behavior optimization Hard to port code to new platforms - Mobile devices, GPUs, clusters/cloud - e. g. Adobe restarted from scratch for their i. Pad apps

The problem Clean sequential code is slow High performance requires vectorization, parallelism, tiling, and they require deep changes throughout the code

The problem, continued • Code must be rewritten more or less from scratch to be fast on different platforms - computer, mobile devices, GPU

Image processing • • • Operate on large images - data parallelism Often a pipeline - e. g. demosaick, denoise, sharpen, color, compress dynamic range, compress to JPEG Computation must be organized in space (image) and time (pipeline) - must exploit parallelism - must optimize memory (cache) behavior - tradeoffs between recomputation and locality - often need to merge pipeline stages

Solution: Halide • • • Functional language Instructions for schedule/optimization Compiler with backends for x 86, ARM, GPU High performance Readable Portable Modular

Halide Results • • Compared to hand-optimized code - Here: parallelized with Open. MP, Intel Performance Primitives Here: Our new algorithm (featured in Photoshop CS 6)

Halide Results • • Concise Enables easy optimization exploration As fast or faster than hand-optimized code Portable

Eulerian Video Magnification for Revealing Subtle Changes in the World Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Fredo Durand, William Freeman MIT CSAIL and QRC

Eulerian video magnification • • Mostly pure temporal filtering of videos - Although we add a spatial pyramid to make it better User selects band of temporal frequencies We amplify the pixel value variations - Can also amplify subtle changes such as redness when blood flows Although purely temporal, it amplifies spatial motion - we have maths to prove it.

Video