Volume Stylizer Tomographybased Volume Painting Oliver Klehm 1

Volume Stylizer: Tomography-based Volume Painting Oliver Klehm 1, Ivo Ihrke 1, 2, Hans-Peter Seidel 1, Elmar Eisemann 3 Max-Planck-Institut für Informatik 2 Saarland University 3 TU Delft 1 I 3 D 2013 23/03/2012

Oliver Klehm, 2

Motivation Oliver Klehm, 3 § Want to influence the appearance of volumes (voxels) • Editing on a fine-grained level is non-intuitive • No suitable tools available o A Programmable System For Artistic Volumetric Lighting [Nowrouzezahrai 2011] § Approach: • Inverse rendering: editing by painting

Our Goal Oliver Klehm, 4 § What do we modify • Volume parameters (per voxel) o Emission o Albedo o Extinction

Our Goal Oliver Klehm, 5 § What do we modify • Volume parameters (per voxel) § From what do we edit the parameters • Arbitrary input images (target views) • In practice: rendered images that were painted

Our Goal: Volume Painting Oliver Klehm, 6

Our Goal: Volume Painting Oliver Klehm, 7

Our Goal Oliver Klehm, 8 § What do we modify • Volume parameters (per voxel) § From what do we edit the parameters • Arbitrary input images (target views) • In practice: rendered images that were painted § Additional required information • Per image: camera parameters (viewing position, direction) • Lighting, scene § How do we optimize • Per-pixel difference to input image • Linear problem • Inconsistent input -> least-squares solution

Specific Goal Oliver Klehm, 9 § Given: each pixel k of target view as a constraint pixel § Data associated: ray origin, direction, target radiance § Try to reconstruct volume properties, such that computed radiance matches target radiance (per pixel k)

Problem overview Oliver Klehm, 10 § A single pixel may influence many voxels § And vice versa…

Volume Rendering Oliver Klehm, 11

Volume Reconstruction Oliver Klehm, 12 § Represent the target field (e. g. basis functions ) as linear combination of • Common for stored volumes (== voxels: box/triangle basis fct. ) § Reconstruct coefficient vector , such that Includes target Includes volume rendering

Emission Reconstruction Oliver Klehm, 13

Optimization Oliver Klehm, 14 § W is huge, ill-conditioned, …! => iterative conjugate gradient method § We need to compute: • Objective Function: • Derivative: § The trick: = => this is rendering / back-projection!

Implementation Oliver Klehm, 15 § Pre-computation • Cache incoming light in volume per voxel • Sub-sample environment map, deep shadow maps § Optimization • 1. objective function evaluation o Voxel traversal, gathering pre-cached incoming light

Implementation Oliver Klehm, 16 § Pre-computation • Cache incoming light in volume per voxel • Sub-sample environment map, deep shadow maps § Optimization • 1. objective function evaluation • 2. gradient of objective function

Implementation Oliver Klehm, 17 § Pre-computation • Cache incoming light in volume per voxel • Sub-sample environment map, deep shadow maps § Optimization • 1. objective function evaluation • 2. gradient of objective function • Additional vector arithmetic (addition, norm 2, …)

Results - Video Oliver Klehm, 18

Limitations / Future Work Oliver Klehm, 19 § How to handle inconsistent inputs? § How to define intermediate views? § How to generalize? – requires different optimization § Optimize for volume and lighting

Conclusion Oliver Klehm, 20 § 3 D volume properties by 2 D painting § Standard Rendering Model § Fast GPU Optimization for interactive sessions Thank you

Fast Implementation Oliver Klehm, 21 § All Open. GL § Vectors are actually 3 D Textures § Operations as shaders • Matrix multiplication is volume rendering • Arithmetic operations are “simple” shaders § => Reconstruction in order of seconds

Results Oliver Klehm, 22

Results Oliver Klehm, 23

Results - Extinction Oliver Klehm, 24

Related Work Oliver Klehm, 25 § Light Editing • Lighting with paint [Pellacini 2007] o Non-linear light optimization • Bendy. Lights [Kerr 2010] o Editing of spot lights • Optimizing Environment Maps for Material Depiction [Bousseau 2011] o Optimization with adhoc metrics § Volume editing • A Programmable System For Artistic Volumetric Lighting [Nowrouzezahrai 2011] o o Derivation of volume properties for single light beam only Replacing physically-motivated parts of rendering with shaders Homogeneous media only Treat light beam as string of light (e. g. flow of particle) • Automatically Adjusting Parameters for Rendering Clouds using Photographs [Dobashi 2012]

Related Work Oliver Klehm, 26 § Tomography • Ihrke • Wetzstein • Lanmann § Fabrication • Shadow Art [Pauly 2009] • Manufactoring Layered Attenuators for Multiple Prescribed Shadow Images [Baran 2012]

Volume Properties Oliver Klehm, 27 § Properties to reconstruct: • Extinction (overall density of volume) • Albedo (amount of scattering) • Emission (self glow) § Assumption: Single Scattering! • Light scatters ONCE in volume

Emission reconstruction Oliver Klehm, 28 Background / first surface, attenuated by volume Integration along view ray Attenuation by volume Number of particles (extinction) In-scattered light