Hardware Assisted Visibility Sorting for Tetrahedral Volume Rendering

Hardware Assisted Visibility Sorting for Tetrahedral Volume Rendering Steven Callahan João Comba Milan Ikits Cláudio Silva

Overview • • • Introduction Previous Work Hardware Assisted Visibility Sorting Results Future Work Conclusion Scientific Computing and Imaging Institute, University of Utah

Research Goal Real time volume rendering • Scalable (machine performance) • Data of arbitrary size • Simple and robust implementations Scientific Computing and Imaging Institute, University of Utah

Volume Rendering Regular Irregular Scientific Computing and Imaging Institute, University of Utah

Why Irregular Grids ? • Unstructured grids are the preferred data type in scientific computations • Level Of Detail (LOD) techniques intrinsically need unstructured grids El-Sana et al, Ben-Gurion Scientific Computing and Imaging Institute, University of Utah

Optical Models Absorption plus emission Light s s Scientific Computing and Imaging Institute, University of Utah

Compositing Front to back I 0 I 1 I 2 0 1 2 I 01 I 2 01 2 Scientific Computing and Imaging Institute, University of Utah

Volume Rendering: (Intersection) Sampling + Sorting Scientific Computing and Imaging Institute, University of Utah

Sampling: Triangle Based Approach Class 1 (+, +, +, ) Class 2 (+, +, , ) Projected Tetrahedra [Shirley Tuchman 1990] Scientific Computing and Imaging Institute, University of Utah

Sorting Application Object Space Sorting i. e. , let’s sort the geometry! Rasterization Image Space Display Scientific Computing and Imaging Institute, University of Utah

Cell Projection 7 B 5 6 3 A 4 2 1 p A <p B Scientific Computing and Imaging Institute, University of Utah

Object Space Sorting: Williams’ MPVO Idea: Define ordering relations by looking at shared faces. D A C F B E B<A A<C B<E C<D E<F D<F Viewing direction Scientific Computing and Imaging Institute, University of Utah

MPVO Limitations Missing relations! Scientific Computing and Imaging Institute, University of Utah

XMPVO Idea: Using ray shooting queries to complement ordering relations. C D A Viewing direction B A<C B<D A<B Scientific Computing and Imaging Institute, University of Utah

Sorting Application Object Space Rasterization Image Space Sorting i. e. , let’s sort the pixels! Display Scientific Computing and Imaging Institute, University of Utah

Image Space Sorting: A Buffer Idea: Keep a list of intersections for each pixel. [Carpenter 1984] Scientific Computing and Imaging Institute, University of Utah

Cell Projection With An A Buffer Scientific Computing and Imaging Institute, University of Utah

Cell Projection With An A Buffer Not sorted! Scientific Computing and Imaging Institute, University of Utah

Cell Projection With An A Buffer Sorted! Scientific Computing and Imaging Institute, University of Utah

A Buffer Limitations 2 Number of Intersections: O(cn ) n x n pixels c cells Problems • Time: sorting takes too long • Memory: storage too high Scientific Computing and Imaging Institute, University of Utah

Sorting Application Object Space Sorting Rasterization Image Space Sorting Display Scientific Computing and Imaging Institute, University of Utah

Approximate Object Space Sorting 1 Scientific Computing and Imaging Institute, University of Utah

Approximate Object Space Sorting 1 2 Scientific Computing and Imaging Institute, University of Utah

Approximate Object Space Sorting 1 3 2 Scientific Computing and Imaging Institute, University of Utah

Approximate Object Space Sorting 1 3 2 4 5 Scientific Computing and Imaging Institute, University of Utah

Approximate Object Space Sorting 1 3 2 4 5 6 7 Scientific Computing and Imaging Institute, University of Utah

Approximate Object Space Sorting 1 3 2 4 5 7 6 A Solution: Use an insertion sort A buffer! Scientific Computing and Imaging Institute, University of Utah

Approximate Object Space Sorting What about the space problem? 1 3 2 4 5 7 6 Use a conservative bound on the intersections Scientific Computing and Imaging Institute, University of Utah

Hardware Assisted Visibility Sorting (HAVS) Sort in image space and object space • Do an approximate object space sorting of the cells on the CPU (i. e. sort by face centroid) • Complete the sort in image space by using a fixed depth A buffer (called a k buffer) implemented on the GPU • Can handle non convex meshes, has a low memory overhead, and requires minimal pre processing of data Scientific Computing and Imaging Institute, University of Utah

HAVS Overview Scientific Computing and Imaging Institute, University of Utah

k buffer • Fixed size A buffer of depth k • Fragment stream sorter • Stores k entries for each pixel. Each entry consists of the fragment’s scalar value and its distance to the viewpoint • An incoming fragment replaces the entry that is closest to the eye (front to back compositing) • Given a sequence of fragments such that each fragment is within k positions from its position is sorted order, it will output the fragments in sorted order Scientific Computing and Imaging Institute, University of Utah

k buffer: Hardware Implementation • Use multiple render target capability of ATI graphics cards (ATI_draw_buffers in Open. GL) • Use P buffer to accumulate color and opacity and three Aux buffers for the k buffer entries r g b a P buffer r comp g comp b comp a comp Aux 0 v 1 d 1 v 2 d 2 Aux 1 v 3 d 3 v 4 d 4 Aux 2 v 5 d 5 v 6 d 6 Scientific Computing and Imaging Institute, University of Utah

Fragment Shader Overview Scientific Computing and Imaging Institute, University of Utah

Details • Fix incorrect screen space texture coordinates caused by perspective correct interpolation Perspective interpolation Projecting vertices to find tex coords Projecting tex coords in shader Scientific Computing and Imaging Institute, University of Utah

Details • Simultaneously reading and writing to a buffer is undefined when fragments are rasterized in parallel Scientific Computing and Imaging Institute, University of Utah

Details • The buffers are initialized and flushed using k screen aligned rectangles with negative scalar values • Handling non convex objects requires the exterior faces to be tagged with a negative distance d and keeping track of when we are inside or outside of the mesh with the sign of the scalar value v Scientific Computing and Imaging Institute, University of Utah

Details • Early ray termination reads accumulated opacity and kills fragment if it is over a given threshold. Early z test is currently not available on ATI 9800 when using multiple rendering targets Scientific Computing and Imaging Institute, University of Utah

Pre Integrated Transfer Function Previous Work • Volume density optical model Williams and Max 1992 • Pre integration on GPU Roettger et al. 2000 5 s to update a 128 x 128 table • Incremental pre integration on CPU Wieler et al. 2003 1. 5 s to update a 128 x 128 table Scientific Computing and Imaging Institute, University of Utah

Pre Integrated Transfer Function Williams and Max Sb Sf l Scientific Computing and Imaging Institute, University of Utah

Pre Integrated Transfer Function Roettger et al. n = 0…lmax Sb Sf T 3 D Scientific Computing and Imaging Institute, University of Utah

Pre Integrated Transfer Function Weiler et al. Sf Sb Sp l l’ l Scientific Computing and Imaging Institute, University of Utah

Pre Integrated Transfer Function Our Approach • Incremental pre integration of the 3 D transfer function completely on the GPU Compute base slice using [Roettger et al. ] Compute the other slices using the base slice and the previously computed slice [Weiler et al. ] • 0. 067 s to update a 128 x 128 table • This allows interactive updates to the colormap and transfer function opacity Scientific Computing and Imaging Institute, University of Utah

Experiments Environment • 3. 0 GHz Pentium 4 • 1024 MB RAM • Windows XP • ATI Radeon 9800 Pro Results • k buffer analysis • Performance results Scientific Computing and Imaging Institute, University of Utah

K buffer Analysis Accuracy analysis • Analysis of k depth required to correctly render datasets • Max values from 14 fixed viewpoints Datas Max A Max k k > 2 et Spx 2 476 22 10, 26 2 Torso 649 15 43, 31 7 Fighte 904 3 1 r k>6 512 1, 683 0 Scientific Computing and Imaging Institute, University of Utah

k buffer Analysis Distribution analysis • Shows actual pixels that require large k depths to render correctly for each viewpoint k <= 2 (green) 2 < k <= 6 (yellow) k > 6 (red) Scientific Computing and Imaging Institute, University of Utah

Results Performance • Average values from 14 fixed viewpoints • Does not include partial sort on CPU • 512 x 512 viewport with a 128 x 128 pre integrated transfer function Dataset Cells Spx 2 0. 8 M K=2 Fps 2. 07 K=2 K=6 Tets/s Fps 1712 K 1. 7 K=6 Tets/s 1407 K Torso 1. 1 M 3. 13 3390 K 1. 86 1977 K Fighter 1. 4 M 2. 41 3387 K 1. 56 2190 K Scientific Computing and Imaging Institute, University of Utah

Image – Blunt Fin Scientific Computing and Imaging Institute, University of Utah

Image Spx Scientific Computing and Imaging Institute, University of Utah

Image – Torso Scientific Computing and Imaging Institute, University of Utah

Image Fighter Scientific Computing and Imaging Institute, University of Utah

Future Work • Optimize partial sort on CPU • Develop techniques to refine datasets to respect a given k (subdivide degenerate tets) • Incorporate isosurface rendering • Parallel techniques • Proper hole handling • Dynamic data • Use early z test Scientific Computing and Imaging Institute, University of Utah

Conclusion • Renders up to 6 million Tets/sec when using a linear transfer function • Handles arbitrary non convex meshes • Requires minimal pre processing of data • Maximum data size is bounded by main memory • Uses simple vertex and fragment shaders Scientific Computing and Imaging Institute, University of Utah