Brookhaven Science Associates U S Department of Energy

Brookhaven Science Associates U. S. Department of Energy

Objectives Develop visualization : n with real time interaction (high performance); n for large sets of data (1 GB and larger); n high resolution (but cost efficient). Motivated by Today’s technological advances n increasingly complicated mathematical models; n masses of data produced by devices such as MRI and lasermicroscope scanners, etc. ; n the increasing need for interactive extraction of insight from extremely large data sets. Brookhaven Science Associates U. S. Department of Energy 2 2

Some Applications MRI Mouse Brain volume visualization (512 x 256) Brookhaven Science Associates U. S. Department of Energy Fluid Flow isosurface visualization 3 Material Micro-Geometry Studies crushed rock volume visualization (1007 x 256) 3

Approach l l l Remote parallel rendering to a large tiled display; Clusters connected with high speed network; Extend and combine already existing APIs: v Chromium; v Open Inventor; v VTK, etc. Alternatives High resolution flat panels driven by a single workstation; Faster machines; Build entire visualization system from scratch. Brookhaven Science Associates U. S. Department of Energy 4 4

An Example l ivview window interface l display with 4 tiles Mouse brain volume visualization (512 x 256) Brookhaven Science Associates U. S. Department of Energy 5 5

The Visualization Pipeline Open. GL Applications #1 Rendering Servers #2 #1 • #2 • Open. GL calls • • Tiled Display #M #N Mouse (and keyboard input) Brookhaven Science Associates U. S. Department of Energy High Speed Network 6 6

Chromium Overview l l Based on Wire. GL, developed at Stanford University; Open-source API providing : v Open. GL interface for rendering on commodity-based clusters; v scalable display technology. How does it work? Accelerate the data flow Open. GL calls (1) simplified network protocol (2) “bucketing” (3) state tracking + display lists Brookhaven Science Associates U. S. Department of Energy 7 • Rendering • Servers • 7

Chromium Parallelization Model (CPM) gl. XMake. Current(get. Display(), get. Normal. Window(), get. Normal. Context()); if (clear. Flag) gl. Clear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); gl. Barrier. Exec. CR(MASTER_BARRIER); do sequential Open. GL rendering of the local scene gl. Barrier. Exec. CR(MASTER_BARRIER); if (swap. Flag) gl. XSwap. Buffers(get. Display(), get. Normal. Window()); else gl. XSwap. Buffers(get. Display(), CR_SUPPRESS_SWAP_BIT); Display is render: - first : clear. Flag = swap. Flag = 0, and both 1 for proc. 0 - last : clear. Flag = 1, swap. Flag = 0, except swap. Flag = 1 for proc. 0 Brookhaven Science Associates U. S. Department of Energy 8 8

Open Inventor extension l Open Inventor : an open source API of C++ objects and methods for interactive 3 D graphics applications. Extend to support interactive Cluster Visualization (1) Apply the CPM to the Open Inventor rendering method: So. Xt. Render. Area: : redraw() (2) Implement the “Para. Mouse” parallel user interface (the idea sketched on slide #6) or next slide extend the Open Inventor GUI Brookhaven Science Associates U. S. Department of Energy 9 9

Open Inventor GUI extension l We want the GUI that ivview provides (modify main()): declare processor with rank 0 as Master; Extend callback functions if (Master) triggered by mouse, keyboard, etc. run GUI as implemented in ivview; MPI_Isend the invoked user input else in the Master node to the Slave nodes listen for instruction from the Master; So. DB: : init(); So. Node. Kit: : init(); So. Interaction: : init(); // Initialize Inventor So. Separator *root = new So. Separator; root->ref(); read. Scene(root, files); // Build the Inventor’s objects and SGs So. Examiner. Viewer *viewer = new So. Examiner. Viewer(crrank); viewer->Set. Scene. Graph(root); crctx = cr. Create. Context. CR(0 x 0, visual); // Chromium Initialization cr. Make. Current. CR(crwindow, crctx); gl. Barrier. Create. CR(MASTER_BARRIER, crsize); gl. Enable(GL_DEPTH_TEST); viewer->main. Loop(); Brookhaven Science Associates U. S. Department of Energy 10 10

The Result GUI Tiled display ivview window interface Performance (discussion): – Beowulf cluster, 4 nodes, each with 2 Pentium III processors, 1 GHz each Quadro 2 Pro graphics cards, communications through 100 Mb/s fast Ethernet or 1 Gb/s fiber optic network (machine performance [8]); – consider static scene: – spheres with spikes : 11, 540 triangles Brookhaven Science Associates U. S. Department of Energy 11 11

Performance, experiences and conclusions l l for the parallel model considered performance is problem specific; GUI communications time is negligible compared to the visualization time, leading to scalability results in [2, 3, 6]; can send approximately 5, 000 “small” messages/s (see [8]); Chromium automatically minimizes data flow, except for “geometry” flow : consider animation with small static scenes Rendering 2 spheres : Sequential 117 f/s 2, 709, 130 tr. /s 2 CPU, 1 tile 3 f/s 2 CPU, 2 tiles 6 f/s Minimize geometry flow using display lists 80 f/s Brookhaven Science Associates U. S. Department of Energy 12 90 f/s 12

Other experiences and conclusions l l Fast sequential visualization often relies on v graphics acceleration hardware; v data sampling methods; v static scenes; v data size within the hardware limitations; The proposed visualization system (as a remote visualization) v does not require expensive graphics acceleration; v high resolution and interactive speed with ray casting (no need of data sampling); v facilitates well the visualization of time-varying scenes; v can handle extremely large data sets. VTK : a good candidate for parallel GUI extension Brookhaven Science Associates U. S. Department of Energy 13 13

VTK and its extension l VTK : an open source API for visualization and image processing (see [11]); Extend to support interactive Cluster Visualization (as on slide 9) (1) Apply the CPM to VTK’s rendering methods : done by David Thompson, SNL; (2) Implement the “Para. Mouse” parallel user interface (as shown on slide 6) or extend the VTK’s GUI : Extend vtk. XRender. Window. Interactor by: (1) implement “pure” X Windows GUI (no GL calls); (2) add MPI send/receive calls for mouse & keyboard events. Brookhaven Science Associates U. S. Department of Energy 14 14

Conclusions We developed Interactive Parallel Visualizationby v remote parallel rendering to a large tiled display; v commodity-based visualization clusters connected with high speed network; v extending APIs. l l The proposed idea works v Open Inventor; v VTK. Achieved objectives v real time interaction (high performance); v for extremely large sets of data; v high resolution v inexpensive : easy to develop. Brookhaven Science Associates U. S. Department of Energy 15 15

References [ 1] J. Ahrens, C. Law, K. Martin, M. Papka, W. Schroeder, A Parallel Approach for Efficiently Visualizing Extremely Large, Time-Varying Data Sets, LANL, TR #LAUR-00 -1620. [ 2] I. Buck, M. Eldridge, P. Hanharah, G. Humphreys, Distributed Rendering for Scalable Displays, Proceedings of Supercomputing, 2000. [ 3] I. Buck, P. Hanharah, and G. Humphreys, Tracking Graphics State for Networked Rendering, Proceedings of SIGRAPH/Eurographics Workshop on Graphics Hardware, August 2000. [ 4] Eric S. Brugger (LLNL), Visit: A Component-Based, Parallel Visualization Package, DOE, Computer Graphics Forum 2001, Internet address (accessed on 08/2002): http: //www. emsl. pnl. gov: 2080/docs/doecgf 2001/abstractlist. html [ 5] W. Gropp, Tutorial on MPI: The Message-Passing Interface, Mathematics and Computer Sci. Division, ANL, Internet address (accessed on 06/2002): http: //www-unix. mcs. anl. gov/mpi/tutorial/index. html. [ 6] P. Hanharah, H. Igely, G. Stroll, The Design of a Parallel Graphics Interface, Proceedings of SIGRAPH, 1998. [ 7] G. Humphreys, Chromium Documentation, Version BETA, Internet address (accessed on 06/2002): http: //graphics. stanford. edu/~humper/chromium_documentation/ [ 8] V. Mirinavicius, Investigation of MPI performance in Parallel Linear Algebra Software on Linux Beowulf Supercomputers, BNL, Technical Report (to appear). [ 9] Kitware Inc. , Para. View, Internet address (accessed on 08/2002): http: //public. kitware. com/ [10] NASA, Jet Propulsion Laboratory, Par. Vox, A Parallel Splatting Volume Rendering System, Internet address (accessed on 08/2002): http: //alphabits. jpl. nasa. gov/Par. Vox/ [11] W. Schroeder, K. Martin, L. Avila, C. Law, The VTK User’s Guide, Kitware, Inc. [12] J. Wernecke, The Inventor Mentor : Programming Object-Oriented 3 D Graphics with Open Inventor, Addison-Wesley Publishing Company, NY, November 1995. [13] Open Inventor C++ Reference Manual: The Official Reference Document for Open Inventor, Release 2, Addison-Wesley Publishing Company, January 1994. Brookhaven Science Associates U. S. Department of Energy 16 16