Case Study PETSc ex 19 r Nonlinear solver

Case Study: PETSc ex 19 r Non-linear solver (snes) ¦ ¦ ¦ r 2 -D driven cavity code uses velocity-velocity formulation finite difference discretization on a structured grid Instrumentation: ¦ MPI Ø MPI ¦ wrapper interposition library PDT (tau_instrumentor) Ø PETSc library sources Ø Application ¦ Selective instrumentation (tau_reduce)

Configuring TAU r Profiling with wallclock time (on a quad PIII Linux machine) ¦ r Tracing ¦ r % configure –mpiinc=/usr/local/packages/mpich/include –mpilib=/usr/local/packages/mpich/lib –pdt=/usr/pkg/pdtoolkit/ -useropt=-O 2 –LINUXTIMERS % configure –mpiinc=/usr/local/packages/mpich/include –mpilib=/usr/local/packages/mpich/lib –pdt=/usr/pkg/pdtoolkit –useropt=-O 2 –LINUXTIMERS Profiling with PAPI ¦ % configure –mpiinc=/usr/local/packages/mpich/include –mpilib=/usr/local/packages/mpich/lib –pdt=/usr/pkg/pdtoolkit/ -useropt=-O 2 –papi=/usr/local/packages/papi % setenv PAPI_EVENT PAPI_FP_INS Ø % setenv PAPI_EVENT PAPI_L 1_DCM Ø

Compiling: TAU Makefiles r r Include TAU Stub Makefile (<arch>/lib) in the user’s Makefile. Variables: ¦ ¦ ¦ r TAU_CXX TAU_CC, TAU_F 90 TAU_DEFS TAU_LDFLAGS TAU_INCLUDE TAU_LIBS TAU_SHLIBS TAU_MPI_FLIBS TAU_FORTRANLIBS TAU_DISABLE Specify the C++ compiler used by TAU Specify the C, F 90 compilers Defines used by TAU. Add to CFLAGS Linker options. Add to LDFLAGS Header files include path. Add to CFLAGS Statically linked TAU library. Add to LIBS Dynamically linked TAU library TAU’s MPI wrapper library for C/C++ TAU’s MPI wrapper library for F 90 Must be linked in with C++ linker for F 90. TAU’s dummy F 90 stub library Note: Not including TAU_DEFS in CFLAGS disables instrumentation in C/C++ programs (TAU_DISABLE for f 90).

Profiling ex 19 (wallclock time)

Profiling ex 19 (wallclock time)

Tracing ex 19

Callgraph Display for ex 19

Using TAU with PAPI (PAPI_FP_INS)

Using TAU with PAPI (PAPI_L 1_DCM)

Using TAU with PAPI (PAPI_FP_INS)

Using TAU with PAPI (PAPI_FP_INS)

Support Acknowledgement r TAU and PDT support: ¦ Department of Energy (DOE) Ø DOE 2000 ACTS contract Ø DOE MICS contract Ø DOE ASCI Level 3 (LANL, LLNL) Ø U. of Utah DOE ASCI Level 1 subcontract ¦ ¦ DARPA NSF National Young Investigator (NYI) award