Par FUM from an Application Perspective M Scot

Par. FUM from an Application Perspective M. Scot Breitenfeld

Motivation • Finite element code – 3 -D Cohesive/volumetric finite element (CVFE) scheme for spontaneous dynamic fracture • Uses cohesive element for fracture events

Pure tensile (mode I) - Dt = 0 Normal Traction Law Shear Traction Law

Motivation – Explicit dynamic (central difference time stepping scheme) • No large system of equations to solve – Nonlinear kinematics (large strains, large rotations) – Nonlinear constitutive models • Hyperelastic – Neo-Hookean (compressible, incompressible) – Mooney-Rivlin – Arruda-Boyce • Nonlinear constitutive laws – Viscoplastic – Original implementation used MPI

Motivation • MPI based code limitations – No adaptive load balancing • Non-trivial using MPI • Load balancing for adaptive mesh changes – insertion/activation of cohesive elements Deflection of a crack by a material interface Strong interface Weak interface (Experimental results by Xu, Huang and Rosakis, 2001)

Motivation – Load imbalance due to material models • 3 D Plastic Fracture • A single edge notched specimen pulled at both ends with a ramping magnitude of 1 m/s over . 01 seconds • Isosurface is the extent of the plastic zone

Motivation • A 1 x 1 x 10 bar loaded with a ramped velocity of magnitude 0. to 40. over a time period of (0. 1*length of bar)/(fastest wave speed) and held constant afterward

Application Typical Procedure for a Parallel FEM Code • Par. FUM • MPI PRE-PROCESSOR • Reads in mesh – created with a meshing tool • Partitions the mesh • Create communication lists for the processors • Transfer boundary conditions to the partitioned mesh • No pre-processor needed • Automatically – Partitions – Creates the communication list – Handles the boundary conditions

Application Typical Procedure for a Parallel FEM Code • Par. FUM • MPI Solution Code • Insert MPI calls to handle communications: Send, Recieves waits, etc…. • Solution Code • Communication calls are simplified. .

Application Typical Procedure for a Parallel FEM Code DO j 1 = 1, Num. Neigh. Procs k = Id. Neigh. Proc(j 1) CALL MPI_IRECV(frmproc(k)%rcvbuf(1), COMMON(j 1)*3, & MPI_DOUBLE_PRECISION, k, 10, MPI_COMM_WORLD, & req_rcv(j 1), ierr) ENDDO • Par. FUM CALL FEM_Update_Field(fid_R_net, R_net) k 2 = 1 DO j 1 = 1, Num. Neigh. Procs k = Id. Neigh. Proc(j 1) k 3 = COMMON(j 1)*3 CALL MPI_ISEND(buf(k 2), k 3, MPI_DOUBLE_PRECISION, & k, 10, MPI_COMM_WORLD, req_snd(j 1), ierr) k 2 = k 2 + k 3 ENDDO CALL MPI_WAITALL(Num. Neigh. Procs, req_rcv, stat_rcv, ierr) CALL MPI_WAITALL( Num. Neigh. Procs, req_snd, stat_snd, ierr) DO j 1 = 1, Num. Neigh. Procs k = Id. Neigh. Proc(j 1) k 1 = 1 DO j = 1, COMMON(j 1) k 2 = XCom(j 1)%node_list(j)*3 Rnet(k 2 -2)= Rnet(k 2 -2) + frmproc(k)%rcvbuf(k 1) Rnet(k 2 -1)= Rnet(k 2 -1) + frmproc(k)%rcvbuf(k 1+1) Rnet(k 2) = Rnet(k 2) + frmproc(k)%rcvbuf(k 1+2) k 1 = k 1 + 3 ENDDO

Application Typical Procedure for a Parallel FEM Code • Par. FUM • MPI Post-Processor • Typically needed to put the mesh back together for visualization • Typically Not Needed – Par. FUM will output the “serial” mesh

Conclusion • Par. FUM is considerably easier then creating an MPI version • MPI is not practical (in terms of development time) for applications needing adaptive load balancing • Par. FUM is continually adding new features of interest to the FE community: – Parallel Adaptive Mesh Refinement (including activation of cohesive elements) – Parallel contact detection and enforcement