Parallel Computing Sciences Department Multilevel Combinatorial Methods in

Parallel Computing Sciences Department Multilevel Combinatorial Methods in Scientific Computing Bruce Hendrickson Sandia National Laboratories Parallel Computing Sciences Dept. MOV’ 01

An Overdue Acknowledgement Parallel Computing Sciences Department l Parallel Computing Uses Graph Partitioning l We owe a deep debt to circuit researchers » » KL/FM Spectral partitioning Hypergraph models Terminal propagation MOV’ 01

In Return … Parallel Computing Sciences Department l We’ve given you » Multilevel partitioning » h. METIS l Our applications are different from yours » » Underlying geometry More regular structure Bounded degree Partitioning time is more important – Different algorithmic tradeoffs MOV’ 01

Multilevel Discrete Algorithm Parallel Computing Sciences Department l Explicitly mimic traditional multigrid l Construct series of smaller approximations » Restriction l Solve on smallest » Coarse grid solve l Propagate solution up the levels » Prolongation l Periodically perform local improvement » Smoothing MOV’ 01

Lots of Possible Variations Parallel Computing Sciences Department l More complex multilevel iterations » E. g. V-cycle, W-cycle, etc. » Not much evidence of value for discrete problems l Key issue: properties of coarse problems » Local refinement = multi-scale improvement l I’ll focus on graph algorithms » Most relevant to VLSI problems MOV’ 01

Not a New Idea Parallel Computing Sciences Department l Idea is very natural » Reinvented repeatedly in different settings l Focus of this workshop is on heuristics for hard problems l Technique also good for poly-time problems » E. g. Geometric point detection (Kirkpatrick’ 83) MOV’ 01

Planar Point Detection Parallel Computing Sciences Department l. O(n log n) time to preprocess l. O(log n) time to answer query MOV’ 01

Multilevel Graph Partitioning Parallel Computing Sciences Department l Invented Independently Several Times » » Cong/Smith’ 93 Bui/Jones’ 93 H/Leland’ 93 Related Work – Garbers/Promel/Steger’ 90, Hagen/Khang’ 91, Cheng/Wei’ 91 – Kumar/Karypis’ 95, etc. l Multigrid Metaphor H/Leland’ 93 (Chaco) » Popularized by Kumar/Karypis’ 95 (METIS) MOV’ 01

Multilevel Partitioning Parallel Computing Sciences Department l l l Construct Sequence of Smaller Graphs Partition Smallest Project Partition Through Intermediate Levels » Periodically Refine l Why does it work so well? » Refinement on multiple scales (like multigrid) » Key properties preserved on (weighted) coarse graphs – (Weighted) partition sizes – (Weighted) edge cuts » Very fast MOV’ 01

Coarse Problem Construction Parallel Computing Sciences Department 1. 2. 3. Find maximal matching Contract matching edges Sum vertex and edge weights Key Properties: Preserves (weighted) partition sizes Preserves (weighted) edge cuts Preserves planarity Related to min-cut algorithm of Karger/Stein’ 96 MOV’ 01

Extension I: Terminal Propagation Parallel Computing Sciences Department l Dunlop/Kernighan’ 85 » Skew partitioning to address constrained vertices l Also useful for parallel computing » Move few vertices when repartitioning » Assign neighboring vertices to near processors » H/Leland/Van Dreissche’ 96 l Basic idea: » Vertex has gain-like preference to be in particular partition MOV’ 01

Multilevel Terminal Propagation Parallel Computing Sciences Department l How to include in multilevel algorithm? l Simple idea: » When vertices get merged, sum preferences » Simple, fast, effective » Coarse problem precisely mimics original MOV’ 01

Extension II: Finding Vertex Separators Parallel Computing Sciences Department l Useful for several partitioning applications » E. g. sparse matrix reorderings l One idea: edge separator first, then min cover » Problem: multilevel power on wrong objective l Better to reformulate multilevel method » Find vertex separators directly » H/Rothberg’ 98 MOV’ 01

Multilevel Vertex Separators Parallel Computing Sciences Department l Use same coarse constructor » Except edge weights don’t matter l Change local refinement & coarse solve » Can mimic KL/FM l Resulted in improved matrix reordering tool » Techniques now standard MOV’ 01

Extension III: Hypergraph Partitioning Parallel Computing Sciences Department Coarse construction » Contract pairs of vertices? » Contract hyperedges? l Traditional refinement methodology l See talk tomorrow by George Karypis MOV’ 01

Envelope Reduction Parallel Computing Sciences Department l Reorder rows/columns of symmetric matrix to keep nonzeros near the diagonal MOV’ 01

Graph Formulation Parallel Computing Sciences Department l Each row/column is a vertex Nonzero in (i, j) generates edge eij l For row i of matrix (vertex i) l » Env(i) = max(i-j such that eij in E) » Envelope = Env(i) l Find vertex numbering to minimize envelope » NP-Hard MOV’ 01

Status Parallel Computing Sciences Department l Highest Quality algorithm is spectral ordering » Sort entries of Fiedler vector (Barnard/Pothen/Simon’ 95) » Eigenvector calculation is expensive » Fast Sloan (Kumfert/Pothen’ 97) good compromise l Now multilevel methods are comparable » (Boman/H’ 96, Hu/Scott’ 01) l Related ordering problems with VLSI relevance » Optimal Linear Arrangement – Minimize |i-j| such that eij in E MOV’ 01

Challenges for Multilevel Envelope Minimization Parallel Computing Sciences Department l No precise coarse representation » Can’t express exact objective on coarse problem l No incremental update for envelope metric » I. e. no counterpart of Fiduccia/Mattheyses l Our solution: Use approximate metric » 1 -sum / minimum linear arrangement » Allows for incremental update – But still not an exact coarse problem » VLSI applications? MOV’ 01

Results Parallel Computing Sciences Department l Disappointing for envelope minimization » We never surpassed best competitor – Fast Sloan algorithm » But Hu/Scott’ 01 succeeded with similar ideas l Better for linear arrangement, but … » Not competitive with Hur/Lillis’ 99 – Multilevel algorithm with expensive refinement MOV’ 01

Lessons Learned Parallel Computing Sciences Department l Good coarse model is the key » » l Need to encode critical properties of full problem Progress on coarse instance must help real one Must allow for efficient refinement methodology Different objectives require different coarse models Quality/Runtime tradeoff varies w/ application » Must understand needs of your problem domain » For VLSI, quality is worth waiting for » All aspects of multilevel algorithm are impacted MOV’ 01

Conclusions Parallel Computing Sciences Department l Appropriate coarse representation is key » Lots of existing ideas for construction coarse problem – Matching contraction, independent sets, fractional assignment, etc. l Multigrid metaphor provides important insight » We’re not yet fully exploiting multigrid possibilities » Do we have something to offer algebraic multigrid? l Need for CS recognition of multilevel paradigm » Rich, general algorithmic framework, but not in any textbook » Not the same as divide-and-conquer MOV’ 01

Acknowledgements Parallel Computing Sciences Department l Shanghua Teng » "Coarsening, Sampling and Smoothing: Elements of the Multilevel Method“ l Rob Leland Erik Boman Ed Rothberg Tammy Kolda Chuck Alpert l DOE MICS Office l l MOV’ 01