Combinatorial and algebraic tools for multigrid Yiannis Koutis
Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science Department Carnegie Mellon University Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
multilevel methods • • • www. mgnet. org 3500 citations 25 free software packages 10 special conferences since 1983 Algorithms not always working Limited theoretical understanding Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
multilevel methods: our goals • provide theoretical understanding • solve multilevel design problems • small changes in current software • study structure of eigenspaces of Laplacians • extensions for multilevel eigensolvers Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
Overview • • • Quick definitions Subgraph preconditioners Support graph preconditioners Algebraic expressions Low frequency eigenvectors and good partitionings • Multigrid introduction and current state • Multigrid – Our contributions Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
quick definitions • Given a graph G, with weights wij • Laplacian: A(i, j) = -wij, row sums =0 • Normalized Laplacian: • (A, B) is a measure of how well B approximates A (and vice-versa) Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
linear systems : preconditioning • Goal: Solve Ax = b via an iterative method • A is a Laplacian of size n with m edges. Complexity depends on (A, I) and m • • Solution: Solve B-1 Ax = B-1 b Bz=y must be easily solvable (A, B) is small B is the preconditioner Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
Overview • • • Quick definitions Subgraph preconditioners Support graph preconditioners Algebraic expressions Low frequency eigenvectors and good partitionings • Multigrid introduction and current state • Multigrid – Our contributions Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
combinatorial preconditioners the Vaidya thread • B is a sparse subgraph of A, possibly with additional edges Solving Bz=y is performed as follows: 1. Gaussian elimination on degree · 2 nodes of B 2. A new system must be solved 3. Recursively call the same algorithm on to get an approximate solution. Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
combinatorial preconditioners the Vaidya thread • • • Graph Sparsification [Spielman, Teng] Low stretch trees [Elkin, Emek, Spielman, Teng] Near optimal O(m poly( log n)) complexity Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
combinatorial preconditioners the Vaidya thread • • • Graph Sparsification [Spielman, Teng] Low stretch trees [Elkin, Emek, Spielman, Teng] Near optimal O(m poly( log n)) complexity • • • Focus on constructing a good B (A, B) is well understood – B is sparser than A B can look complicated even for simple graphs A Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
Overview • • • Quick definitions Subgraph preconditioners Support graph preconditioners Algebraic expressions Low frequency eigenvectors and good partitionings • Multigrid introduction and current state • Multigrid – Our contributions Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
combinatorial preconditioners the Gremban - Miller thread • the support graph S is bigger than A 1 2 Carnegie Mellon School of Computer Science 1 3 1 2 2 1 1 05/11/2005 Aladdin Lamps 05
combinatorial preconditioners the Gremban - Miller thread • the support graph S is bigger than A 1 2 1 3 1 Carnegie Mellon School of Computer Science 1 4 1 2 2 1 2 3 3 2 3 4 3 3 1 Quotient 1 4 2 1 3 2 2 1 1 1 05/11/2005 Aladdin Lamps 05
combinatorial preconditioners the Gremban - Miller thread • • • The preconditioner S is often a natural graph S inherits the sparsity properties of A S is equivalent to a dense graph B of size equal to that of A : (A, S) = (A, B) • Analysis of (A, S) made easy by work of [Maggs, Miller, Ravi, Woo, Parekh] • Existence of good S by work of [Racke] Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
Overview • • • Quick definitions Subgraph preconditioners Support graph preconditioners Algebraic expressions Low frequency eigenvectors and good partitionings • Multigrid introduction and current state • Multigrid – Our contributions • Other results Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
algebraic expressions • • Suppose we are given m clusters in A R(i, j) = 1 if the jth cluster contains node i R is n x m Quotient • R is the clustering matrix Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
algebraic expressions • The inverse preconditioner • The normalized version • RT D 1/2 is the weighted clustering matrix Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
Overview • • • Quick definitions Subgraph preconditioners Support graph preconditioners Algebraic expressions Low frequency eigenvectors and good partitionings • Multigrid introduction and current state • Multigrid – Our contributions • Other results Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
good partitions and low frequency invariant subspaces • Suppose the graph A has a good clustering defined by the clustering matrix R • Let y be any vector such that Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
good partitions and low frequency invariant subspaces • Suppose the graph A has a good clustering defined by the clustering matrix R • Let y be any vector such that Theorem: ? t s e t y t quali The inequality is tight up to a constant for certain graphs Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
good partitions and low frequency invariant subspaces • Let y be any vector such that • Let x be mostly a linear combination of eigenvectors corresponding to eigenvalues close to Theorem: • Prove ? • We can find random vector x and check the distance to the closest y 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05
Overview • • • Quick definitions Subgraph preconditioners Support graph preconditioners Algebraic expressions Low frequency eigenvectors and good partitionings • Multigrid introduction and current state • Multigrid – Our contributions Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
multigrid – short introduction • General class of algorithms • Richardson iteration: • High frequency components are reduced: Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
initial and smoothed error initial error Carnegie Mellon School of Computer Science smoothed error 05/11/2005 Aladdin Lamps 05
the basic multigrid algorithm • Define a smaller graph Q • Define a projection operator Rproject • Define a lift operator Rlift how many? 1. 2. 3. 4. 5. which iteration ? Apply t rounds of smoothing Take the residual r = b-Axold Solve Qz = Rprojectr Form new iterate xnew = xold + Rlift z Apply t rounds of smoothing Carnegie Mellon School of Computer Science recursion is this needed ? 05/11/2005 Aladdin Lamps 05
algebraic multigrid (AMG) Goals: The range of Rproject must approximate the unreduced error very well. The error not reduced by smoothing must be reduced by the smaller grid. Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
algebraic multigrid (AMG) Goals: The range of Rproject must approximate the unreduced error very well. The error not reduced by smoothing must be reduced in the smaller grid. • • • Jacobi iteration: or ‘scaled’ Richardson: Find a clustering Rproject = (Rlift)T Q = Rproject. T A Rproject Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
algebraic multigrid (AMG) Goals: The range of Rproject must approximate the unreduced error very well. The error not reduced by smoothing must be reduced in the smaller grid. • • • Jacobi iteration: or ‘scaled’ Richardson Find a clustering [heuristic] Rproject = (Rlift)T [heuristic] Q = Rproject. T A Rproject Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
two level analysis • Analyze the maximum eigenvalue of • where • The matrix T 1 eliminates the error in • A low frequency eigenvector has a significant component in Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
two level analysis • Starting hypothesis: Let X be the subspace corresponding to eigenvalues smaller than . Let Y be the null space of Rproject. Assume, <X, Y>2 · / • Two level convergence : error reduced by • Proving the hypothesis ? Limited cases Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
current state ‘there is no systematic AMG approach that has proven effective in any kind of general context’ [BCFHJMMR, SIAM Journal on Scientific Computing, 2003] Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
Overview • • • Quick definitions Subgraph preconditioners Support graph preconditioners Algebraic expressions Low frequency eigenvectors and good partitionings • Multigrid introduction and current state • Multigrid – Our contributions Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
our contributions – two level • There exists a good clustering given by R. The quality is measured by the condition number (A, S) • Q = RT A R • Richardson’s with • Projection matrix Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
our contributions - two level analysis • Starting hypothesis: Let X be the subspace corresponding to eigenvalues smaller than . Let Y be the null space of Rproject = RTD 1/2 Assume, <X, Y>2 · / • • Two level convergence : error reduced by Proving the hypothesis ? Yes! Using (A, S) Result holds for t=1 smoothing Additional smoothings do not help Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
our contributions - recursion • There is a matrix M which characterizes the error reduction after one full multigrid cycle • We need to upper bound its maximum eigenvalue as a function of the two-level eigenvalues the maximum eigenvalue of M is upper bounded by the sum of the maximum eigenvalues over all two-levels Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
towards full convergence • Goal: The error not reduced by smoothing must be reduced by the smaller grid A different point of view The small grid does not reduce part of the error. It rather changes its spectral profile. Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
full convergence for regular d-dimensional toroidal meshes • A simple change in the implementation of the algorithm: • where • T 2 has eigenvalues 1 and -1 • T 2 xlow = xhigh Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
full convergence for regular d-dimensional toroidal meshes • With t=O(log n) smoothings • Using recursive analysis: max(M) · 1/2 • Both pre-smoothings and post-smoothings are needed • Holds for perturbations of toroidal meshes Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
Overview • • • Quick definitions Subgraph preconditioners Support graph preconditioners Algebraic expressions Low frequency eigenvectors and good partitionings • Multigrid introduction and current state • Multigrid – Our contributions Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
Thanks! Carnegie Mellon School of Computer Science 05/11/2005 Aladdin Lamps 05
- Slides: 40