LargeScale Methods in Inverse Problems Per Christian Hansen

Large-Scale Methods in Inverse Problems Per Christian Hansen Informatics and Mathematical Modelling Technical University of Denmark With contributions from: • Michael Jacobsen, Toke Koldborg Jensen - Ph. D students • Line H. Clemmensen, Iben Kraglund, Kristine Horn, Jesper Pedersen, Marie-Louise H. Rasmussen - Master students Large-Scale Methods in Inverse Problems

Overview of Talk A survey of numerical methods for large-scale inverse problems 1. 2. 3. 4. 5. 6. 7. Some examples. The need for regularization algorithms. Krylov subspace methods for large-scale problems. Preconditioning for regularization problems. Signal subspaces and (semi)norms. GMRES as a regularization method. Alternatives to spectral filtering. Many details are skipped, to get the big picture!!! Large-Scale Methods in Inverse Problems 2

Related Work Many people work on similar problems and algorithms: • Åke Björck, Lars Eldén, Tommy Elfving • Martin Hanke, James G. Nagy, Robert Plemmons • Misha E. Kilmer, Dianne P. Oleary • Daniela Calvetti, Lothar Reichel, Brian Lewis • Gene H. Golub, Urs von Matt • Uri Asher, Eldad Haber, Douglas Oldenburg • Jerry Eriksson, Mårten Gullikson, Per-Åke Wedin • Marielba Rojas, Trond Steihaug • Tony Chan, Stanley Osher, Curtis R. Vogel • Jesse Barlow, Raymond Chan, Michael Ng Recent Matlab software packages: • Restore Tools (Nagy, Palmer, Perrone, 2004) • MOORe Tools (Jacobsen, 2004) • Geo. Tools (Pedersen, 2005) Large-Scale Methods in Inverse Problems 3

Inverse Geomagnetic Problems Large-Scale Methods in Inverse Problems 4

Inverse Acoustic Problems Oticon/ Rhinometrics Large-Scale Methods in Inverse Problems 5

Image Restoration Problems blurring deblurring Io (moon of Saturn) You cannot depend on your eyes when your imagination is out of focus – Mark Twain Large-Scale Methods in Inverse Problems 6

Model Problem and Discretization Vertical component of magnetic field from a dipole Large-Scale Methods in Inverse Problems 7

The Need for Regularization: keep the “good” SVD components and discard the noisy ones! Large-Scale Methods in Inverse Problems 8

Regularization – TSVD & Tikhonov Large-Scale Methods in Inverse Problems 9

Singular Vectors (Always) Oscillate Large-Scale Methods in Inverse Problems 10

Large-Scale Aspects (the easy case) Large-Scale Methods in Inverse Problems 11

Large-Scale Aspects (the real problems) Toeplitz matrix-vector multiplication flop count. Large-Scale Methods in Inverse Problems 12

Large-Scale Tikhonov Regularization Large-Scale Methods in Inverse Problems 13

Difficulties and Remedies I Large-Scale Methods in Inverse Problems 14

Difficulties and Remedies II Large-Scale Methods in Inverse Problems 15

The Art of Preconditioning Large-Scale Methods in Inverse Problems 16

Explicit Subspace Preconditiong Large-Scale Methods in Inverse Problems 17

Krylov Signal Subspaces Smiley Crater, Mars Large-Scale Methods in Inverse Problems 18

Pros and Cons of Regularizing Iterations Large-Scale Methods in Inverse Problems 19

Projection, then Regularization Large-Scale Methods in Inverse Problems 20

Bounds on “Everything” Large-Scale Methods in Inverse Problems 21

A Dilemma With Projection + Regular. Large-Scale Methods in Inverse Problems 22

Better Basis Vectors! Large-Scale Methods in Inverse Problems 23

Considerations in 2 D … … Large-Scale Methods in Inverse Problems 24

Good Seminorms for 2 D Problems Large-Scale Methods in Inverse Problems 25

Seminorms and Regularizing Iterations Large-Scale Methods in Inverse Problems 26

Krylov Implementation Large-Scale Methods in Inverse Problems 27

Avoiding the Transpose: GMRES Large-Scale Methods in Inverse Problems 28

GMRES and CGLS Basis Vectors Large-Scale Methods in Inverse Problems 29

CGLS and GMRES Solutions Large-Scale Methods in Inverse Problems 30

The “Freckles’’ DCT spectrum spatial domain Large-Scale Methods in Inverse Problems 31

Preconditioning for GMRES Large-Scale Methods in Inverse Problems 32

A New and Better Approach Large-Scale Methods in Inverse Problems 33

(P)CGLS and (P)GMRES Large-Scale Methods in Inverse Problems 34

Io (moon of Saturn) Away From 2 -Norms q=2 Large-Scale Methods in Inverse Problems q = 1. 1 35

Functionals Defined on Sols. to DIP Large-Scale Methods in Inverse Problems 36

Large-Scale Algorithm MLFIP Large-Scale Methods in Inverse Problems 37

Confidence Invervals with MLFIP Large-Scale Methods in Inverse Problems 38

Many Topics Not Covered … • Algorithms for other norms (p and q ≠ 2). • In particular, total variation (TV). • Nonnegativity constraints. • General linear inequality constraints. • Compression of dense coefficient matrix A. • Color images (and color TV). • Implementation aspects and software. • The choice the of regularization parameter. Large-Scale Methods in Inverse Problems 39

“Conclusions and Further Work” I hesitate to give any conclusion – • • the work is ongoing; there are many open problems, lots of challenges (mathematical and numerical), and a multitude of practical problems waiting to be solved. Large-Scale Methods in Inverse Problems 40