DESIGN IN IMAGING FROM COMPRESSIVE TO COMPREHENSIVE SENSING
- Slides: 62
DESIGN IN IMAGING FROM COMPRESSIVE TO COMPREHENSIVE SENSING Lior Horesh Joint work with E. Haber, L. Tenorio, A. Conn, U. Mello, J. Fohring IBM TJ Watson Research Center SIAM Imaging Science 2012, Philadelphia, PA - May 2012
INTRODUCTION
EXPOSITION - ILL - POSED INVERSE PROBLEMS • Aim: infer model • Given • Experimental design y • Measurements • Observation model • Ill - posedness Naïve inversion. . . Fails. . . • Need to fill in the missing information
HOW TO IMPROVE MODEL RECOVERY ? • How can we. . . • Improve observation model ? • Extract more information in the measurement procedure ? • Define appropriate distance measures / noise model ? • Incorporate more meaningful a-priori information ? • Provide more efficient optimization schemes ?
EVOLUTION OF INVERSION FROM SIMULATION TO DESIGN • Forward problem (simulation, description) • Given: model m & observation model • Simulate: data d • Inverse problem (estimation, prediction) • Given: data d & observation model • Infer: model m (and uncertainties) • Design (prescription) • Given: inversion scheme & observation model • Find: the ‘best’ experimental settings y, regularization S, …
WHY DESIGN ?
WHICH EXPERIMENTAL DESIGN IS BEST ? Design 1 Design 2 Design 3
DESIGN QUESTIONS – DIFFUSE OPTICAL TOMOGRAPHY • Where light sources and optodes should be placed ? • In what sequence they should be activated ? • What laser source intensities should be used ? Arridge 1999
DESIGN QUESTIONS – ELECTROMAGNETIC INVERSION • What frequencies should be used ? • What trajectory of acquisition should be considered ? Newman 1996, Haber & Ascher 2000
DESIGN QUESTIONS – SEISMIC INVERSION • How simultaneous sources can be used ? Clerbout 2000
DESIGN QUESTIONS – LIMITED ANGLE TOMOGRAPHY • How many projections should we collect and at what angles ? • How accurate should the projection data be ?
WHAT CAN WE DESIGN ?
DESIGN EXPERIMENTAL LAYOUT Stonehenge 2500 B. C.
DESIGN EXPERIMENTAL PROCESS Galileo Galilei 1564 -1642
RESPECT EXPERIMENTAL CONSTRAINTS… French nuclear test, Mururoa, 1970
HOW TO DESIGN ?
ILL VS. WELL - POSED OPTIMAL EXPERIMENTAL DESIGN • Previous work • Well-posed problems - well established [Fedorov 1997, Pukelsheim 2006 ] • Ill-posed problems - under-researched [Curtis 1999, Bardow 2008 ] • Many practical problems in engineering and sciences are ill-posed What makes ill-posed problems so special ?
OPTIMALITY CRITERIA IN WELL-POSED PROBLEMS • For linear inversion, employ Tikhonov regularized least squares solution • Bias - variance decomposition • For over-determined problems • A-optimal design problem
OPTIMALITY CRITERIA IN WELL-POSED PROBLEMS • Optimality criteria of the information matrix • A-optimal design average variance • D-optimality uncertainty ellipsoid • E-optimality minimax • Almost a complete alphabet…
THE PROBLEMS. . . • Ill-posedness controlling variance alone reduces the error mildly [Johansen 1996 ] • Non-linearity bias-variance decomposition is impossible What strategy can be used ? Proposition 1 - Common practice so far Trial and Error…
EXPERIMENTAL DESIGN BY TRIAL AND ERROR • Pick a model • Run observation model of different experimental designs, and get data • Invert and compare recovered models • Choose the experimental design that provides the best model recovery
THE PROBLEMS. . . • Ill-posedness controlling variance alone reduces the error mildly [Johansen 1996 ] • Non-linearity bias-variance decomposition is impossible What other strategy can be used ? Proposition 2 - Minimize bias and variance altogether How to define the optimality criterion ?
OPTIMALITY CRITERIA FOR DESIGN • Loss û Depends on the noise û Depends on unknown model • Mean Square Error û Depends on unknown model • Bayes risk û Computationally infeasible
OPTIMALITY CRITERIA FOR DESIGN • Bayes empirical risk • Assume a set of authentic model examples is available • Discrepancy between training and recovered models [Vapnik 1998 ] How can y and S be regularized ? • Regularized Bayesian empirical risk
OTHER DESIGNERS / KEY PLAYERS • This is one doctrine • Other interesting choices were developed by • Y. Marzouk et. al, MIT (2011, 2012) • A. Curtis et. al, University of Edinburgh (1999, 2010) • D. Coles & M. Prange, Schlumberger (2008, 2012) • S. Körkel et. al, Heidelberg University (2011) • A. Bardow, RWTH Aachen University (2008, 2009)
DIFFERENTIABLE OBSERVATION SPARSITY CONTROLLED DESIGN • Assume: fixed number of observations • Design preference: small number of sources / receivers is activated • The observation model • Regularized risk Horesh, Haber & Tenorio 2011
DIFFERENTIABLE OBSERVATION SPARSITY CONTROLLED DESIGN • Total number of observations may be large • Derivatives of the forward operator w. r. t. y Difficult… • Effective when activation of each source and receiver is expensive Horesh, Haber & Tenorio 2011
WEIGHTS FORMULATION SPARSITY CONTROLLED DESIGN • Assume: a predefined set of candidate experimental setups is given • Design preference: small number of observations Haber, Horesh & Tenorio 2010
WEIGHTS FORMULATION SPARSITY CONTROLLED DESIGN • Let be discretization of the space [Pukelsheim 1994] • Let • The (candidates) observation operator is weighted • w inverse of standard deviation • - reasonable standard deviation - conduct the experiment • - infinite standard deviation - do not conduct the experiment Haber, Horesh & Tenorio 2010
WEIGHTS FORMULATION SPARSITY CONTROLLED DESIGN • Solution - more observations give better recovery • Desired solution many w‘s are 0 • Add penalty to promote sparsity in observations selection • Less degrees of freedom • No explicit access to the observation operator needed
THE OPTIMIZATION PROBLEMS • Leads to a stochastic bi-level optimization problem • Direct formulation • Weights formulation Haber, Horesh & Tenorio 2010, 2011
THE OPTIMIZATION PROBLEM(S) • In general (multi-level) optimization problem • Difficult to solve (although not impossible) [Alexaderov & Denis 1994 ] • To simplify make assumptions on • F - linear, nonlinear • S - quadratic, matrix form • y - discrete, continuous • Important - sufficient improvement can be “good enough”
HOW MANY SAMPLES ARE NEEDED ? • The optimist - choose a single m and design for it • The pessimist - assume that m is the worst it can be • The realist - choose an ensemble of m’s and average • A Bayesian estimate of the frequentist risk [Rubin 1984 ]
OPTIMAL DESIGN IN PRACTICE
LINEAR DESIGN
LINEAR DESIGN • Assume • Then
LINEAR DESIGN • Assume m has second moment • Then • Best linear design
CASE STUDY I LINEAR EXPERIMENTAL DESIGN
THE BEST (LINEAR) EXPERIMENT • Considering discretization of the design space • We obtain
BOREHOLE RAY TOMOGRAPHY OBSERVATION MODEL • Use 322 × 3 = 3072 rays • Goal: choose 500 optimal observations
BOREHOLE RAY TOMOGRAPHY EXPERIMENTAL DESIGN ASSESSMENT • Assess design performance upon unseen models (cross validation) Test model Non-optimal design Optimal design
NON - LINEAR DESIGN
NON - LINEAR DESIGN • Use Stochastic Optimization - approximate [Shapiro, Dentcheva & Ruszczynski 2009 ] by sampling
NON - LINEAR DESIGN • Solution through sensitivity calculation and therefore • The sensitivities
NON - LINEAR DESIGN • The reduced gradient • Can use steepest descent/LBFGS/Truncated Gauss-Newton to solve the problem
CASE STUDY II NON-LINEAR EXPERIMENTAL DESIGN
IMPEDANCE TOMOGRAPHY – OBSERVATION MODEL • Governing equations • Following Finite Element discretization • Given model • Find data and design settings , n<k • Design: find optimal source-receiver configuration
IMPEDANCE TOMOGRAPHY – DESIGNS COMPARISON Naive design True model Optimized design Horesh, Haber & Tenorio 2011
MAGNETO - TULLERICS TOMOGRAPHY – OBSERVATION MODEL • Governing equations • Following Finite Volume discretization • Given: model • Find: data and design settings , • Design: find an optimal set of frequencies (frequency )
MAGNETO - TELLURICS TOMOGRAPHY – DESIGNS COMPARISON Test model Optimal linearized design Naive design Optimized non-linear design Haber, Horesh & Tenorio 2008, 2010
THE PARETO CURVE – A DECISION MAKING TOOL Risk • To drill or not to drill ? Shakespeare 1564 -1616 Haber, Horesh & Tenorio 2010
CASE STUDY III REGULARIZATION DICTIONARY DESIGN
OPTIMAL OVER-COMPLETE DICTIONARY DESIGN • Use popular L 1 to get a sparse solution - but what dictionary should be used ? • Accounts for the model space, observation operator and noise characteristics of the problem • Requires more sophisticated algorithms to compute • Non-smooth optimization framework Modified L-BFGS [Overton 2003 ] Horesh & Haber 2009
OPTIMAL DICTIONARY DESIGN NUMERICAL RESULTS Horesh & Haber 2009
OPTIMAL DICTIONARY DESIGN ASSESSMENT WITH NOISE Horesh & Haber 2009
OPTIMAL DICTIONARY DESIGN GEOPHYSICAL ASSESSMENT Horesh & Haber 2009
SUMMARY
THE QUOTE CORNER • What I see in nature is a grand design that we can comprehend only imperfectly, and that must fill a thinking person with a feeling of humility — A. Einstein • Don’t design for everyone. It’s impossible. All you end up doing is designing something that makes everyone unhappy — L. Reichelt • It’s art if can’t be explained It’s fashion if no one asks for an explanation It’s design if it doesn’t need explanation — W. Stokkel
TAKE HOME MESSAGES • Only two (important) elements in the big (inversion) puzzle. . . • Experimental design • Regularization design • Design in ill-posed inverse problems is an important topic which requires more attention • New frontiers in inverse problems and optimization
EPILOGUE
DESIGN IN INVERSION – OPEN COLLABORATIVE RESEARCH • IBM Research • MITACS • University of British Columbia www. wix. com/ocrdesign/home
ACKNOWLEDGMENTS Michele Benzi Jennifer Fohring Andrew Conn Questions? Thank you Eldad Haber Hui Haung Raya Horesh Ulisses Mello Jim Nagy
- "green imaging" -g -"green imaging technologies"
- Engineering properties of rocks
- Spinal cord spasticity
- Which rock possesses very high compressive strength?
- Energy transfer
- Remote sensing platforms
- Pressure sensing elements
- Active passive remote sensing
- What is sensing
- Thinking feeling
- Advantages of remote sensing
- Primary sensing element example
- Digital number in remote sensing
- Demand sensing and shaping
- Limitations of remote sensing
- Idealized remote sensing system
- Pacemaker not sensing
- Nbg pacemaker code
- Aphg
- Ap human geography remote sensing
- What is stereoscopic parallax
- Strip camera in remote sensing
- Nuvations
- Remote sensing image
- Ifov and fov in remote sensing
- Geometry of aerial photography pdf
- Virtual carrier sensing
- National authority for remote sensing and space sciences
- The internship mbti
- Fiber optic force sensing catheter
- Characteristics of market driven strategy
- Capacitive rain sensor
- Canada centre for remote sensing
- Ap human geography unit 1
- Remote sensing applications center
- Remote sensing applications center
- Remote sensing physics
- Microwave remote sensing lecture notes
- Remote sensing in precision agriculture
- Aerial photography in remote sensing pdf
- Introduction to microwave remote sensing
- Remote sensing physics
- Image sensing and acquisition
- Tangential scale distortion
- Remote sensing
- "sensing systems"
- Process
- Light-sensing input device that reads printed text
- Hub
- Produce text and graphics on a physical medium
- Quality by design
- Applied remote sensing training program
- The sensing portion of a bi-metallic stem thermometer is:
- Advantages of remote sensing
- Remote sensing
- What type of wire is best used with u-cogged drive rolls
- Sensing clues
- Digital interpretation in remote sensing
- Extroverted sensing
- "sensing systems"
- Aditi padhi
- "sensing-systems"
- Sensing learners characteristics