Inverse problems in earth observation cartography Inverse Problems

Inverse problems in earth observation & cartography Inverse Problems in Earth Observation and Cartography Theme Cog. B

2 Project and People u u u Joint CNRS/INRIA/UNSA project team, created in 1998. Project leader: Josiane Zerubia. Members: n 3 INRIA: J. Zerubia (DR 1), X. Descombes (CR 1), I. Jermyn (CR 1). n 1 CNRS: L. Blanc-Féraud (DR 2). n 1 postdoc. n 10 Ph. D students. n 5 Masters students (ENS Cachan, Sup’Aéro, ENAC, CPE Lyon). n 5 student Interns (ETH Zurich, Szeged University , Sup’Com Tunis, IIT New Delhi, IIT Roorkee). n 1 assistant INRIA (50%), 1 assistant CNRS/UNSA (10%).

3 Research domain

4 Principal goals Taking into account the physics of the sensor (visible, infrared, radar, laser, …). u Extraction of information relevant for high-level interpretation. u Reconstruction of 3 D information (digital elevation map) from 2 D data. u Map updating for cartography. u

5 Application areas u Commercial n u Public interest n u Precision farming, GIS, … Forestry management, environmental monitoring, urban planning, cartography, … Homeland security n Intelligence, pre- and post-mission analysis, …

6 Scientific foundations u Probabilistic approaches n n u Variational approaches n n u MRFs: geometric properties, MRFs on trees, … Stochastic geometry: marked point processes. Regularization and functional analysis: BV space, -convergence, … Contours and regions: level sets, higher-order active contours, … Optimization/Parameter estimation n n MCMC, RJMCMC, … Diffusion processes.

7 Main contributions 2001 -2005

8 Probabilistic models (I) u Markov Random Fields n n u Markov Random Fields for hyperspectral data [IEEE TGRS 04] New models on trees for segmentation [IEEE TGRS 05] Impact n n Lyapunov Inst. (01 -02); NATORussia (03 -04); ECO-NET (05). Contract and transfers: Alcatel Alenia Space Cannes.

9 Probabilistic models (II) u Marked Point Processes n n u A new general modelling framework [IEEE SP Mag. 02, IJCV 04, IEEE PAMI 05, IJCV to appear]. New kernels for RJMCMC optimization [LNS Springer 05]. Impact n n n PNTS (04 -05); COLORS Arbres (01); ARC MODE de VIE (05 -06). Contracts and transfers: DGA; IGN; BRGM. Eusipco 04 Young Author Best Paper award.

10 Variational methods (I) u Image decomposition [JMIV 05] n n n u New algorithm for decomposing an image into geometry (in BV) + oscillations (in G). Numerically challenging (L 1 norm). Applications: restoration, compression, inpainting… Impact n n n Math/STIC (02 -03); ACI Multim (04 -07). Used for IR target detection by DGA. Best Ph. D thesis prize 05 from EEA Club, Signal and Image section.

11 Variational methods (II) u Higher-order active contours n n u New method for the inclusion of geometric information in active contour models. [IJCV to appear] Reformulation as phase fields to simplify implementation and allow model learning. [ICCV 05] Impact n n n EU No. E MUSCLE (04 -08); ACI Query. Sat (04 -06). Contract: Alcatel Alenia Space Cannes. Transfers: U. Szeged, Hungary; LIAMA, China. n(p) n(p 0) p p 0 Long-range interactions Effect of prior

12 Parameter Estimation u u u Estimation of model hyperparameters for deconvolution of HR visible and TIR data. Estimation of sensor parameters for blind deconvolution of HR visible data. Impact n n n Contracts and transfers: CNES; Astrium; Sagem. Patent #0110189 (France: 01; Europe, Israel, USA, Canada, Japan: 02). Usage rights granted to French Space Agency for future satellites.

13 Application to other domains: astrophysics u Image deconvolution n u Galaxy filament detection n u Using complex wavelet packets. Using a marked point process. Impact n n n COLORS DECONVOL (02); COLORS FIGARO (05). Transfers: Côte d’Azur Observatory. First automatic extraction of galaxy filaments from a real galaxy catalogue (provided by Harvard).

14 Application to other domains: confocal microscopy u Deconvolution n u Using Richardson-Lucy algorithm taking into account the edges of the object (TV regularization) [MRT Journal to appear] Impact n n ARC De. Mi. Tri (03 -04); P 2 R Franco Israeli Programme (05 -06). Transfers: Pasteur and Weizmann Institutes.

15 Project-team positioning

16 Positioning: INRIA scientific challenges u Develop multimedia data and multimedia information processing (40%): n n n u Couple models and data to simulate and control complex systems (15%) n u EU project MOUMIR (00 -04). EU No. E MUSCLE (04 -08). ACI Query. Sat (04 -07) ARC MODE de VIE (05 -06) Model living structures and mechanisms (15%) n n ARC De. Mi. Tri (03 -04) P 2 R Franco-Israeli programme (05 -06)

17 Positioning: INRIA (I) u Digiplante: peer. n u Imedia: peer. n n u Strong collaboration on forestry growth modelling (ARC Mode de Vie). Joint partners in EU No. E MUSCLE & ACI Query. Sat. Strongly complementary roles: Imedia = database retrieval; Ariana = image processing. Vista: peer. n n Joint partners in EU No. E MUSCLE. Benchmarking of segmentation methods developed in the two project-teams.

18 Positioning: INRIA (II) u Clime: peer. n u Epidaure/Asclepios: peer. n n u Joint partners in EU IMAVIS project. Complementary work on biological microscopy imaging but otherwise different applications. Mistis: peer/competitor. n n u Works on satellite data, but mainly for atmospheric & meteorological modelling & on time series. Collaboration with previous project-team IS 2. Methodological overlap (MRFs, variational, & comparison) but different applications. Odyssée: peer/competitor. n n n Joint partners in EU IMAVIS project. Shape from shading: different approaches (stochastic vs. PDEs). Variational methods (different applications) & shape information (different approaches).

19 Positioning: France (I) u Signal & Image Processing department, ENST, Paris: peer/competitor. (Campedel, Maître, Nicolas, Roux, Sigelle, Tupin) n n u Main competitor in France for remote sensing applications. Joint partners in ACI Query. Sat & in PNTS initiative; joint Ph. D. PASEO group, MIV team, LSIIT, Strasburg: peer. (Collet, Jalobeanu) n New group: possible collaboration on remote sensing.

20 Positioning: France (II) u CMLA, ENS Cachan: peer. (Aujol, Chalmond, Morel, Younes) n n u CMAP, École Polytechnique, Palaiseau: peer. (Chambolle) n u Joint partners in ACI Multim & MATH/STIC project. Complementary work on variational and stochastic methods. Joint partners in ACI Multim & joint work on functional analysis for image processing. CEREMADE/Paris XIII: peer/competitor. (Cohen, Dibos) n n Variational methods for shape description: different approaches but some overlap of application domains. Complementary work on TV regularization and PDEs.

21 Positioning: international (I) u Stochastic geometry: n n Ariana is one of the key developers of object processes for image analysis and understanding. Collaboration with CWI (van Lieshout). Peer. Related to work at Brown (Grenander), Florida State (Srivastava). Simpler objects but complex inter-object interactions. Peer. German Space Agency DLR and U. Jaume I have adapted models developed by Ariana. Peer/competitor.

22 Positioning: international (II) u Higher-order active contours: n n Distinct from other methods for the inclusion of prior geometric information. Better adapted to remote sensing than the many template-based methods: u U. Bonn (Cremers); U. Florida (Chen, Thiruvenkadam, Huang, Wilson, Geiser); Yale University (Tagare); U. Mannheim (Kohlberger, Schnorr); UCLA (Soatto); Saarland University (Weickert); Siemens Corporate Research (Paragios); MIT (Leventon, Grimson, Faugeras); … Peers/competitors.

23 Positioning: international (III) u Functional analysis for image processing: n n Image Processing group at UCLA (Chan, Esedoglu, Osher, Vese). Also work on functional analytic and PDE approaches. Peer/competitor. U. Pompeu Fabra Barcelona (Caselles). Related work on image inpainting, and strong links via exchange of students and researchers. Peer. U. Minnesota (Sapiro). Works more on PDEs than variational methods. Peer/competitor. CNR Rome (March). Collaboration on convergence. Peer.

24 Evolution of the objectives during 2001 -2005 u All the objectives defined in 2001 have been reached except: n The work on parameter estimation for blind deconvolution of optical and infrared satellite images was stopped due to legal problems over industrial property between INRIA-CNRS vs CNES or Sagem). u u Despite patent deposit in 2001 and extension in 2002. Two new topics emerged: n n Astrophysical image processing. Biological image restoration and deconvolution.

25 Recommendations from previous evaluation u Ariana has continued to: n n n u Develop innovative theoretical methods Publish in the best international journals and conferences. Maintain high international visibility. Ariana has increased its activity in: n Validation: u u n Applications: u n Via data and ground truth obtained from end users (CNES, IAURIF, IGN, IFN, BRGM, Alcatel Alenia Space). Via end user validation and use (IGN, BRGM). Environmental research (IFN, Alcatel Alenia Space, Silogic): trees, fires, … Image retrieval: u Collaborations with INRIA project-teams, French and European researchers (ACI Query. Sat, EU projects MOUMIR and MUSCLE, Math/STIC Tunisia) and end users (CNES, DLR).

26 Goals for the next four years

27 Future: probabilistic methods u Multi-object processes. n n u Different types of objects (e. g. roads, buildings, trees, …) will be included in a single model in which they mutually interact. High-risk. Improved models of texture. n n Quartic models of the joint statistics of adaptive wavelet packet coefficients. Medium-risk.

28 Future: variational methods u Image decomposition & restoration. n n u Detection of objects of codimension > 1 in 3 D images. n n u Multispectral extension using inter-band correlation. Medium-risk. New functionals based on Ginzburg-Landau theory. High-risk. Higher-order active contours & phase fields. n n Multiscale/wavelet models. Medium- to high-risk.

29 Future: parameter estimation u Parameter estimation for marked point processes using new diffusion processes & comparison to RJMCMC. n u Model learning for higher-order active contours & phase field models. n u High-risk. Parameter learning and resolution dependence for adaptive wavelet packet texture models. n u Medium- to high-risk. Medium-risk. Estimation of image acquisition parameters for the blind deconvolution of microscopy images. n High-risk.

30 Future: applications u Automatic urban scene analysis & cartography. n n n u Forest monitoring (tree population counting & classification, disaster damage evaluation). n n u Using multi-object marked point processes. Using phase field higher-order active contours. High-risk. Using marked point processes & Markov random fields. Medium-risk. Blind deconvolution of microscopy imagery. n n Using physical models of the image acquisition process and TV and wavelet regularizations. Medium- to high-risk.

31 Some statistics 2001 -2005 u Publications: n n n u Ph. Ds & Habilitations: 13. Books edited: 2. Journal articles + book chapters: 44. Conference articles: 80. Research reports: 49. Contracts: n n Industrial: 15. Academic: 29. u Software: n n u u Transfers: 13. Patents: 2. Student interns: 23. Visiting scientists: 62. Seminars: ~100. Teaching: ~550 h taught.