Applied Mathematics in Defense Applications Andrea Bertozzi Department

Applied Mathematics in Defense Applications Andrea Bertozzi Department of Mathematics University of California, Los Angeles

Topics • Multisensor fusion • Image and human event fusion for statistical density estimation • Automated boundary tracking for autonomous robots and high dimensional imagery • Collaborative searching through swarming • Diffuse interface methods in imaging • Segmentation with corners • Imaging through turbulence • Direct sparse deblurring • Geographic profiling • Crime hotspots • Gang violence data • Predicting crime

Data Fusion – Multiple sensors • Pan Sharpening – panchromatic (greyscale) higher spatial resolution, multiband – lower spatial resolution – IKONOS and QUICKBIRD satellite • Hyperspectral sharpening – panchromatic obtained separately (may not be perfect match) – hyperspectral can have hundreds of bands – contain material information • Human Event data – events in space and time fused with geographical data (e. g. residential burgalaries) • Point sensor data – mobile sensor data

Data Fusion and Segmentation– Multiple sensors • Pan Sharpening – panchromatic (greyscale) higher spatial resolution, multiband – lower spatial resolution – IKONOS and QUICKBIRD satellite • Hyperspectral sharpening – panchromatic obtained separately (may not be perfect match) – hyperspectral can have hundreds of bands – contain material information • Human Event data – events in space and time fused with geographical data (e. g. residential burgalaries) • Point sensor data – mobile sensor data

Panchromatic signal is not a linear combination of isolated bands

Recent pansharpening techniques • • • IHS Brovey PCA Wavelet Fusion First variational approach: ’A Variational Model for P+XS Image Fusion’, Ballester, Casselles, Igual, Verdera, 2006

Intensity Hue Saturation Results Assumes panchromatic is a linear combination of spectral bands.

Variational Wavelet Pansharpening Michael Moeller, Todd Wittman, ALB, preprint

Wavelet matching – data must be registered to dyadic scaling (pansharpening)

Full VWP variational problem

Alternate VWP – avoids switching from wavelet to physical space

Numerical results

Hyperspectral data fusion Michael Moeller, Todd Wittman, and Andrea L. Bertozzi, A Variational Approach to Hyperspectral Image Fusion, Proc. SPIE Conference on Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XV. Orlando, Florida. April 2009.

Spatial detail inherited from master image – spectral detail from AVIRIS data

Spectral preservation - examples

Spectral angle is preserved

George Mohler, Andrea Bertozzi, Tom Goldstein, Stan Osher Fast TV regularization for 2 D Maximum penalized likelihood estimation, preprint 2009 • Method for estimating non-smooth probability densities • Important for estimating threat level based on event data and other intel. • TV based regularization allows for best estimation of densities with spatial discontinuities. • Computationally challenging in multi-D • Challenge solved using Split Bregman L 1 minimization technique. • Tested using V-fold Cross Validation with large 2 D data sets. • Tested on data from LAPD for residential burglaries.

Maximum Penalized Likelihood Estimation basic problem • • Estimate probability density u(x) from point data x 1, x 2, x 3, etc. • For discontinuous densities, choose R = TV General approach for regularizer R(u).

Example from San Fernando Valley Data courtesy of LAPD • • Point process data for residential burglaries No residences in area in middle Actual data TV method (new) kernel estimation (old) TV method is much closer to real problem, does not bleed threat level into region where threat is not active. Can be fused with other types of data, such as spatial visual, infrared, LIDAR etc as long as one has a model to incorporate this into the problem.

Density Estimation for Sparse Data Laura Smith, Matthew Keegan, Todd Wittman, ALB UCLA • Point data of individual events that come from a background source • Examples – human event activity – burglaries – what is the probability of event as a function of space? • Data is sparse – want to fuse with other information e. g. overhead imagery Improving Density Estimation by Incorporating Spatial Information, preprint 2009.

Overhead imagery vs. human events

San Fernando Valley Burglaries

Orange County Coastline

Experimental Validation of Cooperative Environmental Boundary Tracking with On-board Sensors A. Joshi, T. Ashley, Y. Huang, and A. L. Bertozzi, Experimental validation of cooperative environmental boundary tracking with on-board sensors, American Control Conference, St. Louis, MO, June 2009, pp. 2630 -2635.

Control Algorithms for Boundary Tracking • • • UUV-gas bang-bang type steering controller Time-corrected algorithm Robotic path planning – Hsieh et al Amer. Contrl. Conf. 2005 Jin and ALB, IEEE CDC 2008 Joshi et al Amer. Contrl. Conf. 2009 25

Bang-bang type steering Control Law • UUV-gas algorithm: 26

Time-corrected Steering Control Law • Time-corrected algorithm: • Includes time difference between crossing points on boundary, • Reduces to the bang-bang type controller when, 27

Decision Algorithm CUSUM Filter • Upper : indicates teal tape • Lower : indicates black tape 29

Single Vehicle Implementation • A Kalman inspired pre-filter was used to weakly damp the noisy signal. • Essentially a simple proportional model with the empirical gain factor from Kalman filtering a previous data set. n The Gain value is in fact the convergent Kalman gain (this is expected in view of the fairly constant, though high, noise covariance of the testbed) 30

Single Vehicle Implementation • The sharp features when on the black tape cause the decision algorithm to slip-up occasionally • Pre-filtering reduces these errors considerably 31

Single Vehicle Implementation Vehicle speed 0. 3 m/s Left raw data -> CUSUM Right raw data -> prefilter -> CUSUM 32

Cooperative boundary tracking • The global control law held admirably n The time difference between data points is 24 s 33

A. Chen, T. Wittman, A. Tartakovsky, and A. L. Bertozzi Image segmentation through efficient boundary sampling, in SAMPTA '09, Marseille, May 18 -22, 2009.

W. Liu, M. B. Short, Y. E. Taima, and A. L. Bertozzi, Multiscale Collaborative Searching Through Swarming, preprint. • • Sensor data processing – Kalman filter to reduce noise – CUSUM filter to check threshold Movement control of agents – Models for search phase, target locating phase – Target location estimated Performance – Average time to locate a target – Average error in estimate of location – Number of false registers Scaling properties – Estimate for swarm size, measured by diameter – Optimal swarm diameter (analytical approximation, upper bound) 35

Collaborative Searching through Swarming W. Liu, M. B. Short, Y. E. Taima, and A. L. Bertozzi Introduction 2 • Overview of the algorithm 36

Collaborative Searching through Swarming W. Liu, M. B. Short, Y. E. Taima, and A. L. Bertozzi Movement Control of Agents • Search phase (with Levy flight) • Target locating phase • Location estimation 37

Collaborative Searching through Swarming W. Liu, M. B. Short, Y. E. Taima, and A. L. Bertozzi Sensor data processing • Agent sensor reading • Kalman filter to reduce noise • CUSUM filter to detect threshold 38

Collaborative Searching through Swarming W. Liu, M. B. Short, Y. E. Taima, and A. L. Bertozzi Performance • • Simulations of a 20 by 20 dimensionless board, 32 agents, target sensing radius 1. 0, 200 trials Divide-and-conquer and whole region search strategies Average time to locate a target and average location estimate error measured Larger swarms are more accurate, multiple smaller ones more efficient 39

Collaborative Searching through Swarming W. Liu, M. B. Short, Y. E. Taima, and A. L. Bertozzi Scaling Properties • • Swarm diameter D scales with interagent distance For 25% of agents to sense before deciding to locate, Optimal D maximizes separation between the center of the swarm and the target location The average time to locate a target is 40

Diffuse interface methods Total variation Ginzburg-Landau functional

Cahn-Hilliard Inpainting Bertozzi, Esedoglu, Gillette, IEEE Trans. Image Proc. 2007, SIAM MMS 2007 Patent pending. Transitioned to NGA for road inpainting. Transitioned to In. Qtel for document exploitation. Continue edges in the same direction – higher order method for local inpainting. Fast method using convexity splitting and FFT H-1 gradient flow for diffuse TV L 2 fidelity with known data

Wavelet Allen-Cahn Image Processing • Dobrosotskaya, Bertozzi, IEEE Trans. Image Proc. 2008, Preprint subm. IFB. • Transitioned to NGA for road inpainting. Transitioned to In. Qtel for document exploitation. • Nonlocal wavelet basis replaces Fourier basis in classical diffuse interface method. • Analysis theory in Besov spaces. • Gamma convergence to anisotropic TV. H-1 gradient flow for diffuse TV L 2 fidelity with known data

Schoenlieb and Bertozzi, submitted Convex Splitting Schemes Basic idea: Art is to choose Ec to give an implicit problem that is easy to solve - e. g. Ec is H 1 semi norm – can be solved using FFT - in wavelet case Ec is wavelet Laplace operator Contraints on Ec and Ee so that splitting is unconditionally stable Proof of convergence of splitting schemes for various higher order inpainting methods.

Marc Droske and Wenhua Gao Segmentation with Corners Idea – segmentation requires a regularization It is analogous to denoising. CV, Snakes reduce length of curve. Removes corners as well as noise. Image Snakes (KWT ‘ 88) Instead regularize with the Chan-Vese 2001 “curve” analogy of TV – Droske & Bertozzi – nonlinear penalization of geometric corner snakes curvature-based functional. 2009 Low Curvature Image Simplifiers (Tumblin & Turk SIGGRAPH 2000, Bert. And Greer CPAM 2004) Extend to curve evolution using either Lagrangian framework or Level sets. CV with corners 2009 Higher-order feature-preserving geometric regularization, SIAM J. Img. Sci. 2010.

Imaging through turbulence morning afternoon Images taken at China Lake – courtesy of Alan Vannevel and Gary Hewer When you image at a kilometer anisplanatic effects are relevant – we need better deblurring and deconvolution methods. Often imformation is known about the image – the difficulty is to extract features.

Lou, Bertozzi, Soatto, submitted 2009 Direct Sparse Deblurring Uses training data Dictionary based Blurry data ROF deblurring Our method Inverse problem not solved Fit data to blurred dictionary then directly unblur Solves problem of amplifying noise with solution of inverse problem

Geographic Profiling George Mohler and Martin Short

Estimation of probabilities

2004 San Fernando Valley Data

2007 Los Angeles Burglary Data

M. B. Short, M. R. D'Orsogna, V. B. Pasour, G. E. Tita, P. J. Brantingham, A. L. Bertozzi, and L. Chayes, A statistical model of criminal behavior, M 3 AS: Mathematical Models and Methods in Applied Sciences, special issue on Traffic, Crowds, and Swarms, volume 18, Supp. , pages 1249 -1267, 2008.

G. Mohler, M. Short, P. Brantingham, F. Schoenberg, and G. Tita, Self-exciting point process modeling of crime, submitted.

Gang violence Data Mike Egesdal, Chris Fathauer, Kym Louie, and Jeremy Neuman, Statistical Modeling of Gang Violence in Los Angeles, to appear in SIURO online.

Gang violence Data

Courtesy of George Mohler Burglary 2005 Robbery 2007

Acknowledgments ONR grant N 000141010221, Information Fusion of Human Activity, Social Networks, and Geography Using Fast Compressive Sensing, co PIs Stan Osher, Jeff Brantingham, George Tita (UCI). ONR grant N 000140810363 Geometry Based Image Analysis and Understanding, 1/0812/10. ARO grant (STIR) W 911 NS-09 -1 -0559 Mathematical modeling of insurgent activities as compared to urban street crime, 10/09 -6/10. ARO MURI grant 50363 -MA-MUR Spatio-temporal event pattern recognition subcontract from Brown University, Boris Rozovsky, PI. The Department of Defense. NSF grant DMS-0914856 Algorithms for Threat Detection (ATD): adaptive sensing and sensor fusion for real time chemical and biological threats, (jointly funded by DTRA). NSF grant DMS-0601395 Research Training Group in Applied Differential Equations and Scientific Computing, 6/06 -5/11 (supports undergraduate research).