Spatial and Spectral Evaluation of Image Fusion Methods

Spatial and Spectral Evaluation of Image Fusion Methods Sascha Klonus Manfred Ehlers Institute for Geoinformatics and Remote Sensing University of Osnabrück

Content n Introduction Image Fusion n Test Site n Fusion Results n Color Distortions n Evaluation Methods and Results n Ehlers Fusion n Conclusions and Future Work

Data Fusion: Why is it Necessary? n Remote sensors have different spatial resolution for panchromatic and multispectral imagery n The ratios vary between 1: 2 and 1: 5 n For multisensor fusion the ratios can exceed 1: 30 (e. g. Ikonos/Landsat)

Objectives of Image Fusion n Sharpen images n Improve geometric corrections n Provide stereo-viewing capabilities n Enhance certain features n Complement data sets n Detect changes n Substitute missing information n Replace defective data Pohl & van Genderen (1998)

Meaning of Pan-Sharpening Spatial panchromatic & high geometric resolution multi-/hyperspectral & high geometric resolution + Spectral multi-/hyperspectral image & low geometric resolution

Fusion Methods n Color Transformations w Modified IHS Transformation n Statistical Methods w Principal Component Merge n Numerical Methods w Brovey w CN Spectral Sharpening w Gram-Schmidt Spectral Sharpening w Wavelet based Fusion n Combined Methods w Ehlers Fusion

Test Site

Original Data Quickbird Multispectral image (2004 -09 -04) Quickbird Panchromatic image (2004 -09 -04) Formosat Multispectral image (2004 -01 -30) Ikonos Multispectral image (2005 -08 -03)

Single Sensor Fusion: Quickbird Fused with Fused CN with Spectral with Ehlers Brovey Sharpening Fused with Gram-Schmidt Quickbird Multispectral image Wavelet Fused with modified IHS Fused with PC

Multisensor Fusion: Ikonos Fused with Ehlers Fused with Fused CN with Fused Spectral with Gram-Schmidt Brovey with. Sharpening PC Fused with modified Wavelet IHS Ikonos Multispectral image

Multisensor Fusion: Formosat Fused with Ehlers Fused with modified with Gram-Schmidt PC Sharpening IHS Fused with Brovey Fused with Fused CN with Spectral Wavelet Formosat Multispectral image

Fusion Problem: Color Distortion n Panchromatic band has a different spectral sensitivity n Multisensoral differences (e. g. Ikonos and SPOT merge) n Multitemporal (seasonal) changes between pan and ms image data ØInconsistent panchromatic information is fused into the multispectral bands

Spectral Comparison Methods (1) n Visual (Structure and Colour Preservation) n RMSE s = standard deviation org = Original image fused = Fused image x = Mean n Correlation coefficients

Results RMSE Quickbird Ikonos Formosat Mod. IHS 7. 1822 11. 0714 1. 7792 PC 42. 5058 23. 8979 23. 3095 Brovey 222. 1732 143. 8303 14. 3147 CN-Sharpening 46. 7389 123. 5038 70. 4803 Gram-Schmidt 5. 6883 5. 1425 0. 4078 Wavelet 3. 1915 2. 5379 0. 0290 Ehlers 1. 0028 4. 3645 0. 4201

Results Correlation Coefficients Quickbird Ikonos Formosat Mod. IHS 0. 8737 0. 5731 0. 5892 PC 0. 8811 0. 6512 0. 6199 Brovey 0. 8611 0. 6020 0. 5677 CN-Sharpening 0. 8611 0. 6020 -0. 0546 Gram-Schmidt 0. 8690 0. 6774 0. 6110 Wavelet 0. 9698 0. 9719 0. 9720 Ehlers 0. 9510 0. 8094 0. 9324

Spectral Comparison Methods (2) Per Pixel Deviation Fused image (Formosat 2 m) Degraded to ground resolution of original image (Formosat = 8 m) Degrade Original multispectral image (Formosat 8 m) Result: Vector containing the deviation per pixel Band 1 2. 56 Band 2 2. 92 Band 3 3. 49 Band 4 3. 35

Mean Per Pixel Deviation Quickbird Ikonos Formosat IHS 27. 28 42. 95 11. 93 PC 48. 80 53. 27 22. 52 Brovey 209. 15 136. 62 19. 48 CN-Sharpening 48. 73 117. 61 70. 71 Gram-Schmidt 30. 35 45. 29 11. 99 Wavelet 7. 20 13. 47 3. 04 Ehlers 17. 28 25. 86 4. 29

Spatial Comparison Methods (1) - Edge Detection - Band 1 91. 16 % Band 2 92. 10 % Band 3 92. 64 % Mean 91. 96 %

Results Edge Detection Quickbird Ikonos Formosat Mod. IHS 92. 71 % 92. 44 % 95. 54 % PC 95. 10 % 93. 28 % 93. 44 % Brovey 94. 69 % 95. 16 % 97. 87 % CN-Sharpening 94. 69 % 95. 16 % 90. 69 % Gram-Schmidt 95. 02 % 95. 53 % 97. 82 % Wavelet 85. 00 % 83. 82 % 84. 81 % Ehlers 91. 85 % 90. 35 % 94. 40 %

Spatial Comparison Methods (2) Highpass Filtering Correlation Band 1 0. 8012 Band 2 0. 7820 Band 3 0. 7912 Mean 0. 7918

Highpass Correlation Results Quickbird Ikonos Formosat Mod. IHS 0. 9336 0. 9149 0. 9420 PC 0. 9900 0. 0021 0. 8073 Brovey 0. 9715 0. 9765 0. 9895 CN-Sharpening 0. 9714 0. 9764 -0. 0170 Gram-Schmidt 0. 9857 0. 9879 0. 9652 Wavelet 0. 3976 0. 3627 0. 3799 Ehlers 0. 8997 0. 8689 0. 9349

FFT Filter Based Data Fusion (Ehlers Fusion) Basis: IHS Transform and Filtering in the Fourier Domain Multispectral Image Panchromatic Image FFT Fourier Spectrum HPF Pan. HP FFT-1 FFT Fourier Spectrum LPF ILP+Pan. HP R‘ H H G‘ S S B‘ R I G B ILP IHS-1

Panchromatic image and its spectrum Original panchromatic image Panchromatic Spectrum

Filtersetting effects Intensity Filtered Panchromatic Spectrum Cut-off Frequency fn Frequency

Effects in the spatial domain Filtered panchromatic image Fused image

Filtersetting effects Intensity Filtered Panchromatic Spectrum Cut-off Frequency fn Frequency

Effects in the spatial domain Filtered panchromatic image Fused image

Filtersetting effects Intensity Filtered Panchromatic Spectrum Cut-off Frequency fn Frequency

Effects in the spatial domain Filtered panchromatic image Fused image

Results n Ehlers Fusion shows the best overall results in all images n It works also if the panchromatic Information does not match the spectral sensitivity of the merged bands (multitemporal and multisensoral fusion) n Its performance is superior to standard fusion techniques (IHS, Brovey Transform, PC Merge) n Wavelet preserves the spectral characteristics at the cost of spatial improvement n Ehlers Fusion is integrated in a commercial image processing system (Erdas Imagine 9. 1)

Future Work n Fusion of radar- and optical Data n Development of one method to evaluate the spatial and spectral quality of an fused image n Comparison with the algorithm of Zhang (PCI Geomatica) n Research on automation for filter design

Thanks for your Attention Questions? ? ?

Ehlers Fusion Program

Ehlers Fusion Program

Ehlers Fusion Program

Ehlers Fusion Program

Ehlers Fusion Program

Multispectral image and its spectrum Original multispectral intensity Multispectral intensity spectrum

Filtersetting effects Intensity Filtered multispectral spectrum Cut-off Frequency fn Frequency

Filtersetting effects Intensity Filtered multispectral spectrum Cut-off Frequency fn Frequency

Filtersetting effects Intensity Filtered multispectral spectrum Cut-off Frequency fn Frequency