Hyperspectral Imaging Techniques Issues and Applications ANTONY LAM

Hyperspectral Imaging: Techniques, Issues, and Applications ANTONY LAM HUMAN-MACHINE INTERACTION SYSTEMS ENGINEERING

Biography �Name: Antony Lam �Home Town: Los Angeles �Education (Graduated Sept. 2010) Ph. D in Computer Science (Computer Vision, Machine Learning); University of California, Riverside �Postdoc (Oct. 2010 – June 2014) National Institute of Informatics, Tokyo �Assistant Professor (July 2014 -Present) Saitama University

Quiz �There will be a short online quiz. You can do the quiz after the talk. Please finish it by Monday. Link: http: //goo. gl/forms/5 ONgty 6 d 64 You can also find the link on my homepage: http: //yankee. cv. ics. saitamau. ac. jp/~kunolab/antonylam/

Click!

Click!

Enter Name, ID, E-mail

Remember to submit!

Light Waves and Color Light

Prism Splitting White Light into Different Colors

Color Images Need spectrum over wavelengths to described color. Typical RGB Color Image (Only 3 Bands) Why do we only have three colors in images?

Human Vision is Trichromatic

Spectral Distribution of a Point

Specific Wavelengths can Reveal Characteristics of an Object Sugar Distribution of Melon Imaged at 676 nm (top) and RGB Image (bottom) Taken from Sugiyama (1999)

Different wavelengths reveal characteristics of an object. (E. g. strawberry. ) Bruising easily identified at 1000 nm. Bruising not easily identified at 640 nm. Taken from “Computer Vision Technology for Food Quality Evaluation” Edited by Da -Wen Sun

Hyperspectral Imaging Applications Satellite Imaging Medical Diagnostics Food Quality Inspection …and more!

How to Capture? x y

How to Capture? Standard Methods (Slow and noisy. ) Fastest but expensive (May have lower resolution. ) Prototype introduced in 2014. (Needs moving of imaging equipment to cover the entire cube. )

Narrowband Filter to Only Let Certain Wavelengths In

Hy. Spex (Norsk Elektro Optikk AS) Collimation makes light rays parallel. Diffraction Grating Splits Light Like a Prism

Optics Split Light Make Light Rays into Different Spectral Parallel Channels Limits Field of View Result is Multiple Band Images

Combines Ideas of Camera Obscura with a Grating Connects Wavelength and Viewing Directions Grusche (Applied Optics 2014)

Combines Ideas of Camera Obscura with a Grating Connects Wavelength and Viewing Directions Full Spectral Cube Scanned by Moving the Camera Grusche (Applied Optics 2014)

Problems with Hyperspectral Imaging �Noise �Low Resolution �Slow �We address these problems.

Denoising: Sample Result

Past Work Spatial Structure Improvement of target-detection algorithms based on adaptive threedimensional filtering. Bourennane, et al. , 2011 y Adaptive linear minimum mean square error restoration: influence on hyperspectral detection strategy. Gaucel, et al. 2008 Noise removal from hyperspectral images by multidimensional filtering. Lextexierand Bourennane. 2008 x Denoised HSI

Past Work Separable Spatial and Spectral Domains Denoising of hyperspectral imagery using principal component analysis and wavelet shrinkage. Chen and Qian, 2011 Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage. Othman and Qian, 2006 y λ x Denoised HSI

Our Approach y Leverage Statistics λ x Denoised HSI

Why Use the Spectral Domain? �Real life spectra have less varied patterns than spatial patches. 6 -8 Principal Components Dozens of Principal Components

Many Spectra in a Single HSI λ Say band image is of size 300 X 300. There are 90, 000 spectra.

Overview of Our Method Spectral Domain Processing Spatial Domain Processing

Denoising Results

Noisy Ours

520 nm Noisy Original Ours

High Resolution Hyperspectral Imaging Problem: Hyperspectral Camera Spatial Resolution is Low Combine High-Res RGB Low-Res Hyperspectral High-Res Hyperspectral Image Kawakami et al. (CVPR 2011)

High Resolution Hyperspectral Imaging Input RGB Input (550 nm) Low Resolution Band Image Estimated (550 nm) Input (620 nm) Estimated High Low Resolution Band Image Estimated High Resolution Band Image Kawakami et al. (CVPR 2011)

Different Lighting Changes Color Appearance

Reflectance L(400 nm)R(400 nm) = 1*0. 8 = 0. 8 80% Reflectance at 400 nm 1 400 nm light 0. 8 Object Surface Finding R is important for analyzing material. Reflected Light at Wavelength = L( )R( )

Fundus (眼底) Hyperspectral Imaging �Fundus Hyperspectral Imaging For diagnostic purposes. Take images of eye over different wavelengths. Camera Diagnostic Hyperspectral Image

Drawback of Standard Fundus Photography Need to change color filter. Slow Patient remains still for about 31 band images. Using incident lights from fundus statistics Recovered Spectral Reflectance E. g. only four images required instead of 31 in standard methods.

Fast Fundus Photography Advantages Fast, Less Noise ELS But… Capture Images under 4 Incident Lights Recovered Images for Medical Diagnosis Drawbacks ELS is expensive. Need to synchronize with camera. +

Informative Sparse Wavelengths Six Most Informative Wavelengths Database of Fundus Spectral Reflectance Statistics Capture Only Under Six Wavelengths Estimate Full Spectra Full Hyperspectral Image (E. g. 31 bands)

Sparse Wavelength Results 560 nm 630 nm Ground Truth Estimated

Fast Spectral Reflectance Recovery Using DLP Projector Fast color switch with fast camera can capture scene under multiple lights quickly. The spectral reflectance can be recovered for moving targets. Taken from Han et al. (ACCV 2012)

Application: Face Recognition Pose Expression Lighting

Application: Face Recognition NIR Spectral Reflectance of Skin: 1. Sense Subsurface Tissue 2. Stable Over Time 3. Nearly Invariant to Face Orientations and Expressions Pan et al. (PAMI 2003)

Application: Face Recognition

Application: Remote Sensing Shaw and Burke (Lincoln Lab Journal 2003)

Application: Remote Sensing Shaw and Burke (Lincoln Lab Journal 2003)

Application: Remote Sensing Shaw and Burke (Lincoln Lab Journal 2003)

Application: Security and Defense Camouflage and Target Identification http: //www. resonon. com/

Conclusion �Hyperspectral Imaging Sees More than the Human Eye �Various Imaging Techniques Developed and Being Developed �Many Applications