Spectral Imaging at Heriot Watt University Dr Andy

Spectral Imaging at Heriot Watt University Dr Andy R Harvey School of Engineering and Physical Sciences Heriot Watt University Edinurgh, EH 14 4 AS Tel +0131 451 3356 a. r. harvey@hw. ac. uk 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 1

Some Heriot Watt spectral imaging solutions • Birefringent 2 D Fourier-transform imaging spectrometer (FTIS) • Snapshot 2 D foveal imaging spectrometer (OFIS) • Snapshot 2 D imaging spectrometer (IRIS) 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 2

Birefringent Fourier Transform Imaging Spectrometer Scanning mirror Fixed mirror Detector array • Conventional FTIS offers • High SNR in low flux • MWIR, twilight • Very high spectral resolution • Wide spectral range • But conventional time-sequential interferometry in real-world applications is highly problematic 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 3

Birefringent FTIS • Mechanical sensitivity of conventional FTIS makes real-world applications almost impossible • Introduce temporal path difference with scanning Wollaston prisms • Inherently vibration insensitive since path difference due by birefringence within a single crystal and common path • Optical gearing reduces required accuracy of movement by a factor ~200 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 4

25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 5

Movie of spectral image cube Colour image 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 6

Foveal hyperspectral imaging in 2 D Optical Fibre-coupled Imaging Spectrometer • Real-time hyperspectral imaging in 2 D would require excessive information throughput • GVoxel/sec • Bottlenecks include • detector – 20 MVoxel/sec • Computer processing • Biological systems with this problem employ a scanning fovea…. 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 7

Foveal hyperspectral imager: OFIS Schematic 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 8

OFIS: Hardware & raw data First fibre Spatial extent Last fibre 400 nm Wavelength 700 nm • Raw image at CCD prior to reformatting The hyperspectral fovea assembly: • Custom fibre optic image refromatter • 1 D dispersive hyperspectral imager • CCD camera 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 9

OFIS: Movie demonstrating real-time spectral ID with simple recognition • Colour image 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 10

Snapshot spectral imaging in 2 D Image Replication Imaging Spectrometer 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 11

Image Replication Imaging Spectrometer: IRIS • • • Single image multiplexed onto 2 N passband images ‘ 100%’ optical efficiency Snapshot image • no temporal misregistration Trade spectral resolution for Fo. V • Low resolution, wide Fo. V • High resolution, small Fo. V • Gas detection • High spectral resolution • Few Bands • Modest Fo. V Conceptually related to Lyot filter World’s only snapshot, 2 D spectral imager (almost !) 25 -May-05 HSWG Spectral Demultiplexo r Andy Harvey: a. r. harvey@hw. ac. uk Large format detector 12

• • • IRIS snapshot spectral imager: Wollaston prism polarisers replicate images Each Wollaston prism-waveplate pair provides both cos 2 and sin 2 responses All possible products of spectral responses are formed at detector 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 13

Components & Assembly • 8 channel system • 3 Quartz retarders • 3 Calcite Wollaston prisms 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 14

Absolute total transmission Absolute response curves in polarised light • Bandpass filter & polariser dominate losses • Improved system: T>80% • Theoretical throughput is 2 n times higher than for other techniques! Demonstrated 96% transmission for IRIS-only components Response (%) 50 • 25 0 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 15

An example medical application: Blood oxymetry in the retina 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 16

Requirements for a snapshot technique: retinal imaging • Improved calibration • Patient patience • Remove misregistration artefacts; imperfect coregistration arises due to • • Distortion of eye ball with pulse • Variations in imaging distortion between images PC 15 Similar issues with other in vivo applications • 25 -May-05 Imaging epithelial cancers HSWG Andy Harvey: a. r. harvey@hw. ac. uk 17

Blood oximetry 80 40 • Optimal spectral band for retinal oximetry • Vessel thickness ~ optical depth • 570 -615 nm • Eight bands approximately equally spaced 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 18

Spectral Retinal Imaging • Difficult imaging conditions render application of traditional HSI techniques problematic • IRIS enables real-time and snapshot spectral imaging Canon 25 -May-05 CR 4 -45 NM HSWG Andy Harvey: a. r. harvey@hw. ac. uk 19

Video sequence recorded with low-power, CW tungsten illumination 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 20

Retinal image recorded with flash illumination 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 21

Coregistered and PCA images 607 595 574 581 592 613 585 603 25 -May-05 PC 1 PC 2& PC 2 PC 1 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 22

Application to microscopy: Imaging of multiple fluorophors Response (%) 5 0 2 5 • • 0 25 -May-05 HSWG IRIS fitted to conventional epi-fluorescence microscope Germinating spores of Neurospora crassa stained with • GFP – nucleii fluoresce at 510 nm • FM 4 -64 – membranes fluoresce at >580 nm Andy Harvey: a. r. harvey@hw. ac. uk 23

This document gives only a general description of the product(s) or services and except where expressly provided otherwise shall not form part of any contract. MWIR IRIS Consists of: n n n COTS Phoenix MWIR Camera Specac Polariser IRIS II Optical Telescope May 2005 Hyperspectral Working Group 24

Conclusions • The transfer of spectral imaging from scientific to military and laboratory applications must address the needs of high SNR, accurate coregistration and logistics. • No single technique can satisfy all requirements simultaneously • ‘Horses for courses’ • New techniques such as described here illustrate how these requirements can be satisfied • Similar issues occur in both military and civilian (eg medical) applications introducing significant scope for dual use. 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 25

Additional information • Linked by previous slide buttons 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 26

The co-registration problem • • • Co-registration required for time sequential direct and FT imaging • Not for snapshot/fully-staring Misregistration of spectral images distorts spectral basis sets Video spectrum frame rates insufficient to freeze motion from most aerial platforms Target 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 27

The magnitude of the co-registration problem • • 25 -May-05 HSWG Co-registration should be better than 1/20 1/50 of a pixel Deployment of time sequential DIS and FTIS will normally require ‘step and track’ Andy Harvey: a. r. harvey@hw. ac. uk 28

Bandpass functions • Bandpass are overlapping bell shapes • Can be optimised by adjusting waveplate thickness and dispersion 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 29

Spectral discrimination Contiguous ‘top -hat’ • Bell-shaped IRIS transmission functions tend to smooth spectra • Typically 6% reduction in separation in 8 D spectral space • 8 x improvement in SNR 25 -May-05 HSWG IRIS Andy Harvey: a. r. harvey@hw. ac. uk 30

Imaging Spectrometry Summary and novel(Fourier) HWUTransform techniques in red Snapshot/fully staring Temporally scanned Direct Imaging Spectrometry • Mature • The traditional technique for 2 D static spectral imaging • Low MPLX efficiency Nl(t) • • Very high spectral resolution Scanning Highest mirror SNR in low-light conditions The optimum technique Ns for MWIR Unsuitable for poorly. Fixed controlled mirror environments. . . ND(t) • FTIS FT Ny Nx Nx • No temporal coregistration problem • The traditional Ntechnique for 1 D l ND remote sensing • 2 D very immature…. Ny(t) • IRIS N x • OFIS 25 -May-05 HSWG Nx Ny Ny Detector array Ns FT Nx Ny(t) Andy Harvey: a. r. harvey@hw. ac. uk Nx Ny(t) 1 D image x path difference 31 D

Ratio of SNRs in 3 -5 mm band -temporal scan 40 Hz, 10 bands Zero range 25 -May-05 1 Hz, 10 bands 1500 m nadir path HSWG Andy Harvey: a. r. harvey@hw. ac. uk 32

Ratio of SNRs in 8 -14 mm band - temporal scan Zero range 25 -May-05 HSWG 40 Hz, 10 bands 1500 m nadir path Andy Harvey: a. r. harvey@hw. ac. uk 33

IRIS: FTIS SNR 40 Hz, 10 bands Zero 1500 m nadir range path 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 34

Lyot filter: principle of operation Waveplate Polariser 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 35

Optical scaling laws Polariser, retarders & Wollaston prisms Field stop Primary lens Collimating lens Bandpass (index matched) filter Imaging lens Camera Hamamatsu ORCA-ER Inputs: Outputs: Fo. V Field stop size Sub image size on CCD Collimating lens rear element diameter CCD pixel size Splitting angles, apertures & depths of prisms Primary lens magnification & F# Apertures of retarders, polarisers and filters Collimating lens back focal distance, focal length & front element diameter Imaging lens focal length & front element diameter Prism birefringence 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 36

Spectral retinal Imaging • By 2020 there will be 200 million visuallyimpaired people world wide • Glaucoma, diabetic retinopathy, ARMD • 80% of those cases are preventable or treatable • Screening and early detection are crucial • Spectral imaging provides a non-invasive route to monitoring retinal biochemistry • Blood oximetry, lipofuscin accumulation Normal Diabetic Retina 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 37

Measured & predicted spectral responses 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 38

Imaging Concepts Group • Research Group • Head • Dr Andy Harvey • PDRA • Dr Colin Fraser • Dr Eirini Theofanidou • Bertrand Lucotte • Ph. D Students • Alistair Goreman • Asloob Mudassar • Gonzalo Muyo • Sonny Ramachandran • Ied Abboud • Beatrice Graffula • External Ph. D students • Ruth Montgomery (NPL) • Robert Stead (Thales) 25 -May-05 HSWG • Funders/Collaborators • Astra. Zeneca • AWE • BAE Systems • DSTL • EPSRC • NATO • NPL • Qineti. Q • Royal Society • Scottish Enterprise • South Glos. NHST • SAAB • Thales Andy Harvey: a. r. harvey@hw. ac. uk 39

Research areas • Imaging Concepts Group • Spectral imaging • Retinal imaging • Wavefront coding • Aperture synthesis imaging (optical and mm-wave) • Optical encryption for communications • mm-wave imaging • Biophotonics • Insect flight dynamics 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 40

Overview • Introduction to spectral imaging • Spectral imaging techniques at Heriot-Watt University • FTIS • Inherently robust FT imaging spectrometer • IRIS • Snapshot, ‘ 100%’ optical throughput imaging spectrometer • OFIS • Foveal hyperspectral imaging spectrometer • An example application • Spectral imaging of the retina • Conclusions 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 41

What are the issues • High SNR required • >100 • No spatial or spectral multiplexing desirable • Fourier-transform • in some conditions • Accurate coregistration required (<1/20 pixel) • Snapshot spectral imaging preferred • Spectral resolving power matched to requirement • 100 s for data acquisition • ~10 for many applications • As few as two if clutter allows (eg spectral lines) • Detector is ‘information bottleneck’ • 20 MVoxel/second per tap 25 -May-05 HSWG Andy Harvey: a. r. harvey@hw. ac. uk 42