Spectral Imaging of the Retina Andy R Harvey

Spectral Imaging of the Retina Andy R. Harvey, Ied Abboud, Alistair Gorman, Andy I. Mc. Naught* School of Engineering & Physical Sciences, Heriot Watt University, Edinburgh, UK *Eye Unit, Cheltenham General Hospital, Cheltenham, UK PIBM Sept 05 Andy Harvey 1

Outline • What is spectral imaging? • Spectral retinal imaging • Why? • Spectral time-sequential spectral imaging • For flexibility and research • 2 D snapshot • For real-time, high throughput screening • Conclusions PIBM Sept 05 Andy Harvey 2

Conventional Spectral Imaging Dysplastic cell Superficial squamous cell Lymphocyte Intermediate cell PMN Spectrally classified image RGB Image Classification spectra Courtesy CRI PIBM Sept 05 Andy Harvey 3

Imaging the eye Sclera Lens Choroid Iris Cornea Macula Light Anterior chamber (full of aqueous humour) Posterior chamber PIBM Sept 05 Retina Optic disc nerve Vitreous humour Andy Harvey 4

The Role of Spectral Retinal Imaging • • By 2020 there will be 200 million visuallyimpaired people world wide • Glaucoma, diabetic retinopathy, ARMD • 80% of those cases are preventable or treatable • Screening and early detection are crucial Can spectral imaging offer enhancements to current screening techniques ? Spectral imaging is non-invasive and safe • cf. fluorescein angiogram Spectral imaging can enable imaging of • Retinal biochemistry • Blood oximetry • Diabetic retinopathy, glaucoma • Lipofuscin etc • Age-related macula degeneration PIBM Sept 05 Andy Harvey 5

Spectral Imaging: Traditional approaches Time-sequential spectral multiplex Time-sequential spatial multiplex Nx Ny Limitations • Optically inefficient • 2 D time-varying scenes • 2 D snapshot • Nl Nl(t) • Nx Ny(t) required for: • Retinal imaging • in vitro, in vivo imaging And Fourier-transform equivalents PIBM Sept 05 Andy Harvey 6

Spectral Fundus Camera • Source filtering by LCTF incorporated into COTS fundus camera • 10 nm spectral width • 20 msec random access • Images captured using a cooled, low-noise CCD camera PIBM Sept 05 Andy Harvey 7

Coregistered Spectral Images of a Healthy Retina Isobestic point Images translationally and rotationally coregistered PIBM Sept 05 Andy Harvey 8

Spectral angle map of healthy and diabetic retina Normal Retina Diabetic Retina • Shading indicates similarity of each pixel spectrum with artery and vein spectra • Qualitative oxymetry PIBM Sept 05 Andy Harvey 9

Supervised spectral classifiaction • • • Implicit calibration based on spectral signatures within the eye • Classification possible without absolute calibration Clear distinction between veins/arteries, on/off optic disc • Spectra depends on local environment Inversion of data to calculate biochemical concentrations (eg oxygenation) requires a model of light propagation and scattering in the retina to remove environmental effects • Monte Carlo, Kubelka Monk, Transfer equation PIBM Sept 05 Andy Harvey 10

Requirements for a snapshot technique: retinal imaging • Improved calibration • Patient patience • Remove imperfect coregistration • • PC 15 due to Variations in imaging distortion between images Similar issues with other in vivo applications • Imaging internal epithelial cancers • PIBM Sept 05 Eg gastrointestinal Andy Harvey 11

Image Replication Imaging Spectrometer: IRIS Snapshot image • zero temporal misregistration • ‘ 100%’ optical efficiency • World’s only snapshot, 2 D spectral imager (almost !) • Conceptually related to Lyot filter PIBM Sept 05 Andy Harvey Spectral Demultiplexo r Large format detector 12

Lyot filter: principle of operation Waveplate Polariser PIBM Sept 05 Andy Harvey 13

IRIS snapshot spectral imager: principle of operation • Wollaston prism polarisers replicate images • Each Wollaston prism-waveplate pair provides both cos 2 and sin 2 responses • All possible products of spectral responses are formed at detector PIBM Sept 05 Andy Harvey 14

Spectral transmission Wollaston/waveplate assembly 2 (2 D)co n p cos 2 (2 pn. D) n i s ) ) os 2 (pn. D 2( D n sp co 2 sin 2 pn. D PIBM Sept 05 s pn. D) c 2 (2 s D)co 2 (pn 2 (2 pn. D) n i s ) D sin 2 (pn cos 2(pn. D)cos 2(2 pn. D)cos 2(4 pn. D) cos 2(pn. D)sin 2(2 pn. D)cos 2(4 pn. D) cos 2(pn. D)cos 2(2 pn. D)sin 2(4 pn. D) cos 2(pn. D)sin 2(4 pn. D) sin 2(pn. D)cos 2(2 pn. D)cos 2(4 pn. D) sin 2(pn. D)sin 2(2 pn. D)cos 2(4 pn. D) sin 2(pn. D)cos 2(2 pn. D)sin 2(4 pn. D) sin 2(pn. D)sin 2(4 pn. D) Andy Harvey 15

Spectral responses • Bands are overlapping bell shapes • Choose cost function to minimise sidelobes • Small (~5%) reduction in spectral separation • Cut-off filters used to define spectral range • 32 channel, visible-band system • 8 channel visible-band system • 520 nm 720 nm • 520 nm 820 m • 5 Quartz retarders • 3 Quartz retarders PIBM Sept 05 Andy Harvey 16

Optical scaling laws Polariser, retarders & Wollaston prisms Field stop Primary lens Collimating lens Bandpass (index matched) filter Imaging lens Camera Hamamatsu ORCA-ER Inputs: Outputs: Fo. V Field stop size Sub image size on CCD Collimating lens rear element diameter CCD pixel size Splitting angles, apertures & depths of prisms Primary lens magnification & F# Apertures of retarders, polarisers and filters Collimating lens back focal distance, focal length & front element diameter Imaging lens focal length & front element diameter Prism birefringence PIBM Sept 05 Andy Harvey 17

Components & Assembly • • 8 channel system 520 nm to 820 nm 3 Quartz retarders 3 Calcite Wollaston prisms PIBM Sept 05 Andy Harvey 18

Spectral Retinal Imaging • Difficult imaging conditions render application of traditional HSI techniques problematic • IRIS enables real-time and snapshot spectral imaging Canon PIBM Sept 05 CR 4 -45 NM Andy Harvey 19

Blood oximetry 80 40 20 • Optimal spectral band for retinal oximetry • Vessel thickness ~ optical depth • 570 -615 nm • Eight bands approximately equally spaced PIBM Sept 05 Andy Harvey 20

Video sequence recorded with bandpass filtered inspection lamp PIBM Sept 05 Andy Harvey 21

Retinal image recorded with flash illumination PIBM Sept 05 Andy Harvey 22

Coregistered and PCA images 607 595 574 581 592 613 585 603 PIBM Sept 05 PC 1 PC 2& PC 2 PC 1 Andy Harvey 23

Summary • • Spectral imaging of the retina shows promise for non-invasive detection of retinal disease • Clinical trials on-going LCTF-based, time-sequential spectral filtering enables rapid and flexible 2 D spectral retinal imaging • Flexible data acquisition • Pulse and other motion artefacts limit accuracy Snapshot spectral imaging in 2 D (IRIS) promises high-performance real-time multi-spectral imaging • Ideal for in vivo imaging • No temporal misregistration Absolute, quantitative data requires a model of light interaction within the retina PIBM Sept 05 Andy Harvey 24

• • • IRIS snapshot spectral imager Wollaston prism polarisers replicate images Each Wollaston prism-waveplate pair provides both cos 2 and sin 2 responses All possible products of spectral responses are formed at detector PIBM Sept 05 Andy Harvey 25

Measured & predicted spectral responses PIBM Sept 05 Andy Harvey 26

Absolute total transmission Absolute response curves in polarised light • Bandpass filter & polariser dominate losses • Improved system: T>80% • Theoretical throughput is 2 n times higher than for spatial/spectral multiplexed techniques! Response (%) 50 25 0 PIBM Sept 05 Andy Harvey 27

Application to microscopy: Imaging of multiple fluorophors Response (%) 5 0 2 5 • • 0 PIBM Sept 05 IRIS fitted to conventional epi-fluorescence microscope Germinating spores of Neurospora crassa stained with • GFP – nucleii fluoresce at 510 nm • FM 4 -64 – membranes fluoresce at >580 nm Andy Harvey 28

Principle component decomposition PC 1 PC 3 • Artery structure is a pulse artefact • Very difficult to coregister by image processing means PC 15 • PIBM Sept 05 Andy Harvey Snapshot technique desirable 29

Conclusions • IRIS is a new spectral imaging technique that enables snapshot spectral imaging in 2 D • No rejection of light • No data inversion • Highest-possible signal-to-noise ratios • Simple logistics • Inherently compact and robust • Simply fitted to conventional imaging systems • Birefringent materials exist for applications from 0. 2 m to 12 m • Applications • In vivo, in vitro imaging • Retinal imaging • Microscopy • Multiple fluorophors • Quantum dots • Surveillance • Remote sensing • Etc. PIBM Sept 05 Andy Harvey 30

Spectral Characteristics of the Retina • Optical depth of Hb & Hb. O 2 dominates variation of penetration with l • Tissues vary between highly turbid and transparent • Blue light images retinal surface • Light at ~600 nm enables spectral oximetry within retinal blood vessels • optical depth of Hb. O 2 > vessel thickness so vessels translucent • optical depth of Hb < vessel thickness so vessels are opaque • Light > 640 nm penetrates to coroid Red Green Blue Isobestic point PIBM Sept 05 Andy Harvey 31

Issues for Spectral Retinal Imaging • Spectral imaging of static scenes is relatively ‘easy’ • Spectral imaging of the retina encounters • Imaging through an erratically moving, low-quality f/6 eye-lens system • Calibration • Components of interest within a complex turbid medium • Patient tolerance • Using current technology, time-sequential spectral bandpass offers • Optimal SNR • Reduced light intensity at the retina • Agile selection of spectral bands (data efficient) ± 100 pixels • Issues ± 2º Solutions: 2 D snapshot • Coregistration spectral imaging • Calibration PIBM Sept 05 Andy Harvey 32

The End PIBM Sept 05 Andy Harvey 33

Snapshot/fully staring Temporally scanned Direct Imaging Spectrometry (Fourier) Transform Imaging Spectrometry Scanning mirror Ns Fixed mirror ND(t) Nl(t) PIBM Sept 05 Nx Ny FT Nx Nx Ny Ny Detector array Ns Nl ND FT Nx Ny(t) Nx Andy Harvey Ny(t) Nx Ny(t) 1 D image x path difference 34 D

Why another spectral imaging technique? • Traditional approaches • Time sequential spectral multiplex • Monochromatic two-dimensional image in snapshot • Time sequential spatial multiplex • One-dimensional spectral image in a snapshot • (and Fourier-transform equivalents) • Problems • Cannot record two-dimensional spectral images of time-varying scenes • Optically inefficient • Time-resolved (snapshot) spectral imaging is required for • Dynamic scenes • In vitro, in vivo imaging and microsocopy • Combustion dynamics, surveillance… • Irregular motion between scene and imager • In vivo imaging • Ophthalmology • Remote sensing, airborne surveillance, industrial inspection… PIBM Sept 05 Andy Harvey 35