Large Analysis of microvascular lesions in the brain

Large Analysis of microvascular lesions in the brain and retina using MRI and colour fundus imaging for early detection of CVD Prof. Ramamohanarao Kotagiri Department of Computing and Information Systems The University of Melbourne, Australia 3010 Email: kotagiri@unimelb. edu. au 1

Presentation Outline § Motivation § Brief Background § (Magnetic Resonance Imaging) MRI § Retinal Imaging § Brain microvascular lesions § White matter lesions (WMLs) § Brain infarcts § Retinal microvascular lesions § Arteriovenus nicking (AV nicking) § Focal arteriolar narrowing (FAN) § Objective § Proposed Method § Segmentation and quantification of WMLs § Detection and quantification of AV nicking § Detection and quantification of FAN § Summary 2

Vessel Calibre: A New Biomarker Retinal Circulation = ‘Window’ to Brain Circulation • Only place blood vessels can be viewed & monitored non-invasively • Damage to the retinal circulation may reflect impact of both recognised & unrecognised risk factors, and susceptibility • Indicator of structural vascular damage

Human Retina

Retinal Imaging • A retinal camera is used to capture image • A picture is taken showing the optic nerve (i. e. , disc), fovea, surrounding vessels, and the retinal layer A retinal camera (left) retinal fundus image (right)

Significance of Retinal Imaging Retinal vascular signs may predict Cardiovascular Diseases and Diabetes! • CVD (Cardiovascular Disease) (heart disease & stroke) is the most common cause of death in the developed world • 36% all deaths in Australia (2004) • Kills one Australian every 10 minutes. Condition Diabetes Australia of about 23 Million Population 1. 7 M (0. 9 M diagnosed) Pre-diabetes 1. 5 M Hypertension 3. 7 M Coronary Heart Disease (CHD) 26, 000 Deaths/Year 400, 000 Deaths in the USA in 2010 Stroke 12, 500 Deaths/Year All CVD 3. 7 M Cases 50, 000 Deaths/Year

Diagnostic Tools • Currently available prediction tools based on assessment of traditional risk factors (e. g. , blood pressure, cholesterol, smoking history, MRI after the symptoms of a stroke) • Account for only 50% of CVD cases Need for improved diagnostic tools

New Diagnostic Tools Traditional CVD Risk Factors e. g. blood pressure Unknown CVD Risk Factors e. g. genetic factors Sub clinical vascular damage CARDIOVASCULAR DISEASE AND DEATH Retinal Vascular Signs

Retinal Signs Predict CVD Retinal Observations Vascular Sign Retinal vascular calibre Associations Arteriolar narrowing 2. 6 fold increased risk of incident hypertension 2 fold increased risk of incident CHD Venular dilatation 2 fold increased risk of incident stroke 3 fold increased risk of incident CHD

COMPUTER-BASED RETINAL IMAGING PROGRAM FOR IDENTIFICATION OF CARDIOVASCULAR DISEASE RISK Patient’s retina photographed Photos transmitted to Retinal reading grading centre Retinal vascular scan report generated

Blood Vessels • Artery (Red) • Vein (Blue) Retinal Blood Vessel. Analysis

Arterial Vane (AV) Nicking AV nicking

Focal Arterial Narrowing (FAN)

Cheap Way to image Retina Using smartphone –lens adaptor cost$5. 00

Magnetic Resonance Imaging (MRI) § MRI is used for non invasively visualizing internal structures of the body. § Does not have any side effects of radiation. § Property of nuclear magnetic resonance (NMR) is used to image nuclei of atoms inside the body. § Provides higher details about soft-tissues 1 5

Magnetic Resonance Imaging (MRI) Different modalities are used to characterize and discriminate among tissues § T 1 weighted MRI is effective for visualizing various anatomical structures such as White matter, Gray matter and CSF very useful for identifying the location of tissues § T 2 weighted MRI is effective for visualizing pathologies such as lesions and tumors § Fluid attenuated inversion recovery (FLAIR) is effective in suppressing CSF (Cerebral Spinal Fluid) and enhancing lesions. 1 6

T 1 MRI Anatomical Structures are clearly identifiable. Gray Matter White Matter CSF (Cerebral Spinal Fluid) WMLs 1 7

Pathology (WMLs) looks more clearer. T 2 MRI WMLs 1 8

Intensity of CSF is suppressed Flair MRI WMLs 1 9

Motivation Relationship between WMLs and Risk of Stroke 1 [1] Vermeer, Sarah E. , et al. "Silent brain infarcts and white matter lesions increase stroke risk in the general population The Rotterdam Scan Study. " Stroke 34. 5 (2003): 1126 -1129. 20

Motivation Correlation between severity of Sub-Cortical WMLs and AV nicking 2 Sub-cortical WMLs Load • Q# is the severity scale of sub-cortical WMLs. • # Reference group; * p<0. 05; § p<0. 01. [2] Qiu, Chengxuan, et al. "Retinal and cerebral microvascular signs and diabetes the age, gene/environment susceptibility-reykjavi study. " Diabetes 57. 6 (2008): 1645 -1650. 21

Motivation Correlation between severity of Periventricular WMLs and AV nicking 2 Periventricular WMLs Load • T# is the severity scale of peri-ventricular WMLs. • # Reference group; * p<0. 05; § p<0. 01. [2] Qiu, Chengxuan, et al. "Retinal and cerebral microvascular signs and diabetes the age, gene/environment susceptibility-reykjavi study. " Diabetes 57. 6 (2008): 1645 -1650. 22

Motivation Correlation between severity of Sub-Cortical WMLs and FAN 2 Sub-cortical WMLs Load • Q# is the severity scale of sub-cortical WMLs. • # Reference group; * p<0. 05; § p<0. 01. [2 ] Qiu, Chengxuan, et al. "Retinal and cerebral microvascular signs and diabetes the age, gene/environment susceptibilityreykjavi study. " Diabetes 57. 6 (2008): 1645 -1650. 23

Motivation Correlation between severity of Sub-Cortical WMLs and FAN 2 Periventricular WMLs Load • T# is the severity scale peri-ventricular WMLs. • # Reference group; * p<0. 05; § p<0. 01. [2 ]Qiu, Chengxuan, et al. "Retinal and cerebral microvascular signs and diabetes the age, gene/environment susceptibilityreykjavi study. " Diabetes 57. 6 (2008): 1645 -1650. 24

Motivation Retinal microvascular lesions (AV nicking and FAN) can be an important bio-marker to predict the severity of WMLs load. § Current approach of quantification and correlation analysis is Manual § Limitations § § Highly subjective Expensive Time consuming High intra and inter-grader variability 25

Objective § Automatic segmentation and quantification of WMLs § Automatic detection and quantification of AV nicking § Automatic detection and quantification of FAN § Quantify the correlation between retinal and brain microvascular lesions § Develop a retinal image based brain microvascular lesions prediction model 26

Automated WMLs segmentation method T 1 MRI Flair MRI Pre-processing Post-processing using MRF Feature extraction Classification of WMLs 27

Automated WMLs segmentation method § Pre-processing for noise reduction and spatial normalization § Co-registration of T 1 and Flair MRI using Statistical parametric mapping (SPM 8)3 § Brain skull removal using Brain Extraction Tool (BET) 4 § Intensity normalization using dynamic maximum boundary 5 § Feature extraction § Multimodal MRI (T 1 and Flair) intensity § Tissue probability mask (PWM, PGM, PCSF) § Normalized spatial coordinate (X, Y, Z) § Global neighbourhood based contrast [3] J. Ashburner and K. J. Friston, “Unified segmentation, ” Neuroimage, vol. 26, no. 3, pp. 839– 851, 2005. [4] V. Popescu, M. Battaglini, W. Hoogstrate, S. Verfaillie, I. Sluimer, R. Van Schijndel, B. van Dijk, K. Cover, D. Knol, M. Jenkinson et al. , “Optimizing parameter choice for fsl-brain extraction tool (bet) on 3 d t 1 images in multiple sclerosis, ” Neuroimage, vol. 61, no. 4, pp. 1484– 1494, 2012. [5] Liang, Xi, Kotagiri Ramamohanarao et al. "Nat. ICT Australia (NICTA), Eveleigh, SA, Australia. " Digital Image Computing Techniques and Applications (DICTA), 2012 International Conference on. IEEE, 2012. 28

Proposed automated WMLs segmentation method Multimodal MRI (T 1 and Flair) intensity T 1 MRI Flair MRI 29

Automated WMLs segmentation method § Tissue probability mask § Multiple atlas construction from healthy T 1 MRI using FAST 6 § Non-linear registration of atlases with input T 1 MRI § Tissue mask construction based on multi atlas voting PWM PGM PCSF [6] Y. Zhang, M. Brady, and S. Smith, “Segmentation of brain mr images through a hidden markov random field model and the expectationmaximization algorithm, ” Medical Imaging, IEEE Transactions on, vol. 20, no. 1, pp. 45– 57, 2001. 30

Automated WMLs segmentation method • Normalized spatial coordinate • Subject patient’s T 1 MRI is linearly registered with Montreal Neurological Institute (MNI ) space • The voxel wise corresponding Montreal Neurological Institute (MNI) space is warped back to the experimental subject space. • This results in normalized X, Y and Z space comparable between patients. X Y Z 31

Automated WMLs segmentation method • Global neighbourhood based contrast • Create global neighbours for each pixel • Mask neighbours of candidate pixel using PWM. • A box filter of size m is applied on candidate and its neighbour pixels • Global neighbourhood based contrast (GNC) is computed using Here, X represent candidate pixel and Xn represents its neighbours. 32

Automated WMLs segmentation method Classification of Lesion using Random Forest 33

Automated WMLs segmentation method Post processing using Markov random field (MRF) Neighbourhood structure § § § represents lesion probability computed by RF classifier represents edge potential computed by Eq. 2. xi and yi represent ith pixel intensity and RF given class label. is the penalty constant. Loopy belief propagation (LBP) 6 is used to infer the outcome of MRF [6] K. P. Murphy, Y. Weiss, and M. I. Jordan, “Loopy belief propagation for approximate inference: An empirical study, ” in Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc. , 1999, pp. 467– 475. 34

Automated WMLs segmentation method ØSegmentation outcome after applying MRF Initial Segmentation MRF given Segmentation Ground Truth Segmentation 35

Automated WMLs segmentation method § Classification Parameters § Classifier : Random Forest § Number of trees in RF : 200 § Data § Number of Subject : 24 § Source of Data: ENVISion study 7 § Size : 256 × 36 § Resolution : 0. 94 × 4 mm 3 § Training and Testing procedures § 4 Fold cross-validation (4 times) § 2 fold for training § 1 fold for parameter selection § 1 Fold for testing • [7] C. M. Reid, E. Storey, T. Y. Wong, R. Woods, A. Tonkin, J. J. Wang, A. Kam, A. Janke, R. Essex, W. P. Abhayaratna et al. , “Aspirin for the prevention of cognitive decline in the elderly: rationale and design of a neuro-vascular imaging study (envis-ion), ” BMC 36 neurology, vol. 12, no. 1, p. 3, 2012.

Training of Random Forest Classifier • Number of subjects : • Resolution of each Subject • Number of data points (voxels) after skull stripping • Number of features for each data point: 26 • The number of training sample is very large = 810, 00*12 = 9. 7 million voxels • To reduce the training time we randomly sample 20, 000 voxels to build a random forest tree and build about 200 random forest trees. This method provides adequate accuracy!

Features Description Local Features §T 1 , FLAIR intensity §Tissue probability map of White matter, Gray matter and Cerebral spinal fluid §Normalized spatial coordinates Global neighbourhood based contrast feature (GNCF) Intensity of the lesion is enhanced and normalized based on the contrast of global neighbourhood points Histogram based features §Mean §Variance §Skewness §Kurtosis Gray level co-occurrence matrix (GLCM) [1] §Contrast §Correlation §Energy §Homogeneity Run length matrix (RLM) [2] §Short run emphasis inverse moments §Long run emphasis moments §Gray level non uniformity §Run length non uniformity Isotropic undecimated wavelet transform (IUWT) [3] §Wavelet coefficients for five different scales [1] A. Baraldi and F. Parmiggiani, “An investigation of the textural characteristics associated with gray level cooccurrence matrix statistical parameters, ” Geoscience and Remote Sensing, IEEE Transactions on, vol. 33, no. 2, pp. 293– 304, 1995. [2] M. M. Galloway, “Texture analysis using gray level run lengths, ” Computer graphics and image processing, vol. 4, no. 2, pp. 172– 179, 1975. [3] P. Bankhead, C. N. Scholfield, J. G. Mc. Geown, and T. M. Curtis, “Fast retinal vessel detection and measurement using wavelets and edge location refinement, ” Plo. S one, vol. 7, no. 3, p. e 32435, 2012.

Automated WMLs segmentation method § Evaluation metrics § Sensitivity (SEN) § Positive predictive value (PPV) § Dice Similarity Index (SI) 39

Automated WMLs segmentation method § WMLs Load Category § High Lesion Load (HLL [>10 ml]) § Medium Lesion Load (MLL [between 6 ml to 10 ml]) § Low Lesion Load (LLL [between 1 ml to 5 ml ]) 40

Proposed automated WMLs segmentation method Results 41

Proposed automated WMLs segmentation method Comparison with state of the art methods 8 9 For both methods p-value< 0. 05. [8] M. D. Steenwijk, P. J. Pouwels, M. Daams, J. W. van Dalen, M. W. Caan, E. Richard, F. Barkhof, and H. Vrenken, “Accurate white matter lesion segmentation by k nearest neighbor classification with tissue type priors (knn-ttps), ” Neuro. Image: Clinical, vol. 3, pp. 462– 469, 2013. [9] P. Schmidt, C. Gaser, M. Arsic, D. Buck, A. F¨orschler, A. Berthele, M. Hoshi, R. Ilg, V. J. Schmid, C. Zimmer et al. , “An automated tool for detection of flair-hyperintense white-matter lesions in multiple sclerosis, ” Neuroimage, vol. 59, no. 4, pp. 3774– 3783, 2012. 42

Automatic quantification of AVN and FAN Retinal Image Pre-processing Feature extraction Quantification of AVN and FAN 43

Automatic quantification of AVN and FAN § Pre-processing § Vessel Segmentation § Vessel Cross-over point detection § Candidate vessel region selection § Vessel width measurement 44

First to Analyze the Vessel: Segmentation Although many methods have been proposed, significant improvement is still a necessity due to the limitations in state of theart methods, which include: § Poor segmentation in the presence of central reflex (i. e. , bright strip along the centre of a vessel). § Poor segmentation at bifurcation and crossover points. § The merging of close vessels. § The missing of small vessels. § False vessel detection at the optic disk and pathological regions. 45

Some Limitations of Existing Methods (a) a portion of a retinal image showing the presence of central reflex vessels (white solid arrows), close vessels (white dashed arrows), artery-vein crossing regions (black solid arrows), and small vessels (black dashed arrows) and segmentations obtained by (b) Staal et al. method; (c) Soares et al. method; (d) Ricci-line method ; (e) Ricci-svm method (f) Our proposed method. Uyen et. all (2013), Pattern Recognition 46

Our Proposed Method § A linear combination of line detectors at different scales to produce the vessel segmentation for each retinal image. § A basic line detector uses a set of approximated rotated straight lines to detect the vessels at different angles. § The difference between the average gray level of the winning line (the line with maximum average gray level) and the average gray level of the surrounding window provides a measure of ‘vesselness’ of each image pixel. 47

Vessel Segmentation Accuracy Segmentation results on some selective regions showing the improvements of the proposed method over existing methods in terms of segmentation quality: (a) original image; segmentations of (b) Staal et al. method; (c) Soares et al. method; (d) Ricci-line method; and (e) 48 proposed method.

Vessel Segmentation Accuracy 49

Vessel Segmentation Accuracy (Cont. ) 50

Vessel Crossover Point Detection - Artery-vein crossover points in a retinal image are the locations where retinal artery and vein cross each other - Necessary for Individual Vessel segment Identification Crossover points (left) often complicated to detect due to intensity variation (right) 51

Our Proposed Crossover Point Detection Framework for Crossover Point Detection 52

Crossover Point Detection Analyzing branch and crossover points to detect crossover points 53

Crossover Point Detection A retinal image with all crossover points detected by the proposed method marked as white spots. 54

Accuracy on Crossover Point Detection 55

Vessel Width Measurement § To Find mean Arterial and Venular Width in the Image and Find Association with CVD or Hypertension § Analyze the Vessel Width with Tortuosity or other Features 56

Vessel Width Measurement Method for identifying vessel edge points representing vessel width at a centre point. 57

Width Measurement Accuracy 58

Feature for AVN severity grading § Multi-scale measurement of narrowing Fn near the cross over point is used to classify AVN. where, c is the middle point of the vessel centreline and Wi represent the vein widths. § In this study, we have used 3 scales for Fn with n = 10, 20, 30. 59

Accuracy AVN severity classification Severity Level Prediction Accuracy Proposed Method Uyen et al. 0 78. 43 % 70. 58 % 1 29. 41 % 17. 64 % 2 42. 85 % 14. 28 % 3 81. 81 % 45. 45 % Uyen T. V. Nguyen, Alauddin Bhuiyan, Laurence A. F. Park, Ryo Kawasaki, Tien Y Wong, Jie Jin Wang, Paul Mitchell and Kotagiri Ramamohanarao, An Automated Method for Retinal Arteriovenous Nicking Quantification From Color Fundus Images, IEEE Transactions on Biomedical Engineering, vol. 60, number. 11, pp. 3194– 3203, 2013. 60

Feature for FAN severity grading Lmin and Lmax represents the set of local minima and local maxima for the vessel widths D , For each , the closest right and left local maxima and is computed using is used as a feature to classify FAN 61

Accuracy FAN severity classification Severity Level 0 Prediction Accuracy 87% 1 78% 62

Summary § § § A novel WMLs segmentation method. A novel AV nicking quantification method A novel FAN quantification method In future: Proposed method will be applied on a large dataset to quantify the correlation between brain and retinal micro-vascular lesions. § Quantified correlation will be used to develop a retinal image based brain microvascular lesion severity prediction model. 63

Thank you! 64