Fuzzy logic in medical applications and image processing

Fuzzy logic in medical applications and image processing: main approaches, do's and don't's Santiago Aja-Fernández Medical Imaging Using Bio-Inspired and Soft Computing sanaja@tel. uva. es MIBISOC Technical Course 2012

Fuzzy Logic in Medical Applications and Image Proc. Motivation: • • Many articles and books about fuzzy systems in medicine and image processing. Are all the problems suitable for “fuzzification” or fuzzy approach? When is it justified to use soft computing in medical problems? FACT: Some years ago “fuzzy” was cool. And now? 2

Fuzzy Logic in Medical Applications and Image Proc. Some applications in the field: • Fuzzy logic system for ECG interpretation • EEG analysis for assessment of depth anaesthesia • Fuzzy image processing for enhancement of megavoltage images in radiation therapy. • Fuzzy clustering methods for the segmentation of multimodal medical images • Fuzzy clustering for parametric map construction in myocardial perfusion MRI • Fuzzy quantification of artery lesions in renal arteriographies • Fuzzy reasoning and peacemaker control • Self-learning FL control of anaesthetic intravenous infusions • Fuzzy diagnosis by score-based tests [Szczepaniak et al. (2000)], [Kerre and Nachtegael (2000)], [Teodorescu et al. (1998)], [ Nachtegael et al. (2003)] 3

Fuzzy Logic in Medical Applications and Image Proc. Outline 1. Medical Imaging 2. Fuzzy Logic Systems 3. Practical Considerations – When and when not – How to do it – 10 practical questions 4. Case of Study: MRI filtering 5. Some practical examples 1. Fuzzy filtering of ultrasound 2. Bone age assessment 3. Fuzzy Thresholding 4

Medical Imaging

Fuzzy Logic in Medical Applications and Image Proc. Medical Imaging • • Many different techniques and applications under “Medical Imaging Processing” Techniques: MRI, ultrasound, CT, radiograph (very different features) Applications: diagnosis, chirurgical guidance, research, follow-up… References: [Suetens (2009)], [Sonka and Fitzpatrick (2000)] 6

Fuzzy Logic in Medical Applications and Image Proc. The Beginning: X-ray Wurzburg, Nov 8, 1895 Wilhelm Röntgen discovered X-rays. 7

Fuzzy Logic in Medical Applications and Image Proc. The Beginning: X-ray • • Röntgen refused commercial contracts or patent process Submitted paper to publication = 1 week Lab demonstration to clinical application = 1 year Nobel prize for physics 1901 8

Scanner X-space (complex) K-space F-1 magnitude |. |

Fuzzy Logic in Medical Applications and Image Proc. 12

13

14

Fuzzy Logic in Medical Applications and Image Proc. 15

v 3 l 1 v 1 l 3 l 2 v 2 16

Fuzzy Logic in Medical Applications and Image Proc. Signal conditioning 18

Fuzzy Logic in Medical Applications and Image Proc. Registration CT (x-ray) MRI (T 1) MRI (T 2) 22

MR- slice MR- Atlas

Fuzzy Logic in Medical Applications and Image Proc. Segmentation: 24

Fuzzy Logic in Medical Applications and Image Proc. Chirurgical Planning 27

Fuzzy Logic in Medical Applications and Image Proc. Intervention: 29

Fuzzy Logic Systems

Fuzzy Logic in Medical Applications and Image Proc. Fuzzy Sets • Basis of Fuzzy Logic Systems (FLS) are the fuzzy sets. Crisp Sets Vs. Fuzzy Sets: Degrees of membership • 1 1 0 0 1 2 3 4 5 6. . . Young Middle Age Old 32

Fuzzy Logic in Medical Applications and Image Proc. Fuzzy Sets • Traditionally used in control systems Low Normal High Very High 1 0 32 34 36 38 40 42 Temperature (ºC) 33

Fuzzy Logic in Medical Applications and Image Proc. Membership vs. Probability [Bezdek (1993)], [Mendel (1995)] • • Relationship of fuzziness to probability. Are fuzzy sets just a clever disguise for statistical models? 34

Fuzzy Logic in Medical Applications and Image Proc. Poison bottles Probability of being potable p=0. 9 p=0. 2 Degree of membership to “potable” m=0. 9 m=0. 2 35

Fuzzy Logic in Medical Applications and Image Proc. Membership vs. probability • • Confronted with this pair of bottles and given that you must drink from the one that you choose, which would you choose to drink from first? Why? After an observation is made regarding the content of both bottles what are the (possible) values for membership and probability? 36

Fuzzy Logic in Medical Applications and Image Proc. Membership vs. probability ? Known 37

Fuzzy Logic in Medical Applications and Image Proc. Fuzzy Logic System • • Mendel: “What is a Fuzzy Logic System? In general, a FLS is a nonlinear mapping of an input data (feature) vector into a scalar output (the vector output case decomposes into a collection of independent multi-input/single-output systems). The richness of FL is that there are enormous numbers of possibilities that lead to lots of different mappings. This richness does require a careful understanding of FL and the elements that comprise a FLS. ” Wikipedia: The reasoning in fuzzy logic is similar to human reasoning. It allows for approximate values and inferences as well as incomplete or ambiguous data (fuzzy data) as opposed to only relying on crisp data (binary yes/no choices). Fuzzy logic is able to process incomplete data and provide approximate solutions to problems other methods find difficult to solve. Terminology used in fuzzy logic not used in other methods are: very high, increasing, somewhat decreased, reasonable and very low. 38

Fuzzy Logic in Medical Applications and Image Proc. Fuzzy Logic System • • Fuzzy Logic Systems Are Universal Approximators (proved). FLSs deal well with Vagueness Fuzzy Logic System BASE OF KNOWLEDGE Crisp Output INPUT FUZZIFIER DEFUZZIFIER (Crisp) Fuzzy Output Fuzzy Input FUZZY INFERENCE ENGINE 39

Fuzzy Logic in Medical Applications and Image Proc. Membership functions a b c d Operations 40

Practical Considerations

Fuzzy Logic in Medical Applications and Image Proc. Practical considerations • • When can we use FLS in medical applications? How? 42

Fuzzy Logic in Medical Applications and Image Proc. When: • • • When input data are approximate values, incomplete and/or ambiguous data. When dealing with ``linguistic terms". When re-using data with lack of resolution. When there is a great lack of knowledge of the underlying model. Attempting to simplify a problem. 43

Fuzzy Logic in Medical Applications and Image Proc. When not: • • • "Fuzzifing" a problem without a reason. When the use of FL goes with a loss of resolution or data. When I have not properly studied the real model for the data. When it adds extreme constraints to the algorithm. When the only novelty is the fuzzification 44

Fuzzy Logic in Medical Applications and Image Proc. How 1. 2. 3. 4. 5. Define the system objectives and criteria: What am I trying to do? Which is the ultimate purpose of the system? Are there any other solutions to these problem? What does the FL implementation improve vs other approaches? The FL approach is used for the whole system or only in a part? Select the inputs and outputs of the system. Determine the input and output relationships and choose a minimum number of variables for input to the FL engine. Create the FL system: Database, rules, membership functions. . . If necessary: train the system 45

Fuzzy Logic in Medical Applications and Image Proc. How (2) Some important issues from the Pattern Recognition (PR) Field: [Jain (2000)], [Jain (1997)] • If the system implements a PR system, pay attention to all the process: 1. 2. 3. 4. • Definition of features Extraction and selection of features Definition of classifier. Training and testing the classifier. The "no free lunch theorem“. 46

Fuzzy Logic in Medical Applications and Image Proc. 47

Fuzzy Logic in Medical Applications and Image Proc. 10 Questions: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Are the input and output data intrinsically vague, linguistic, incomplete or ambiguous? Is the use of FL an additional value to the final system? Is the use of Fuzzy Sets reducing somehow the accuracy of the system? Is the FLS taking into account any (known) underlying model for the data? Is really the use of FL simplifying the problem and improving the understanding of it? Is the fuzzification of inputs and outputs robust to scale or range changes? Is this really a membership problem rather than a probabilistic problem? How many parts of the final system requires fine-tuning and human intervention? How much of the final system is based on heuristics? Why am I using FLS or soft-computing for this problem? 48

Case of Study: MRI noise filtering

Fuzzy Logic in Medical Applications and Image Proc. Problem Statement • We want to “remove” noise from some Magnetic Resonance Imaging. • We are going to use a FL-based, similar to some of the ones proposed in literature: [Kerre 00, Szczepaniak 00, Nachtegael 03] • Based on linguistic rules, such as “If the pixels around are very different, then filter…” 50

Fuzzy Logic in Medical Applications and Image Proc. 1. Are the input and output data intrinsically vague, linguistic, incomplete or ambiguous? NO. Data are numerical values from a MRI scanner 2. Is the use of FL an additional value to the final system? Only if I am able to somehow codify “expert” information into the system 51

Fuzzy Logic in Medical Applications and Image Proc. 3. Is the use of Fuzzy Sets reducing somehow the accuracy of the system? A 1 A 2 A 3 Fuzzification of inputs and outputs may reduce the dynamic range of data 4. Is the FLS taking into account any (known) underlying model for the data? No, and it could be a problem… 52

Parallel Imaging Non-parallel MR signal • Rician Model Complex Magnitude Rician Rayleigh 53

Parallel Imaging • • • Parallel imaging uses multiple coils to distribute the data acquisition burden. Acquisition can then be subsampled to reduce image acquisition time. Reconstruction combines coil images and removes the spatial aliasing Hoge: Combining GRAPPA and SENSE to improve parallel MR Imaging 54

Parallel Imaging Composite Magnitude Image 55

Parallel Imaging Composite Magnitude Image • Non-central chi model Non Central Chi 56

Fuzzy Logic in Medical Applications and Image Proc. 4. Is the FLS taking into account any (known) underlying model for the data? No, and it could be a problem… It is well known that these kind of filtering my introduce a bias in the final image, because: E{M(x)2}=A(x)2+2 Ls 2 57

Fuzzy Logic in Medical Applications and Image Proc. 5. Is really the use of FL simplifying the problem and improving the understanding of it? 6. Is the fuzzification of inputs and outputs robust to scale or range changes? Real MR data from the scanner has a wide dynamic range, and very different from scanner to scanner. How to define the axis? 58

Fuzzy Logic in Medical Applications and Image Proc. 7. Is this really a membership problem rather than a probabilistic problem? Membership: Background, noise, tissue… Probabilistic: Underlying model 8. How many parts of the final system requires fine-tuning and human intervention? Rules of the system, FIE… 59

Fuzzy Logic in Medical Applications and Image Proc. 9. How much of the final system is based on heuristics? Basically the rules 10. Why am I using FLS or soft-computing for this problem? CONCLUSIONS? 60

Examples

Fuzzy Logic in Medical Applications and Image Proc. Ultrasound Filtering [Aja-Fernandez (2001)] 62

Fuzzy Logic in Medical Applications and Image Proc. Ultrasound Filtering [Aja-Fernandez (2001)] • Modification of Anisotropic Diffusion filtering to introduce a fuzzy logic system. • c(t) is the diffusion coefficient. It is defined as a monotonically decreasing function of the gradient magnitude, so that the flow increases within homogeneous regions where the gradient is small. 63

Fuzzy Logic in Medical Applications and Image Proc. Ultrasound Filtering • [Aja-Fernandez (2001)] New fuzzy diffusion coefficient 64

Fuzzy Logic in Medical Applications and Image Proc. Ultrasound Filtering [Aja-Fernandez (2001)] 65

Fuzzy Logic in Medical Applications and Image Proc. Bone age assessment [Aja-Fernandez (2004)] 66

Fuzzy Logic in Medical Applications and Image Proc. Bone age assessment • • • [Aja-Fernandez (2004)] Bone age assessment: quantitative assessment of skeletal maturity in children Tanner Whitehouse method: guidelines to analyze each bone are described using words (natural language descriptions), sometimes in a vague way. One particular bone may show features belonging to different stages or a particular bone shape could be classifiable into two possible predefined labels of the same feature. Impossible to “repeat” the acquisition: radiation awareness. Suitable for soft approach? 67

Fuzzy Logic in Medical Applications and Image Proc. • • Stage D. The maximum diameter is half or more the width of the metaphysis. The epiphysis has broadened chiefly at its lateral side, so that this portion is thicker and more rounded, the medial portion more tapering. The center third of the proximal surface is flat and slightly thickened and the gap between it and the radial metaphysis has narrowed to about a millimeter. Stage G. The dorsal surface now has distinct lunate and scaphoid articular edges joined at a small hump. The medial border of the epiphysis has developed palmar and dorsal surfaces for articulation with the ulnar epiphysis; either the palmar or the dorsal surface may be the one that projects medially, depending on the position of the wrist. The proximal border of the epiphysis is now slightly concave. 68

Fuzzy Logic in Medical Applications and Image Proc. Fuzzy Thresholding [Aja-Fernandez (2010)] 69

Fuzzy Logic in Medical Applications and Image Proc. Fuzzy Thresholding [Aja-Fernandez (2010)] Soft segmentation of the tissues using a fuzzy thresholding. 1. Histogram to PTS fuzzy sets (probability to membership) 2. 3. Soft thresholding assigning points to sets Adding local information and/or rules 70

Fuzzy Logic in Medical Applications and Image Proc. Homework: More examples (I) • Image registration: Fuzzy regularisation of deformation fields in image registration [Tristan-Vega (2008)]. • Surgery Simulation: collision handling. A 3 -D Collision Handling Algorithm for Surgery Simulation Based on Feedback Fuzzy Logic [Garcia-Perez et al. (2009)]. • Van der Weken: Image quality assessment using fuzzy similarity measures [Weken et al. (2007, 2004)]. • Russo and Ramponi: Nonlinear fuzzy operators for image processing [Russo and Ramponi (1994)] • Medical Diagnosis: Measurement and valuation of a fuzzy mathematical model for medical diagnosis [Esogbue and Elder (1983)], A weighted fuzzy reasoning algorithm for medical diagnosis [Chen (1994)], A fuzzy clustering based segmentation system as support to diagnosis in medical imaging [Masulli and Schenone (1999)]. 71

Fuzzy Logic in Medical Applications and Image Proc. Homework: More examples (II) • Volume quantification by fuzzy logic modelling in freehand ultrasound imaging [Betrouni et al. (2009)]. • Fuzzy adaptive median filter [Toprak and Guler (2006)]; [Zhou et al. (2008)] • Weighted fuzzy mean filters for image processing [Lee et al. (1997)] • MRI segmentation: A non-local fuzzy segmentation method: Application to brain MRI [Caldairou et al. (2011)]. 72

Questions?

Fuzzy logic in medical applications and image processing: main approaches, do's and don't's Santiago Aja-Fernández Medical Imaging Using Bio-Inspired and Soft Computing sanaja@tel. uva. es MIBISOC Technical Course 2012