Medical Imaging Projects Daniela S Raicu Ph D

  • Slides: 49
Download presentation
Medical Imaging Projects Daniela S. Raicu, Ph. D Assistant Professor Email: draicu@cs. depaul. edu

Medical Imaging Projects Daniela S. Raicu, Ph. D Assistant Professor Email: draicu@cs. depaul. edu Lab URL: http: //facweb. cs. depaul. edu/research/vc/

IMP & Medi. X Labs @ De. Paul Faculty: GM. Besana, L. Dettori, J.

IMP & Medi. X Labs @ De. Paul Faculty: GM. Besana, L. Dettori, J. Furst, G. Gordon, S. Jost, D. Raicu, N. Tomuro CTI Students: W. Horsthemke, C. Philips, R. Susomboon, J. Zhang E. Varutbangkul, S. G. Valencia IMP Collaborators & Funding Agencies • National Science Foundation (NSF) - Research Experience for Undergraduates (REU) • Northwestern University - Department of Radiology, Imaging Informatics Section • University of Chicago – Medical Physics Department • Argonne National Laboratory - Biochip Technology Center • Mac. Arthur Foundation Med. IX REU Program, Summer 2007 2

Outline § Medical Imaging and Computed Tomography § Soft Tissue Segmentation in Computed Tomography

Outline § Medical Imaging and Computed Tomography § Soft Tissue Segmentation in Computed Tomography § Project 1: Region-based classification § Project 2: Texture-based snake approach § Content-based Image Retrieval and Annotation § Project 3: Lung Nodule Retrieval based on image content and radiologists’ feedback § Project 4: Associations discovery between image content and radiologists’ assessment Med. IX REU Program, Summer 2007 3

What is Medical Imaging (MI)? The study of medical imaging is concerned with the

What is Medical Imaging (MI)? The study of medical imaging is concerned with the interaction of all forms of radiation with tissue and the development of appropriate technology to extract clinically useful information from observation of this technology. X-Ray CT Med. IX REU Program, Summer 2007 f. MRI 4

Computed Tomography (CT) • G. Hounsfield (computer expert) and A. M. Cormack (physicist) (Nobel

Computed Tomography (CT) • G. Hounsfield (computer expert) and A. M. Cormack (physicist) (Nobel Prize in Medicine in 1979) • CT overcomes limitations of plain radiography • CT doesn’t superimpose structures (like X-ray) • CT is an imaging based on a mathematical formalism that states that if an object is viewed from a number of different angles than a cross-sectional image of it can be computed (reconstructed) ________________________ Med. IX REU Program, Summer 2007 5

CT Data Stages of construction of a voxel dataset from CT data (a) CT

CT Data Stages of construction of a voxel dataset from CT data (a) CT data capture works by taking many one dimensional projections through a slice (scanning) (b) CT reconstruction pipeline Med. IX REU Program, Summer 2007 6

CT – Data Acquisition Slice-by-slice acquisition • X-ray tube is rotating around patient to

CT – Data Acquisition Slice-by-slice acquisition • X-ray tube is rotating around patient to acquire a slice • patient is moved to acquire the next slice Volume acquisition • X-ray tube is moving continuously along a spiral (helical) path and the data is acquired continuously ________________________ Med. IX REU Program, Summer 2007 7

CT – Data Acquisition (a) slice-by-slice scanning (b) Spiral (volume) scanning Med. IX REU

CT – Data Acquisition (a) slice-by-slice scanning (b) Spiral (volume) scanning Med. IX REU Program, Summer 2007 8

CT – SPIRAL SCANNING • a patient is moved 10 mm/s (24 cm /

CT – SPIRAL SCANNING • a patient is moved 10 mm/s (24 cm / single scan) • slice thickness: 1 mm-1 cm • faster than slice-by-slice CT • no shifting of anatomical structures • slice can be reconstructed with an arbitrary orientation with (a single breath) volume CT multi-slice systems: • parallel system of detectors • 4/8/16 slices at a time • generates a large data of thin slices • better spatial resolution ( better reconstruction) Med. IX REU Program, Summer 2007 9

CT - DATA PROCESSING CT numbers (Hounsfield units) HU: • computed via reconstruction algorithm

CT - DATA PROCESSING CT numbers (Hounsfield units) HU: • computed via reconstruction algorithm (~tissue density/ X-ray absorption) • most attenuation (bone) • least attenuation (air) • blood/calcium increases tissue density Med. IX REU Program, Summer 2007 Understanding Visual Information: Technical, Cognitive and Social Factors 10

CT - DATA PROCESSING Relationship between CT numbers and brightness level Med. IX REU

CT - DATA PROCESSING Relationship between CT numbers and brightness level Med. IX REU Program, Summer 2007 Understanding Visual Information: Technical, Cognitive and Social Factors 11

CT - IMAGE DISPLAY Human eye can perceive only a limited range gray-scale values

CT - IMAGE DISPLAY Human eye can perceive only a limited range gray-scale values Thoracic image: a) width 400 HU/level 40 HU (no lung detail is seen) b) width 1000 HU/level – 700 HU (lung detail is well seen; bone and soft tissue detail is lost) Med. IX REU Program, Summer 2007 12

CT Medical Imaging (MI)@ CTI Filtering Registration Projects 1&2: Texture-based Correction soft-tissue segmentation Segmentation

CT Medical Imaging (MI)@ CTI Filtering Registration Projects 1&2: Texture-based Correction soft-tissue segmentation Segmentation Projects 3&4: Content-based Visualization Analysis medical image retrieval and annotation Classification Retrieval Med. IX REU Program, Summer 2007 13

Outline § Medical Imaging and Computed Tomography § Soft Tissue Segmentation in Computed Tomography

Outline § Medical Imaging and Computed Tomography § Soft Tissue Segmentation in Computed Tomography § Project 1: Region-based classification approach § Project 2: Texture-based snake approach § Content-based Image Retrieval and Annotation § Project 3: Lung Nodule Retrieval based on image content and radiologists’ feedback § Project 4: Associations discovery between image content and radiologists’ assessment Med. IX REU Program, Summer 2007 14

Soft-tissue Segmentation in Computed Tomography Goal: context-sensitive tools for radiology reporting Approach: pixel-based texture

Soft-tissue Segmentation in Computed Tomography Goal: context-sensitive tools for radiology reporting Approach: pixel-based texture classification Pixel Level Texture Extraction Pixel Level Classification Med. IX REU Program, Summer 2007 Organ Segmentation 15

Soft-tissue Segmentation in Computed Tomography Pixel-based texture extraction: Input Patient Data Characteristics: § hundreds

Soft-tissue Segmentation in Computed Tomography Pixel-based texture extraction: Input Patient Data Characteristics: § hundreds of images per patient § image spatial resolution: 512 x 512 § image gray-level resolution: 212 Challenges: § Storage: § Input: 0. 5+ terabyte of raw data dispersed over about 100 K+ images § Output: 90+ terabytes of low-level features in a 180 dimensional feature space § Compute: § 24 hours of compute time = 180 features for a single image on a modern 3 GHz workstation Pixel Level Texture Extraction Output Data Characteristics: § low-level image features (numerical descriptors) § k=180 Haralick texture features per pixel (9 descriptors x 4 directions x 5 displacements) Med. IX REU Program, Summer 2007 16

Project 1: Challenges and opportunities § Calculate image features at region-level instead of pixel-level

Project 1: Challenges and opportunities § Calculate image features at region-level instead of pixel-level § Include Gabor features in the feature extraction phase in addition to the cooccurrence texture features § Explore different approaches for region classification in addition to the decision tree approach Current Implementation: Matlab Stack of CT slices Image Partitioning Feature Extraction Med. IX REU Program, Summer 2007 Region Classification 17

Liver Segmentation Example J. D. Furst, R. Susomboon, and D. S. Raicu, "Single Organ

Liver Segmentation Example J. D. Furst, R. Susomboon, and D. S. Raicu, "Single Organ Segmentation Filters for Multiple Organ Segmentation", IEEE 2006 International Conference of the Engineering in Medicine and Biology Society (EMBS'06) Original Image Initial Seed at 90% Split & Merge at 85% Split & Merge at 80% Region growing at 70% Region growing at 60% Segmentation Result Med. IX REU Program, Summer 2007 18

Soft-tissue Segmentation in Computed Tomography Snake Application Demo Next figures are demonstrated how to

Soft-tissue Segmentation in Computed Tomography Snake Application Demo Next figures are demonstrated how to automatically classify the CT images of heart and liver. Med. IX REU Program, Summer 2007 19

Demo For HEART There are 4 main menu to operate this application. Med. IX

Demo For HEART There are 4 main menu to operate this application. Med. IX REU Program, Summer 2007 OPEN: SEGMENT: Toa automatically To open new Image. segment the region of interest TEXTURE: organ To calculate the texture CLASSIFICATION: models: co. To automatically occurrence/r classify the un-length segmented organ 20

HEART: Segmentation The application allows users to change Snake/ Active contour algorithm parameters Med.

HEART: Segmentation The application allows users to change Snake/ Active contour algorithm parameters Med. IX REU Program, Summer 2007 21

HEART: Segmentation (cont. ) Button is clicked User selects points around the region of

HEART: Segmentation (cont. ) Button is clicked User selects points around the region of Med. IX REU Program, Summer 2007 interest 22

HEART: Segmentation (result) Show segmented organ If the user likes the result of the

HEART: Segmentation (result) Show segmented organ If the user likes the result of the segmentation, then the user will go to the classification step Med. IX REU Program, Summer 2007 23

HEART: Classification Selection of texture models: Co-occurrence, Run-length, Or Combine both models Texture features

HEART: Classification Selection of texture models: Co-occurrence, Run-length, Or Combine both models Texture features corresponding to the selected texture model are calculated and shown here Med. IX REU Program, Summer 2007 24

HEART: Classification Results are shown as follows. Predicted organ: Heart Probability: 0. 86 And

HEART: Classification Results are shown as follows. Predicted organ: Heart Probability: 0. 86 And also rule which is used to predict that this segmented organ is HEART Med. IX REU Program, Summer 2007 25

Demo For LIVER Start application by open and load the image. Med. IX REU

Demo For LIVER Start application by open and load the image. Med. IX REU Program, Summer 2007 26

LIVER: Segmentation The application allows users to change Snake/ Active contour algorithm parameters Med.

LIVER: Segmentation The application allows users to change Snake/ Active contour algorithm parameters Med. IX REU Program, Summer 2007 27

LIVER: Segmentation (cont. ) Segmentation Button is clicked User selects points around the region

LIVER: Segmentation (cont. ) Segmentation Button is clicked User selects points around the region of interest Med. IX REU Program, Summer 2007 28

LIVER: Segmentation Result Show segmented organ If user is satisfied with the result, then

LIVER: Segmentation Result Show segmented organ If user is satisfied with the result, then it will go to the classification step Med. IX REU Program, Summer 2007 29

LIVER: Classification Select texture models: Co-occurrence, Run-length, Or Combine both models Texture features is

LIVER: Classification Select texture models: Co-occurrence, Run-length, Or Combine both models Texture features is calculated for the selected model Med. IX REU Program, Summer 2007 30

LIVER: Classification Results are shown as follows. Predicted organ: Liver Probability: 1. 00 And

LIVER: Classification Results are shown as follows. Predicted organ: Liver Probability: 1. 00 And also rule which is used to predict that this segmented organ is LIVER Med. IX REU Program, Summer 2007 31

Project 2: Challenges and opportunities § Calculate texture image features at the pixel level

Project 2: Challenges and opportunities § Calculate texture image features at the pixel level instead of using the graylevels § Apply snake on the texture features § Investigate different ways to objectively compare two segmentation algorithms, in particular the snake and the classification-based approach Current Implementation: Matlab Med. IX REU Program, Summer 2007 32

Outline § Medical Imaging and Computed Tomography § Soft Tissue Segmentation in Computed Tomography

Outline § Medical Imaging and Computed Tomography § Soft Tissue Segmentation in Computed Tomography § Project 1: Region-based classification approach § Project 2: Texture-based snake approach § Content-based Image Retrieval and Annotation § Project 3: Lung Nodule Retrieval based on image content and radiologists’ feedback § Project 4: Associations discovery between image content and radiologists’ assessment Med. IX REU Program, Summer 2007 33

Outline § Medical Imaging and Computed Tomography § Soft Tissue Segmentation in Computed Tomography

Outline § Medical Imaging and Computed Tomography § Soft Tissue Segmentation in Computed Tomography § Project 1: Region-based classification approach § Project 2: Texture-based snake approach § Content-based Image Retrieval and Annotation § Project 3: Lung Nodule Retrieval based on image content and radiologists’ feedback § Project 4: Associations discovery between image content and radiologists’ assessment Med. IX REU Program, Summer 2007 34

Content-based medical image retrieval (CBMS) systems Definition of Content-based Image Retrieval: Content-based image retrieval

Content-based medical image retrieval (CBMS) systems Definition of Content-based Image Retrieval: Content-based image retrieval is a technique for retrieving images on the basis of automatically derived image features such as texture and shape. Applications of Content-based Image Retrieval: § Teaching § Case-base reasoning § Evidence-based medicine Med. IX REU Program, Summer 2007 35

Diagram of a CBIR Image Database Image Features Feature Extraction [D 1, D 2,

Diagram of a CBIR Image Database Image Features Feature Extraction [D 1, D 2, …Dn] Similarity Retrieval Query Image Feedback Algorithm User Evaluation Med. IX REU Program, Summer 2007 Query Results 36

CBIR as a tool for lookup and reference • Case Study: lung nodules retrieval

CBIR as a tool for lookup and reference • Case Study: lung nodules retrieval – Lung Imaging Database Resource for Imaging Research http: //imaging. cancer. gov/programsandresources/Inf ormation. Systems/LIDC/page 7 – 29 cases, 5, 756 DICOM images/slices, 1, 143 nodule images – 4 radiologists annotated the images using 9 nodule characteristics: calcification, internal structure, lobulation, malignancy, margin, sphericity, spiculation, subtlety, and texture • Goals: – Retrieve nodules based on image features: • Texture, Shape, and Size – Find the correlations between the image features and the radiologists’ annotations Med. IX REU Program, Summer 2007 37

Examples of nodule images Med. IX REU Program, Summer 2007 38

Examples of nodule images Med. IX REU Program, Summer 2007 38

CBIR as a tool for lung nodule lookup and reference Low-level feature extraction: Med.

CBIR as a tool for lung nodule lookup and reference Low-level feature extraction: Med. IX REU Program, Summer 2007 39

Nodule Characteristics – Calcification • (1. Popcorn, 2. Laminated, 3. Solid, 4. Non-Central, 5.

Nodule Characteristics – Calcification • (1. Popcorn, 2. Laminated, 3. Solid, 4. Non-Central, 5. Central, 6. Absent) – Internal Structure • (1. soft tissue, 2. fluid, 3. fat, 4. air) – Subtlety • (1. extremely subtle, . . , 5. obvious) – Sphericity • (1. Linear, 2. . . . , 3. Ovoid, 4. . . , 5. Round) – Texture • (1. Non-Solid, 2. . . , 3. Part Solid, 4. . . . , 5. Solid) Med. IX REU Program, Summer 2007 40

Nodule Characteristics – Margin • (1. Poorly, . . . , 5. Sharp) –

Nodule Characteristics – Margin • (1. Poorly, . . . , 5. Sharp) – Lobulation • (1. Marked, . . , 5. No Lobulation) – Spiculation • (1. Marked, . . , 5. No Spiculation) – Malignancy • (1. Highly Unlikely for Cancer, . . . . , 5. Highly Suspicious for Cancer) Med. IX REU Program, Summer 2007 41

Choose a nodule Med. IX REU Program, Summer 2007 42

Choose a nodule Med. IX REU Program, Summer 2007 42

Choose an image feature& a similarity measure M. Lam, T. Disney, M. Pham, D.

Choose an image feature& a similarity measure M. Lam, T. Disney, M. Pham, D. Raicu, J. Furst, “Content-Based Image Retrieval for Med. IX REU Program, 2007 Nodule Images”, SPIE Medical Imaging 43 Pulmonary Computed. Summer Tomography Conference, San Diego, CA, February 2007

Med. IX REU Program, Summer 2007 Retrieved Images 44

Med. IX REU Program, Summer 2007 Retrieved Images 44

Project 3: Challenges and opportunities § Calculate co-occurrence texture features at the local level

Project 3: Challenges and opportunities § Calculate co-occurrence texture features at the local level instead of global level § Incorporate shape and size features in the retrieval process in addition to texture features § Integrate radiologists’ assessments/feedback into the retrieval process § Investigate different approaches for retrieval in addition to similarity measures § Report the retrieval results with a certain confidence level (probability) instead of just a binary output (similar/not similar) Current implementation: C# Available Open Source at: http: //brisc. sourceforge. net/ Med. IX REU Program, Summer 2007 45

Outline § Medical Imaging and Computed Tomography § Soft Tissue Segmentation in Computed Tomography

Outline § Medical Imaging and Computed Tomography § Soft Tissue Segmentation in Computed Tomography § Project 1: Region-based classification approach § Project 2: Texture-based snake approach § Content-based Image Retrieval and Annotation § Project 3: Lung Nodule Retrieval based on image content and radiologists’ feedback § Project 4: Associations discovery between image content and radiologists’ assessment Med. IX REU Program, Summer 2007 46

Associations between image content and semantics Med. IX REU Program, Summer 2007 47

Associations between image content and semantics Med. IX REU Program, Summer 2007 47

Project 4: Challenges and opportunities § Investigate other approaches for finding the associations between

Project 4: Challenges and opportunities § Investigate other approaches for finding the associations between image features and radiologists’ assessment in addition to logistic regression and decision trees § from image content to semantics § from semantics to semantics § from image features and semantics to semantics § Create GUIs to display examples of images for each semantic concept § Investigate how the current associations discovery approaches apply to mammography assessment (Northwestern project) Current implementation: Matlab, Weka, SPSS Med. IX REU Program, Summer 2007 48

Questions? Thank you! Med. IX REU Program, Summer 2007 49

Questions? Thank you! Med. IX REU Program, Summer 2007 49