Outline Introduction Object of medical image processing Imaging

Outline Introduction ¡ Object of medical image processing ¡ Imaging devices ¡ applications ¡ Related techniques for Medical imaging ¡ Research Results ¡ Future works ¡

Introduction What is Medical imaging? ¡ Why do we need digital image processing? ¡ What kind of problems are often caused in medical images? ¡ l l l ¡ Blurring caused by respiratory or motion Low contrast caused by imaging device or resolution Complicated textures Research trends have been transferred from 2 -D to 3 -D reconstruction

Introduction (continue) ¡ Integrate all possible methods in the filed of DIP, pattern recognition, and computer graphics Qualitative ¡ Quantitative ¡ ¡ Three categories of imaging in different modalities l l l Structural image Functional image Molecular image

Object ¡ Help physicians diagnose l Reduce inter- and intra-variability Produce qualitative and quantitative assessment by computer technologies ¡ Determine appropriate treatments according to the analyses ¡ Surgical simulation or skills to reduce possible errors ¡

Medical Imaging Modalities X-ray ¡ Ultrasound: non-invasive ¡ Computed tomography ¡ Magnetic resonance imaging ¡ SPECT (Single photon emission tomography) ¡ PET( Positron emission tomography) ¡ Microscopy ¡

X-ray

Ultrasound 2 -D sonography ¡ 3 -D sonography ¡ Doppler color sonography ¡ l l ¡ A series of 2 -D projection Reconstruction 4 -D sonography

Computed tomography

MRI-structural and functional image

Related techniques ¡ Image processing l l l Segmentation Registration Feature Extraction ¡ ¡ l ¡ Shape feature Texture Motion tracking Pattern recognition l l l Supervised learning Un-supervised learning Neuro network Fuzzy Support vector machine(SVM) Genetic algorithm

Related techniques ¡ 3 -D graphic l l l Virtual diagnose or visualization Fusion between different modalities Bio-medical visualization

SPECT-functional image

PET (Positron emission tomography)

Cell identification via microscope ¡ Tools l Traditional optical microscope ¡ l Fluorescent microscope ¡ l Identification for nuclear and gene expression Laser confocal microscope ¡ l Stained specimen Identification from 2 -D to 3 -D Multi-photon microscope ¡ Identification from 2 -D to 3 -D

Applications in a hospital ¡ ¡ Assist surgeon plan surgical operation or diagnose Picture archiving system (PACS) l ¡ ¡ ¡ 將醫療系統中所有的影像，以數位化的方式儲存，並經 由網路傳遞至同系統中，供使用者於遠側電腦螢幕閱讀 影像並判讀。 Telemedicine Surgical simulation: Medical Visualization， Surgical augmented Reality, Medicalpurpose robot, Surgery Simulation，Image Guided Surgery，Computer Aided Surgery Estimate the location, size and shape of tumor

PACS System

Virtual Surgery

Related techniques ¡ Classification of normal or abnormal tissues such as carcinoma l l Pre-processing: Contrast enhancement, noise removal, and edge detection Lesion segmentation: extract contours of interest thresholding ¡ 2 -D segmentation ¡ 3 -D segmentation based on voxel data ¡ Color image processing ¡

Our study Virtual colonoscopy ¡ Bone tumor segmentation with MRI and virtual display ¡ Breast carcinoma based on histology and cytology ¡ Visualization of cell activities using confocal laser scanning microscope ¡

Virtual colonscopy-Browsing or navigation within a colon ¡ ¡ ¡ ¡ Helical CT –patients injected contrast medium Re-sampling—Voxel-based Interpolation Automatic segmentation (seed) l threshloding Determination of the skeleton of the colon Connected-Component Labeling Surface rendering and volume rendering Extraction of suspicious sub-volumes for diagnosis

Automatic segmentation

Determination of the skeleton of the colon

Display and measurement

Bone tumor segmentation with MRI and virtual display—Contrast medium ¡ Otsu thresholding l Region growing Tri-linear interpolation ¡ Morphological post-processing ¡ Surface rendering ¡ Measurement ¡

Histogram of T 1 weighted and T 2 weighted

Classification of Breast Carcinoma

59

60

體積比例量測 Case 1 Case 2 Case 3 細胞質區域(Voxels) 224396 521563 629562 蛋白質活動劇烈區域 3819 3387 7785 1. 7019% 0. 649% 1. 236% (Voxels) 比例 67

效能評估 Process Case 1: Time(Sec) Case 2: Time(Sec) Case 3: Time(Sec) 68 74 83 7 6 7 整體三維重建 9 10 9 整體時間 84 90 99 細胞質區域分割 蛋白質活動劇烈區 域分割 68

Requirements for medical image processing system in clinical diagnosis ¡ ¡ Automatic and less human interaction Qualitative and quantitative measurements Stable and reliable (experiments with much more cases) Performance evaluation l l l True positive, true negative, false positive, false negative Accuracy, sensitivity, and specificity Receiving operating characteristic curve (An index for evaluating the effectiveness of classification ¡ Optimal classification threshold ¡ Area under ROC approach 1 – better classification

ROC curve

Analyses of prognosis on breast cancer for a stained tissue Microscopy with different resolution (400 or 100) for a stained tissue ¡ Fluorescent microscopy in detecting the number of chromosome ¡ Immunohistochemistry(IHC) ¡

Her-2 IHC image

Fish image(normal)

Fish image (abnormal)

Preliminaries or problems ? ¡ ¡ ¡ Blurring often caused by patient motion or respiration Clinical opinion or idea obtained from an experienced surgeon Non-absolute answers at some specific conditions Trade-off between complexity and performance Large variations for different image modality

Preliminaries or problems ? ¡ ¡ ¡ Automation is necessary so as to help physicians Prove identification accuracy— comparison between manual and image processing approaches Classification based on neural network, pattern recognition, or fuzzy, . . etc is crucial in practical applications

¡ Thanks for your attention!