Visual Computing Research CTI De Paul University Daniela

Visual Computing Research @ CTI, De. Paul University Daniela Raicu Assistant Professor draicu@cs. depaul. edu http: //facweb. cs. depaul. edu/research/vc Medical Image Processing

Visual Computing Group Ø CTI Faculty: § § § § Gian Mario Besana Lucia Dettori Jacob Furst Gerald Gordon Steve Jost Yakov Keselman Daniela Raicu Ø Collaborators: Department of Radiology, Northwestern University & Northwestern Memorial Hospital, Chicago, IL § Dr. David Channin, Chief of Informatics, Department of Radiology Medical Image Processing 6/12/2021 2

Visual Computing Group Ø Graduate Students: § § John Campion, Ramzy Darwish William Horsthemke, Gabriel Sanchez, Winnie Tsang Ø Undergraduate Students: § § § Stelian Aioanei, Andrew Corboy Jong Lee, Mikhail Kalinin Lindsay Semler, Dong-Hui Xu Ø Visual Computing (VC) area: § § § CSC 381/CSC 481: Introduction to Image Processing CSC 382/CSC 482: Image Analysis and its Applications CSC 384/CSC 484: Introduction to Computer Vision Ø VC research seminar: Fall Quarter, Friday, 5: 00 - 6: 00 pm Ø VC workshop: Spring Quarter, Friday, April 15 th , 2005 Ø Intelligent Multimedia Processing (IMP) lab: http: //facweb. cs. depaul. edu/research/vc Medical Image Processing 6/12/2021 3

Research problems Content-based Image Retrieval: Image retrieval systems that permit image searching based on features automatically extracted from the images’ own visual content are called content-based image retrieval (CBIR) systems. -visual features (primitive or low-level image features) Domain-specific features: - fingerprints, human faces General features: - color, texture, shape Drawback: -lack of expressive power Medical Image Processing 6/12/2021 4

Content-based Image Retrieval Feature Extraction Image Database Semantic Gap ? Mountains and waterfalls It is a nice sunset. Text Database Medical Image Processing Meaning: Sunset 6/12/2021 5

Content-based Image Retrieval Feature Representation: Two examples of original images and their representations. Medical Image Processing 6/12/2021 6

Content-based Image Retrieval Two examples of original images and their representations: Medical Image Processing 6/12/2021 7

Content-based Image Retrieval Similarity Measure: S(q 1, t 1) Image T: Image Q: , bi = masking bit Medical Image Processing 6/12/2021 8

Content-based Image Retrieval Query Medical Image Processing Retrieval Results 6/12/2021 9

Content-based Image Retrieval Image Search Medical Image Processing 6/12/2021 10

Content-based Image Retrieval Medical Image Processing 6/12/2021 11

Medical Imaging Problem statement: Human body organs’ classifications using raw data (pixels) from abdominal and chest CT images. labels for the organs present in the image heart backbone Medical Image Processing 6/12/2021 12

Medical Imaging Segmentation Organ/Tissue segmentation in CT images - Data: 340 DICOM images - Segmented organs: liver (56), kidneys (55), spleen (39), backbone (140), & heart (50) - Segmentation algorithm: Active Contour Mappings (Snakes) - A boundary-based segmentation algorithm - Input for the algorithm: a number of initial points & five main parameters that influence the way the boundary is formed. Medical Image Processing 6/12/2021 13

Segmentation: Matlab Demo Advantage: it detects complex shapes Disadvantage: it needs manual selection of the initial points and of the parameters Our Solution: perform clustering of similar regions using a neural network Medical Image Processing 6/12/2021 14

Segmentation: Examples Medical Image Processing 6/12/2021 15

Segmentation: Examples Medical Image Processing 6/12/2021 16

Texture Analysis & Classification Organ/Tissue segmentation in CT images Calculate numerical texture descriptors for each region [D 1, D 2, …D 21] Medical Image Processing Classification rules for tissue/organs in CT images 6/12/2021 IF HGRE <= 0. 38 AND ENTROPY > 0. 43 AND SRHGE <= 0. 20 AND CONTRAST > 0. 029 THEN Prediction = Heart Probability = 0. 99 17

Medical Imaging Texture Analysis Entropy Energy Contrast 3. 892828 . 034692 2. 764427 Medical Image Processing Homogeneity Sum. Mean. 6345745 Variance Correlation Maximum Probability Inverse Difference Moment Cluster Tendency . 110921 . 112929 . 44697 26. 471211 11. 662886 7. 308909 6/12/2021 18

Medical Imaging Texture Analysis Entropy Energy Contrast 3. 4151415 . 108713 6. 224426 Medical Image Processing Homogeneity Sum. Mean. 631435 Variance Correlation Maximum Probabilit y Inverse Difference Moment Cluster Tendenc y . 0723125 . 3081855 . 280289 31. 139159 13. 628323 9. 340897 6/12/2021 19

Medical Imaging Texture Analysis Entropy Energy Contrast 3. 38482 . 055998 3. 49784 Medical Image Processing Homogeneity Sum. Mean. 5577785 Variance Correlation Maximum Probabilit y Inverse Difference Moment Cluster Tendenc y . 1436305 . 1250245 . 437988 11. 453111 14. 278469 3. 737737 6/12/2021 20

Medical Imaging Texture Analysis Entropy Energy Contrast 3. 3099875 . 049172 3. 066407 Medical Image Processing Homogeneity Sum. Mean. 5369255 Variance Correlation Maximum Probabilit y Inverse Difference Moment Cluster Tendenc y . 0377875 . 0897425 . 460422 3. 471442 12. 309719 1. 634463 6/12/2021 21

Medical Imaging Texture Analysis Entropy Energy Contrast Homogeneity Sum. Mean Variance Correlation Maximum Probability Inverse Difference Moment Cluster Tendency 2. 72509 . 091388 1. 618982 . 6208175 11. 755226 0. 912752 . 123976 . 1742075 . 506894 2. 032082 Medical Image Processing 6/12/2021 22

Texture Descriptors: Matlab Demo Medical Image Processing 6/12/2021 23

Organ/Tissue Classification Calculate numerical texture descriptors for each region [D 1, D 2, …D 21] Classification rules for tissue/organs in CT images IF HGRE <= 0. 38 AND ENTROPY > 0. 43 AND SRHGE <= 0. 20 AND CONTRAST > 0. 029 THEN Prediction = Heart Probability = 0. 99 Algorithm: - decision trees Output: Decision Rules Performance estimated using: - sensitivity - specificity Advantage: Set of decision rules that can be used for annotation Medical Image Processing 6/12/2021 24

Organ/Tissue Classification Examples of Decision Tree Rules for Combined Data: • IF (HGRE <= 0. 3788) & (CLUSTER <= 0. 0383095) & (INVERSE <= 0. 768085) & (SUMMEAN <= 0. 556015) & (SRLGE <= 0. 101655) & (ENEGRY > 0. 106715) THEN Prediction = Spleen, Probability = 0. 928571 • IF (HGRE <= 0. 3788) & (CLUSTER <= 0. 0383095) & (INVERSE <= 0. 768085) & (SUMMEAN <= 0. 556015) & (SRLGE > 0. 101655) THEN Prediction = Liver , Probability = 1. 000000 • IF (HGRE <= 0. 3788) & (CLUSTER <= 0. 0383095) & (INVERSE <= 0. 768085) & (SUMMEAN > 0. 556015) & (GLNU <= 0. 087365) THEN Prediction = Kidney, Probability = 0. 924658 Medical Image Processing 6/12/2021 25

Organ/Tissue Classification Examples of Decision Tree Rules for Combined Data: • IF (HGRE <= 0. 3788) & (CLUSTER > 0. 0383095) & (GLNU > 0. 03184) & (ENTROPY > 0. 433185) & (SRHGE <= 0. 19935) & (CONTRAST > 0. 0295805) THEN Prediction = Heart, Probability = 0. 988372 • IF (HGRE <= 0. 3788) & (CLUSTER > 0. 0383095) & (GLNU <= 0. 03184) & (LRE <= 0. 123405) THEN Prediction = Backbone, Probability = 1. 000000 Medical Image Processing 6/12/2021 26

Organ/Tissue Classification Decision Tree Accuracy on Testing Data (Co-occurrence, Run-length, and Combined): ORGAN Sensitivity Specificity Precision Accuracy Backbone 96% / 98% 99% / 100% / 99% 98% / 99% Liver 64% / 57% / 78% 96% / 98% / 95% 75% / 84% / 71% 92% / 92% Heart 79% / 82% / 75% 96% / 95% / 98% 80% / 77% / 90% 94% / 93% / 95% Kidney 89% / 89% 96% / 93% / 96% 80% / 67% / 77% 94% / 92% / 95% Spleen 60% / 44% / 60% 93% / 95% 53% / 45% / 63% 89% / 87% / 91% Medical Image Processing 6/12/2021 27

Tissue Classification: Matlab Demo Medical Image Processing 6/12/2021 28

Publications (CBIR) [1] Daniela Stan and Ishwar K. Sethi, “Image Retrieval using a Hierarchy of Clusters” in Lecture Notes in Computer Science: Advances in Pattern Recognition – ICAPR 2001, Springer-Verlag Ltd. (Ed), pp. 377 -388, 2001. [2] Daniela Stan and Ishwar K. Sethi, “Mapping Low-level Image Features to Semantic Concepts” in Proceedings of SPIE: Storage and Retrieval for Media databases, pp. 172 -179, 2001. [3] Ishwar K. Sethi, Ioana Coman, Daniela Stan, “Mining Association Rules between Low-level Image Features and High-level Concepts” in Proceedings of SPIE: Data Mining and Knowledge Discovery III, pp. 279 -290, 2001. [4] Daniela Stan and Ishwar K. Sethi, “Color Patterns for Pictorial Content Description”, ACM Symposium on Applied Computing, 2002. [5] Daniela Stan and Ishwar K. Sethi, “e. ID: A System for Exploration of Image Databases”, Information Processing and Management Journal, 2002. [6] Daniela Stan and Ishwar K. Sethi, “Synobins: An intermediate level towards Annotation and Semantic Retrieval”, IEEE Trans. Multimedia Journal. Medical Image Processing 6/12/2021 29

Publications (MI) [1] D. Xu, J. Lee, D. S. Raicu, J. D. Furst, D. Channin. "Texture Classification of Normal Tissues in Computed Tomography", The 2005 Annual Meeting of the Society for Computer Applications in Radiology, June 2 -5, 2005. (Submitted) [2] D. S. Raicu, W. Tsang, M. Kalinin, D. Xu, J. D. Furst, D. Channin. "Automatic Tissue Context Determination in Computed Tomography", SPIE Medical Imaging, February 12– 17, 2005. (Submitted) [3] D. H. Xu, A. Kurani, J. D. Furst, & D. S. Raicu, "Run-length encoding for volumetric texture", The 4 th IASTED International Conference on Visualization, Imaging, and Image Processing - VIIP 2004, Marbella, Spain, September 6 -8, 2004. [4] D. Channin, D. S. Raicu, J. D. Furst, D. H. Xu, L. Lilly, C. Limpsangsri, "Classification of Tissues in Computed Tomography using Decision Trees", Poster and Demo, The 90 th Scientific Assembly and Annual Meeting of Radiology Society of North America (RSNA 04), November 28, 2004. [5] A. Kurani, D. H. Xu, J. D. Furst, & D. S. Raicu, "Co-occurrence matrices for volumetric data", The 7 th IASTED International Conference on Computer Graphics and Imaging – CGIM, August 16 -18, 2004. [6] D. S. Raicu, J. D. Furst, D. Channin, D. H. Xu, & A. Kurani, "A Texture Dictionary for Human Organs Tissues' Classification", Proceedings of the 8 th World Multiconference on Systemics, Cybernetics and Informatics (SCI 2004), July 18 -21, 2004. Medical Image Processing 6/12/2021 30

Daniela Raicu Intelligent Multimedia Processing Laboratory School of CTI De. Paul University THE END! Medical Image Processing 6/12/2021 31