Chair for Computer Aided Medical Procedures Augmented Reality

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu Automatic Feature Generation for Endoscopic Image Classification Ulrich Klank Supervisor: Nicolas Padoy Advisor: Prof. Nassir Navab Chair for Computer Aided Medical Procedures & Augmented Reality 18 January 2007 Department of Computer Science | Technische Universität München

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu Overview Endoscopic images Differences Similarities Image feature generation using Genetic Programming A low level approach A high level approach An example CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 2

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu Endoscopic Images of two Surgical Phases: OP 3 Images from the cutting and clipping phase (OP 3) CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 Images from the bag retraction phase (OP 3) 3

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu …: OP 1 Images from the cutting and clipping phase (OP 1) CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 Images from the bag retraction phase (OP 1) 4

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu Low-level Approach - Short Reminder Genetic Programming: combination of low-level operators FOR PIXEL (With Parameters) ADD MUL Code PUSH LOAD (With Parameters) … Evaluation Mutation CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 5

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu Low-level Approach - Results Distributed evaluation of programs on several computers (up to 7) Nearly 10. 000 programs evaluated (~300 generations) First results: Characteristics of the best programs: returning a short vector in a short time Classification rate with a linear classifier is 62% (64 images of 2 phases of 4 videos) CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 6

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu Problems with the low-level Approach ~40% of the programs contain major errors like Infinite running time, stack overflow No reference to the input image Resulting programs still has structural similarity to the initial program. More generations needed Evaluation of a programs is very slow due to the simulation of basic instructions on images How to improve this method? CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 7

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu Existing Software: GENIE Software published by: Los Alamos National Laboratories First publication ’ 97, Commercial version in development Genetic Programming for segmentation of images Application example: Segmentation of Medical Images Using a Genetic Algorithm by Payel Ghosh, Melanie Mitchell (’ 06) CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 8

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu The Step to a higher level approach Replace the basic commands in a program by higher level operators: Examples for low-level operator : FOR PIXEL (With Parameters) ADD MUL PUSH LOAD (With Parameters) … Examples for high-level operator : Original Erode Dilate (With Parameters) Canny. Edge Histogram Gradient x (With Parameters) CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 (With Parameters) Min. Loc 9

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu Evaluation of a Program Semantic checks Input referred? No infinite loops? Execution with several inputs 16 images per phase 2 phases per video at the moment 4 videos used for evaluation A fitness function with 2 components: A classification of the phases by the output vectors The average execution time per input CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 10

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu Genetic Concept – Cross Over Program 1 Original Erode Dilate (With Parameters) Canny. Edge Histogram Gradient x (With Parameters) Min. Loc New Program 2 Original Gauss (With Parameters) Histogram Push. Image (With Parameters) CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 Original Gradient y (With Parameters) Max 11

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu High-level Approach Benefits Faster evaluation Reduced number of commands Optimized basic image operations (Open. CV) Resulting programs easier to understand CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 12

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu Current Results: Running time: 480 ms (in simulation) Output length: a vector of 48 signed integer Classification rate: 67% Rate based on 512 testing images out of 4 videos and 2 phases Number of generations needed: ~80 CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 13

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu Future Work Completion of the available image operators Extension to multi-phases classification Comparison of the fitness function with a standard classifier Comparison with several standard features Features evaluation within the workflow segmentation system CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 14

Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab. cs. tum. edu Thank you for your attention! CAMP | Department of Computer Science | Technische Universität München | 07 October 2020 15