I R SNApp Image Reconstruction and Segmentation for

I. R. SNApp Image Reconstruction and Segmentation for Neurosurgery Applications May 09 -10 Aaron Logan Dylan Reid William Lim Kyungchul Song THANK YOU: Dr. Namrata Vaswani, Chenlu Qiu, Dr. Greg Smith, Bill Zimmerman

Project Planning © 2008, I. R. SNApp – all rights reserved. 2

Problem Statement & Client Needs q Problem: need real-time MRI m Current MRI is slow: many measurements Ø Slow acquisition (10’s of minutes scan time) Ø Slow reconstruction (Fourier/k-space spatial domain) m Slow and inefficient segmentation (image analysis) algorithms q Solution/ Client Needs: m MR image reconstruction algorithm using Compressed Sensing Ø New technique in statistical signal processing/ estimation theory Ø Permits signal estimation using fewer observations/ measurements – Result is faster acquisition & less computation faster image reconstruction Ø Goal is real-time m Develop algorithm for sequential image segmentation Ø Sequentially segment deforming objects or ROI's from images Ø Utilize prior knowledge about shape change dynamics to segment noisy/low contrast imagery. Use only current and past images. Ø Goal is real-time © 2008, I. R. SNApp – all rights reserved. 3

Requirements Specification System Description & Diagrams q High-Level System Block Diagram Raw MR data Image Reconstruction Reconstructed MR image Image Segmentation © 2008, I. R. SNApp – all rights reserved. Reconstructed & segmented MR image 4

Requirements Specification System Description & Diagrams q Image Reconstruction Block Diagrams KFCS LSCS © 2008, I. R. SNApp – all rights reserved. 5

Requirements Specification System Description & Diagrams q Image Segmentation Block Diagram Reconstructed MR Image Manual ROI Selection – Algorithm Seed MATLAB Segmentation Code © 2008, I. R. SNApp – all rights reserved. Reconstructed & segmented Additional MR image Manual Tracing 6

Requirements Specification System Description & Diagrams q System GUI Sketches © 2008, I. R. SNApp – all rights reserved. 7

Requirements Specification Operating Environment & User Interface q Operating Environment m High-performance computing Ø Lots of memory Ø High speed Ø Future work on this project may require parallel processing Ø Linux application servers meet these requirements m User interacts with algorithm primarily through scripts and command line interfaces q User Interface m Command prompt windows Ø MATLAB command window Ø SSH access to Linux applications servers m Simple GUI for segmentation Ø User input to choose ROI location to seed algorithm Ø Allow user to select options Ø Visual image processing feedback © 2008, I. R. SNApp – all rights reserved. 8

Requirements Specification Project Requirements q Functional Requirements m FR-1: Algorithms shall output correct data. m FR-2: Ported version of reconstruction algorithm shall run significantly faster than MATLAB prototype version. m FR-3: The entire image processing program shall be capable of realtime execution. q Non-functional Requirements m NFR-1: The reconstruction program shall be written in C/C++. m NFR-2: The segmentation program shall be written in MATLAB. m NFR-3: The program shall employ parallel processing. m NFR-4: The source code shall be well documented. © 2008, I. R. SNApp – all rights reserved. 9

Requirements Specification Deliverables q A complete software system that 1. 2. 3. m takes raw MRI data, reconstructs the data into images, and performs segmentation on those images. Attributes of the system: Ø C/C++ code, well documented and working Ø Executable version of the code (MEX) Ø A basic GUI to assist segmentation process q Comprehensive test results © 2008, I. R. SNApp – all rights reserved. 10

Project Plan © 2008, I. R. SNApp – all rights reserved. 11

Project Plan Resource Requirements q C/C++ IDE and compiler q MATLAB installed with m Image Processing Toolbox m Other Toolboxes as necessary q GUI development environment m MATLAB/ GUIDE? m Visual Studio? m Eclipse with add-ons? q Test data m Raw MR data and corresponding images © 2008, I. R. SNApp – all rights reserved. 12

Project Plan Risks q Segmentation m Very data-dependent Ø Image contrast, focus/resolution of objects, etc. influence final results Ø Mitigated with good image preprocessing – Contrast correction/normalization, filtering m “Good” segmentation algorithm difficult to define objectively q Reconstruction m KFCS algorithm could cause PCs to crash, resulting in data loss Ø Eliminated by testing only small images on PCs for framework testing; performing batch tests on full-size images on Linux clusters q Raw data acquisition m “Raw” MRI data only exists internal to the machine m HIPAA privacy laws may pose issue with acquisition © 2008, I. R. SNApp – all rights reserved. 13

Project Plan Work Breakdown & Schedule q Image Reconstruction: m Aaron Logan, William Lim q Image Segmentation: m Kyungchul Song, Dylan Reid Understand Problem and Project Scope Understand Theory Learn Operating Environment Concept Testing Thanksgiving Break Algorithm Development – Modules Winter Break Algorithm Development – Integration Test Algorithm Framework Learning Design and Implementation Testing Breaks Test Algorithm for Robustness and Correctness Spring Break Miscellaneous Wrap-Up Tasks © 2008, I. R. SNApp – all rights reserved. 14

Design Project © 2008, I. R. SNApp – all rights reserved. 15

Design Approach q Image Reconstruction m Specifications are determined with two goals in mind: Ø Runtime improvements Ø Use good programming style and documentation q Image segmentation: research-oriented m Experimentation with IPT functions m See what yields good segmentation results © 2008, I. R. SNApp – all rights reserved. 16

System Design Image Reconstruction © 2008, I. R. SNApp – all rights reserved. 17

System Design – Reconstruction System Requirements q Port: MATLAB C q Faster m MATLAB prototype: ~6 minutes per 64 x 64 frame Ø (Chenlu Qiu’s PC) m Initial requirement: 3 x faster q Identical output © 2008, I. R. SNApp – all rights reserved. 18

System Design – Reconstruction Functional Decomposition & System Analysis q Implement KFCS Reconstruction © 2008, I. R. SNApp – all rights reserved. 19

Engineering Specification Image Reconstruction © 2008, I. R. SNApp – all rights reserved. 20

Engineering Specification – Reconstruction Input / Output Specifications q Inputs m MR k-space (Fourier-domain measurement) data Ø Data file m Process control parameters Ø Passed to program either as input parameters or as. INI file. Ø Observation percentage Ø Others? m Input / output options and directory information q Outputs m An array representation of image Ø Primary output format, passed to image segmentation system m Uncompressed image file -. BMP Ø Optional: for visual inspection by user © 2008, I. R. SNApp – all rights reserved. 21

Engineering Specification – Reconstruction User Interface Specification q Command line interfaces m Little return on investment for creating GUI Ø Research project! Ø Focus our time on client’s needs m Simplest interface to use for operating on Linux server connection © 2008, I. R. SNApp – all rights reserved. 22

Engineering Specification – Reconstruction Hardware Specification q Small scale testing m PCs in 2046 Coover: m CPU: Intel® Pentium® D, 3. 00 GHz m RAM: 2. 00 GB @ 2. 99 GHz q Large scale testing m ECp. E Linux application servers © 2008, I. R. SNApp – all rights reserved. 23

Engineering Specification – Reconstruction Software Specification q Language m C, not C++ Ø Compiles smaller than C++ Ø Runs faster – no overhead for OOP Ø Easier to learn; simple, yet powerful q Compiler m Min. GW (Minimalist GNU for Windows) distribution of GCC (GNU Compiler Collection) Ø Implements ANSI-C 99 standard – support for complex numbers Ø Behaves identically to most common C compilers on UNIX systems – Code compiled for PCs and Linux will behave very similarly © 2008, I. R. SNApp – all rights reserved. 24

Engineering Specification – Reconstruction Software Specification q Libraries m CLAPACK (C Linear Algebra Package) m FFTW (Fast C implementation of Fourier Transform) m Discrete Wavelet Transform package Ø Still looking for a C library that can do this. Part of ongoing research q Good style m Good abstraction practices Ø Happy medium between huge, complicated functions & too many small functions that bog down performance m No compiler warnings © 2008, I. R. SNApp – all rights reserved. 25

Engineering Specification – Reconstruction Software Specification q Documentation m Function header comments /* * Function name * Thorough description of function * List of input arguments * Output * Revision History * Author & date */ m In-context explanations & clarifications y = x^2; // x squared © 2008, I. R. SNApp – all rights reserved. 26

Engineering Specification – Reconstruction Test Specification q Automated test scripts m Generate / read program input m Run C and MATLAB algorithms in succession on same input m Compare output of each algorithm by differencing m Compute statistics on differences m Scripts written in MATLAB; access C algorithms through MEX (MATLAB executable) files © 2008, I. R. SNApp – all rights reserved. 27

Engineering Specification – Reconstruction Prototyping q runsim: MATLAB reconstruction simulation m Generates test data by taking FFT of image set Ø 32 x 32 images – manageable size for testing on PC m Tests different reconstruction techniques and looks at error, runtime of each Ø Kalman Filtering Ø Compressed Sensing, without deletions Ø Full Compressed Sensing (with deletions) Ø Genie-Aided Kalman Filtering m Full compressed sensing yielded best results m Full simulation on 1. 5 GHz PC took roughly 25 minutes Ø 6 minutes for full KFCS © 2008, I. R. SNApp – all rights reserved. 28

System Design Image Segmentation

System Design – Segmentation System Requirements q Develop segmentation algorithm in MATLAB q Achieve a contour that is correct m Includes only the ROI, no extraneous / artifact regions © 2008, I. R. SNApp – all rights reserved. 30

System Design – Segmentation System Requirements q Achieve a contour that is correct (cont. ) m Correct by visual inspection m Good contour with fine width, fine precision Ø ~1 pixel width (8 -connected) right wrong q Runtime upper bound: 1 sec/frame © 2008, I. R. SNApp – all rights reserved. 31

System Design – Segmentation System Analysis q Input: MR images of brain or other organ of interest q Automatically segments images q Goal of segmentation: m Simplify and change the representation of an image into something that is more meaningful and easier to analyze m Extract exact location of regions of interest within image q Output: the segmented MR image © 2008, I. R. SNApp – all rights reserved. 32

System Design – Segmentation Functional Decomposition Outputs: Display montage of original images Expanded on Next Slide Input: MR images Contour extraction Draw contour on original image Save segmented image in DICOM format Increment i until i = no. input images Display montage of segmented images Text file of generated files and image run info © 2008, I. R. SNApp – all rights reserved. 33

System Design – Segmentation Functional Decomposition Contour Extraction Read ith image Threshold (convert to B/W) Fill interior gaps Detect edges Smooth the object Dilate the image Output the contour of object © 2008, I. R. SNApp – all rights reserved. 34

Engineering Specification Image Segmentation © 2008, I. R. SNApp – all rights reserved. 35

Engineering Specification – Segmentation Input / Output Specification q Input: m MR Image sequence Ø File format: DICOM (Digital Imaging and Communications in Medicine)* Ø Size: 256 × 256 pixels* *NOTE: not limited to these Ø Grayscale m Directory information m Indices of images to process q Output: m MATLAB display: Ø the current segmented image Ø Display montage of the original and segmented images Ø a list of the image files that have been segmented m File output: Ø DICOM file of segmented image (original + contours overlaid) Ø CSV file of the contour pixel coordinates Ø Text file listing all the image files created, and execution results. © 2008, I. R. SNApp – all rights reserved. 36

Engineering Specification – Segmentation User Interface Specification q MATLAB Command Window q Segmentation GUI m File list, current segmented image, montages of original/ segmented images m Run and stop buttons; pertinent menu bar items m Prompts user with multiple dialog boxes Ø Input & Output Directories Ø Segmentation options © 2008, I. R. SNApp – all rights reserved. 37

Engineering Specification – Segmentation Hardware and Software Specification q Hardware Specification m Must achieve 1 sec/frame with the following: Ø CPU: Intel® Pentium® D, 3. 00 GHz Ø RAM: 2. 00 GB @ 2. 99 GHz Ø PCs in 2046 Coover meet this spec q Software Specification m MATLAB Version 7. 6 (R 2008 a) m Image Processing Toolbox, Version 6. 1 m Others: discussed for reconstruction Ø Documentation practices Ø Good data abstraction Ø No MATLAB Interpreter warnings © 2008, I. R. SNApp – all rights reserved. 38

Engineering Specification – Segmentation Test Specification q Visual inspection of output contour m Simple and most robust method of checking that output meets requirements q Speed (throughput time) of the system m Measured using the timing functions available with MATLAB Ø tic and toc stopwatch functions Ø clock Ø MATLAB Profiler © 2008, I. R. SNApp – all rights reserved. 39

Engineering Specification – Segmentation Prototype Segmentation Algorithm (1/8) q Step 1: Read Image m Original image m dicomread © 2008, I. R. SNApp – all rights reserved. 40

Engineering Specification – Segmentation Prototype Segmentation Algorithm (2/8) q Step 2: Threshold Image m Threshold the grayscale image to black and white m im 2 bw © 2008, I. R. SNApp – all rights reserved. 41

Engineering Specification – Segmentation Prototype Segmentation Algorithm (3/8) q Step 3: Edge Detection m Detect the edges with binary gradient mask m edge, sobel m Does not yield a good contour Ø need the next steps to improve © 2008, I. R. SNApp – all rights reserved. 42

Engineering Specification – Segmentation Prototype Segmentation Algorithm (4/8) q Step 4: Image Contour Dilation m Dilate the objects in the image using dilated gradient mask m imdilate, strel © 2008, I. R. SNApp – all rights reserved. 43

Engineering Specification – Segmentation Prototype Segmentation Algorithm (5/8) q Step 5: Hole Filling m Holes within objects are filled m imfill © 2008, I. R. SNApp – all rights reserved. 44

Engineering Specification – Segmentation Prototype Segmentation Algorithm (6/8) q Step 6: Object Boundary Smoothening by Erosion m Smoothen the white pixel boundaries using erosion m imerode, strel © 2008, I. R. SNApp – all rights reserved. 45

Engineering Specification – Segmentation Prototype Segmentation Algorithm (7/8) q Step 7: Extraction of Contours m Extract the contour pixels from the object boundaries m bwperim © 2008, I. R. SNApp – all rights reserved. 46

Engineering Specification – Segmentation Prototype Segmentation Algorithm (8/8) q Step 8: Final Segmented Image m Overlay extracted contours on original image © 2008, I. R. SNApp – all rights reserved. 47

Project Design Summary q Implementation already in progress q Project risks: all can be eliminated or mitigated q Project feasibility m Low cost – all resources, except for test data, already available m Test data could be obtained as part of research agreement q Project success would help advance field of medical imaging m Project has been scoped well within our means © 2008, I. R. SNApp – all rights reserved. 48

Questions and Discussion © 2008, I. R. SNApp – all rights reserved. 49

Wrap-up q Required and requested changes q Action items © 2008, I. R. SNApp – all rights reserved. 50