Neural Network Segmentation and Validation Nicole M Grosland

Neural Network Segmentation and Validation Nicole M. Grosland Vincent A. Magnotta

Objective • To develop tools to automate bony structure mesh definitions suitable for patient-specific finite element contact analyses. – Further, automate the identification of the structures of the upper extremity (including hand/fingers, wrist, elbow and shoulder) using a neural network.

Specific Aims • Aim 1: Integrate and enhance a set of novel and robust hexahedral mesh generation algorithms into the NA-MIC toolkit. • Aim 2: Further automate these modeling capabilities by developing tools for automated image region identification via neural networks. • Aim 3: Validate geometry of models using cadaveric specimens and three-dimensional surface scans

Imaging Protocol • 15 cadaveric specimens were acquired and imaged • CT images, Siemens Sensation 64 CT scanner (matrix = 512 x 512, KVP = 120). – 0. 34 -mm in-plane resolution – 0. 4 mm slice thickness • MR images: Siemens 3 T Trio scanner – PD weighted images – 2 D FSE • • TE=12 ms, TR=7060 ms Resolution=0. 5 x 0. 5 mm Slice Thickness = 1. 0 mm Matrix=512 x 512 – T 1 weighted images – 3 D MP-RAGE • • TE=3. 35 ms, TR=2530 ms, TI=1100 ms Resolution=0. 6 x 0. 5 mm Matrix=384 x 96 Post-processing via BRAINS 2 – Spatially normalized – Resampled to 0. 2 -mm 3 voxels

Manual Segmentation • Two trained technicians (Tracer 1 and Tracer 2) manually traced twenty-one phalanx bones (index) – the distal, middle, and proximal bones • Relative overlap: • Records maintained of tracing times

Neural Network Data Probability Map Values Spherical Coordinates Area Iris Values Input Vector: {PS 1, PS 2, Sα, Sβ, Sγ, G-4, … G 4, A 1, … A 12} Mask Values Output Vector: {MS 1, MS 2} Gradient Values

Neural Network Configuration Input Layer Hidden Layer Output Layer test Calculated Error Backpropagation

Neural Network Training • 10 subjects used to train the neural network – Subjects all registered to atlas dataset – Manual segmentations used to define probability information – 200, 000 input vectors x 250 iterations • 5 subjects used to evaluate validity and reliability of network

3 D Laser Scanner • 3 D Laser scanners have been used for rapid prototyping and to non- destructively image ancient artifacts • Roland LPX-250 Scanner Obtained – Planar and rotary scanning modes – 0. 008 inch resolution in planar mode – Objects up to 10 inches wide and 16 inches tall can be scanned – Reverse modeling software tools

LPX-250 Laser Scanner

Finger Dissection • Phalanx and metacarpal bones removed – Care taken to avoid tool marks on the bones • De-fleshing process outlined by Donahue et al (2002) was utilized – Bones allowed to soak in a 5. 25% sodium hypochlorite (bleach) solution for 6 hours • Degreased via a soapy water solution • Thin layer of white primer was used to coat the bony surfaces

CA 05042125 L MD 05010306 R MD 05042226 L SC 05030303 R

Registration of Surfaces • Surface scans origin shifted to center of mass and reoriented to have the same orientation as the CT data • Surfaces registered using a rigid iterative closest point algorithm • Compute Euclidean distance between the surfaces

Specimen CA 05042125 L a b c d Manual (red) and ANN (blue) ROI definitions

Manual Segmentation • Relative overlap (Tracer 1 and Tracer 2) – 0. 89 for the three bones. – Individual bones • Proximal – 0. 91 • Middle – 0. 90 • Distal bones – 0. 87 • The average time required to manually segment the bones of the index finger was 50. 9 minutes, ranging from 39 to 63 minutes.

ANN Results Compared to Manual Rater Relative Overlap of Manual and Neural Network Segmentation Subject ID Proxim al Overla p Middl e Overl ap Distal Index Overla Finger p Overl ap CA 05042 124 R 0. 91 0. 79 0. 83 CA 05042 125 L 0. 91 0. 88 0. 84 0. 87 MD 05021 815 R 0. 85 0. 83 0. 78 0. 82 MD 05042 226 L 0. 86 0. 81 0. 68 0. 79 SC 05030 303 R 0. 84 0. 78 0. 72 0. 78

Example Distance Maps ANN output & 3 D physical surface scans

ANN Validation ANN output & 3 D physical surface scans Subject ID Proximal Phalanx (mm) Middle Phalanx (mm) Distal Phalanx (mm) Finger Average (mm) CA 05042125 L 0. 23 0. 12 0. 17 MD 05021815 R 0. 18 0. 16 0. 17 MD 05042226 L 0. 35 0. 27 0. 97 0. 53 SC 05030303 R 0. 26 0. 17 0. 20 0. 21 Bone Average 0. 26 0. 18 0. 38

Conclusion • Neural networks provide a promising automated segmentation tool for identifying bony regions of interest • Output was compared to both manual raters and 3 D surface scanning – Error was less than the size of 1 voxel • Use of 3 D surface scanning provides a means to have a true gold standard for evaluation of automated segmentation algorithms

Acknowledgements • Grant funding – R 21 (EB 001501) – R 01 (EB 005973) • Stephanie Powell, Nicole Kallemeyn, Nicole De. Vries, Esther Gassman

Validation • Aim 3: Model Validation: Cadaveric specimens will be used (i) to generate three-dimensional surface scans with which surfaces defined both manually and via the automated neural network will be compared and (ii) to directly validate the computational models developed via the automated meshing algorithms.

Validation • True “gold-standard” often very difficult to achieve – Brain imaging often have to live with manual raters – Established guidelines based on anatomical experts • Are there better “gold-standards” for other regions of the body?

Orthopaedic Imaging • Ideas developed out of goal to automate the definition of bony regions of interest. • How can we validate these automated tools? • Orthopaedic applications: Would it be possible to dissect cadaveric specimens? – Use bony specimen as the “gold-standard”

Surface Comparison Proximal Middle • Physical surface scan (white) • Manually segmented surface (blue) Average Distance (mm) Maximum Distance (mm) Proximal 0. 27 1. 7 Middle 0. 27 1. 3 Distal 0. 30 1. 4 Distal

Manual surface definitions with various degrees of smoothing (a) Unsmoothed, (b) Image-based smoothing, & (c) Laplacian surface-based smoothing. a b c

Average Euclidean distance and standard deviation between the manually traced unsmoothed surfaces and the physical surface scans. Finger ID Proximal Phalanx (mm) Middle phalanx (mm) Distal phalanx (mm) Finger Average (mm) CA 05042125 L 0. 19 (0. 43) 0. 12 (0. 16) 0. 15 (0. 19) 0. 15 MD 05010306 R 0. 04 (0. 08) 0. 06 (0. 09) 0. 05 (0. 07) 0. 05 MD 05021815 R 0. 05 (0. 07) 0. 06 (0. 08) 0. 05 MD 05042226 L 0. 21 (0. 32) 0. 24 (0. 35) 0. 21 (0. 29) 0. 22 SC 05030303 R 0. 20 (0. 35) 0. 10 (0. 18) 0. 13 Bone Average 0. 14 0. 11

Average Euclidean distance and standard deviation between the surfaces generated via image-based smoothing and the physical surface scans. Finger ID Proximal Phalanx (mm) Middle phalanx (mm) Distal phalanx (mm) Finger Average (mm) CA 05042125 L 0. 32 (0. 31) 0. 27 (0. 19) 0. 32 (0. 25) 0. 31 MD 05010306 R 0. 16 (0. 15) 0. 24 (0. 19) 0. 28 (0. 23) 0. 22 MD 05021815 R 0. 14 (0. 10) 0. 18 (0. 14) 0. 19 (0. 14) 0. 17 MD 05042226 L 0. 38 (0. 30) 0. 41 (0. 35) 0. 42 (0. 37) 0. 40 SC 05030303 R 0. 36 (0. 31) 0. 24 (0. 20) 0. 27 (0. 26) 0. 21 Bone Average 0. 27 0. 30

Average Euclidean distance and standard deviations between the surfaces generated via Laplacian surfacebased smoothing and the physical surfaces. Finger ID Proximal Phalanx (mm) Middle phalanx (mm) Distal phalanx (mm) Finger Average (mm) CA 05042125 L 0. 18 (0. 48) 0. 10 (0. 15) 0. 21 (0. 39) 0. 16 MD 05010306 R 0. 09 (0. 15) 0. 17 (0. 22) 0. 25 (0. 37) 0. 17 MD 05021815 R 0. 03 (0. 06) 0. 08 (0. 13) 0. 06 (0. 09) 0. 06 MD 05042226 L 0. 22 (0. 37) 0. 32 (0. 54) 0. 37 (0. 67) 0. 30 SC 05030303 R 0. 26 (0. 45) 0. 11 (0. 17) 0. 21 (0. 41) 0. 19 Bone Average 0. 16 0. 22 Average Euclidean distance and standard deviations between the surfaces generated via Laplacian surface-based smoothing and the physical surfaces.

a b c

Neural Networks • A computing paradigm that is designed to modeled how the brain processes data • The network consists of several interconnected neurons that process that the input information through and activation function to form an output • What information can be used to segment regions of interest from images