CSci 6971 Image Registration Lecture 8 Registration Components

CSci 6971: Image Registration Lecture 8: Registration Components February 6, 2004 Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware Image Registration Lecture 8

Image Metrics How similar is image A to image B ? Image Registration Lecture 8 2

Image Metrics Does Image B matches Image A better than Image C ? Image Registration Lecture 8 3

Image Metrics Match( A , B ) Image B Image Registration > < Image A Lecture 8 Match( A , C ) Image C 4

Image Metrics Match( A , B ) Simplest Metric Mean Squared Differences Image Registration Lecture 8 5

Mean Squared Differences For each pixel in A Image B Difference( index ) = A( index ) – B( index ) Sum += Difference( index ) 2 Match( A , B ) = Sum / number. Of. Pixels Image Registration Lecture 8 6

For each pixel in the Fixed Image j j Fixed Image Grid i Moving Image Grid y y’ x Space Transform Fixed Image Physical Coordinates Image Registration i x’ Moving Image Physical Coordinates Lecture 8 7

Image Metrics Fixed Image Valu e Metric Moving Image Interpolator Transform Image Registration Lecture 8 Parameters 8

Mean Squared Differences #include "itk. Image. h" "itk. Mean. Squares. Image. To. Image. Metric. h" "itk. Linear. Interpolate. Image. Function. h" "itk. Translation. Transform. h" typedef itk: : Image< char, 2 > Image. Type; Image. Type: : Const. Pointer fixed. Image = Get. Fixed. Image(); Image. Type: : Const. Pointer moving. Image = Get. Moving. Image(); typedef itk: : Linear. Interpolate. Image. Function< Image. Type, double > Interpolator. Type; Interpolator. Type: : Pointerpolator = Interpolator. Type: : New(); typedef itk: : Translation. Transform< double, 2 > Transform. Type; Transform. Type: : Pointer transform = Transform. Type: : New(); Image Registration Lecture 8 9

Mean Squared Differences typedef itk: : Mean. Squares. Image. To. Image. Metric< Image. Type, Image. Type > Metric. Type; Metric. Type: : Pointer metric = Metric. Type: : New(); metric->Set. Interpolator( interpolator ); metric->Set. Transform( transform ); metric->Set. Fixed. Image( fixed. Image ); metric->Set. Moving. Image( moving. Image ); Metric. Type: : Transform. Parameters. Type translation( Dimension ); translation[0] = 12; translation[1] = 27; double value = metric->Get. Value( translation ); Image Registration Lecture 8 10

Mean Squared Differences Metric. Type: : Transform. Parameters. Type translation( Dimension ); double value[21]; for( int dx = 0; dx <= 20; dx++) { for( int dy = 0; dy <= 20; dy++) { translation[0] = dx; translation[1] = dy; value[dx][dy] = metric->Get. Value( translation ); } } Image Registration Lecture 8 11

Evaluating many matches y y Transform x x Fixed Image Registration Moving Image Lecture 8 12

Registration Plotting the Me Mean Squared Differences Transform Parametric Space Lecture 8

ge Registration Plotting the Me Mean Squared Differences Transform Parametric Space Lecture 8

Evaluating many matches y y Transform (-15, -25) mm x x Fixed Image Registration Moving Image Lecture 8 15

Plotting the Metric Mean Squared Differences Transform Parametric Space Image Registration Lecture 8 16

e Registration Plotting the Me Mean Squared Differences Transform Parametric Space Lecture 8

The Best Transform Parameters Evaluation of the full parameter space is equivalent to perform optimization by exhaustive search Image Registration Lecture 8 18

The Best Transform Parameters Very Safe but Very Slow Image Registration Lecture 8 19

The Best Transform Parameters Better Optimization Methods for example Gradient Descent Image Registration Lecture 8 20

Image Registration Framework Fixed Image Metric Moving Image Registration Interpolator Optimizer Transform Parameters Lecture 8 21

Gradient Descent Optimizer f( x , y ) S = Step L = Learning Rate Image Registration ∆ G( x , y ) = f( x , y ) S = L ∙ G( x , y ) Lecture 8 22

Gradient Descent Optimizer f( x , y ) Image Registration ∆ G( x , y ) = f( x , y ) S = L ∙ G( x , y ) Lecture 8 23

Gradient Descent Optimizer f( x , y ) L too large Image Registration ∆ G( x , y ) = f( x , y ) S = L ∙ G( x , y ) Lecture 8 24

Gradient Descent Optimizer f( x , y ) L too small Image Registration ∆ G( x , y ) = f( x , y ) S = L ∙ G( x , y ) Lecture 8 25

Gradient Descent Optimizer What’s wrong with this algorithm ? Image Registration Lecture 8 26

Gradient Descent Optimizer S Units ? f(x, y) Units ? G(x, y) Units ? = millimeters = intensity / millimeters S = L ∙ G( x , y ) L Units ? Image Registration = millimeters 2 / intensity Lecture 8 27

Gradient Descent Optimizer S = L ∙ G( x ) f( x ) 1 1 Image Registration Lecture 8 28

Gradient Descent Optimizer S = L ∙ G( x ) f( x ) S = Large in high gradients S = Small in low gradients 1 1 Image Registration Lecture 8 29

Gradient Descent Optimizer f( x ) S = L ∙ G( x ) Works great with this function Works badly with this function Image Registration Lecture 8 30

Gradient Descent Variant Driving Safe ! Image Registration Lecture 8 31

Regular Step Gradient Descent f( x ) ^ S = D ∙ G( x ) If G changes direction Di = Di-1 / 2. 0 D 1 D 2 D 1 Image Registration Lecture 8 32

Regular Step Gradient Descent f( x ) ^ S = D ∙ G( x ) User Selects D 1 User Selects Dstop User Selects Gstop D 1 D 2 D 1 Image Registration Lecture 8 33

Optimizers are like a car Watch while you are driving ! Image Registration Lecture 8 34

Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15 mm, -25 mm) Image Registration Lecture 8 35

Plotting the Optimizer’s P Mean Squared Differences Image Registration Step Length = 1. 0 mm Lecture 8 36

Plotting the Optimizer’s P Mean Squared Differences Image Registration Step Length = 2. 0 mm Lecture 8 37

Plotting the Optimizer’s P Mean Squared Differences Image Registration Step Length = 5. 0 mm Lecture 8 38

Plotting the Optimizer’s P Mean Squared Differences Image Registration Step Length = 10. 0 mm Lecture 8 39

Plotting the Optimizer’s P Mean Squared Differences Image Registration Step Length = 20. 0 mm Lecture 8 40

Plotting the Optimizer’s P Mean Squared Differences Image Registration Step Length = 40. 0 mm Lecture 8 41

Watch over your optimizer Example: Optimizer registering an image shifted by (-15 mm, -25 mm) The optimizer starts at (0 mm, 0 mm) Image Registration Lecture 8 42

Image Registration Mean Squared Differences Step Length = 1. 0 mm Lecture 8 43

Image Registration Mean Squared Differences Step Length = 2. 0 mm Lecture 8 44

Image Registration Mean Squared Differences Step Length = 5. 0 mm Lecture 8 45

Image Registration Mean Squared Differences Step Length = 10. 0 mm Lecture 8 46

Image Registration Mean Squared Differences Step Length = 20. 0 mm Lecture 8 47

Image Registration Mean Squared Differences Step Length = 40. 0 mm Lecture 8 48

Other Image Metrics Normalized Correlation Image Registration Lecture 8 49

Normalized Correlation For each pixel in A Image B Sum. AB += A( index ) ∙ B( index ) Sum. AA += A( index ) ∙ A( index ) Sum. BB += B( index ) ∙ B( index ) Match( A , B ) = Sum. AB / √ ( Sum. AA ∙ Sum. BB ) Image Registration Lecture 8 50

Evaluating many matches y y Transform x x Fixed Image Registration Moving Image Lecture 8 51

Registration Plotting the Metric Normalized Correlation Metric Lecture 8 Transform Parametric Space

ge Registration Plotting the Metric Normalized Correlation Metric Lecture 8 Transform Parametric Space

Normalized Correlation Metric #include "itk. Image. h" "itk. Normalized. Correlation. Image. To. Image. Metric. h" "itk. Linear. Interpolate. Image. Function. h" "itk. Translation. Transform. h" typedef itk: : Image< char, 2 > Image. Type; Image. Type: : Const. Pointer fixed. Image = Get. Fixed. Image(); Image. Type: : Const. Pointer moving. Image = Get. Moving. Image(); typedef itk: : Linear. Interpolate. Image. Function< Image. Type, double > Interpolator. Type; Interpolator. Type: : Pointerpolator = Interpolator. Type: : New(); typedef itk: : Translation. Transform< double, 2 > Transform. Type; Transform. Type: : Pointer transform = Transform. Type: : New(); Image Registration Lecture 8 54

Normalized Correlation Metric typedef itk: : Normalized. Correlation. Image. To. Image. Metric< Image. Type, Image. Type > Metric. Type; Metric. Type: : Pointer metric = Metric. Type: : New(); metric->Set. Interpolator( interpolator ); metric->Set. Transform( transform ); metric->Set. Fixed. Image( fixed. Image ); metric->Set. Moving. Image( moving. Image ); Metric. Type: : Transform. Parameters. Type translation( Dimension ); translation[0] = 12; translation[1] = 27; double value = metric->Get. Value( translation ); Image Registration Lecture 8 55

Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15 mm, -25 mm) Image Registration Lecture 8 56

Plotting the Optimizer’s P Image Registration Normalized Correlation Metric Step Length = 1. 0 mm Lecture 8 57

Plotting the Optimizer’s P Image Registration Normalized Correlation Metric Step Length = 2. 0 mm Lecture 8 58

Plotting the Optimizer’s P Image Registration Normalized Correlation Metric Step Length = 5. 0 mm Lecture 8 59

Plotting the Optimizer’s P Image Registration Normalized Correlation Metric Step Length = 10. 0 mm Lecture 8 60

Plotting the Optimizer’s P Image Registration Normalized Correlation Metric Step Length = 20. 0 mm Lecture 8 61

Plotting the Optimizer’s P Image Registration Normalized Correlation Metric Step Length = 40. 0 mm Lecture 8 62

Evaluating many matches y y Transform (-15, -25) mm x x Fixed Image Registration Moving Image Lecture 8 63

Registration Plotting the Metric Normalized Correlation Metric Lecture 8 Transform Parametric Space

e Registration Plotting the Metric Normalized Correlation Metric Lecture 8 Transform Parametric Space

Watch over your optimizer Example: Optimizer registering an image shifted by (-15 mm, -25 mm) The optimizer starts at (0 mm, 0 mm) Image Registration Lecture 8 66

Plotting the Optimizer P Normalized Correlation Metric Image Registration Step Length = 1. 0 mm Lecture 8 67

Plotting the Optimizer P Normalized Correlation Metric Image Registration Step Length = 2. 0 mm Lecture 8 68

Plotting the Optimizer P Normalized Correlation Metric Image Registration Step Length = 5. 0 mm Lecture 8 69

Plotting the Optimizer P Normalized Correlation Metric Image Registration Step Length = 10. 0 mm Lecture 8 70

Plotting the Optimizer P Normalized Correlation Metric Image Registration Step Length = 20. 0 mm Lecture 8 71

Plotting the Optimizer P Normalized Correlation Metric Image Registration Step Length = 40. 0 mm Lecture 8 72

Watch over your optimizer How much can you blindly trust your optimizer… Image Registration Lecture 8 73

I left my Optimizer working… Image Registration Lecture 8 74

I think it’s going to finish soon… Image Registration Lecture 8 75

End Enjoy ITK ! Image Registration Lecture 8 76