C elegans RealTime Egg Detection Jay Silver GIT

The Dream – Automated Behavior Analysis l In goes the worm, out come the

Real Goals Track a worm as it crawls on its Petri dish l Analyze

First: Need Clear Images >3 m 0 0 L Auto gain control: off Manual

Analyzing Images Numerically Images and video have many dimensions l Reduce dimensions (math and

Image Analysis Overview GOAL: Go from original grayscale image to analyzed black and white

Threshold, Label, and Floodfill Threshold e. g. Label and Count Area e. g. Area

Thin to a Skeleton and Prune Structuring element Prune: Maximal distance criterion using graphs,

Keep Head and Tail Consistent If(std. Dev(Fwdi) > std. Dev(Revi)) {reverse. Direction. Of. Backbone();

Outline of Worm (White lines and annotations drawn by algorithm, other colors drawn by

Extract Curvature from Outline Use as template While(traversing. Outline. Pixels){ M=Slope. Between. Pixels(i, i+10);

Match Curvature with Egg Port (Left) Side Starboard (Right) Side Template MATCH (skewed to

Make Decisions BU FFE RH EG RIG HT G RE GI ON RI GH

Future Improvements l l l Convert code to run on W 2 K and

Slides: 14

Download presentation

C. elegans Real-Time Egg Detection Jay Silver: GIT, INTEL MRL, & UCSD 1

The Dream – Automated Behavior Analysis l In goes the worm, out come the parameters 1 mm long ~302 neurons Black Box Caenorhabditis elegans Measurements of Behavior Jay Silver: GIT, INTEL MRL, & UCSD 2

Real Goals Track a worm as it crawls on its Petri dish l Analyze its behavior, especially egg laying, in real-time l Theory Practice Jay Silver: GIT, INTEL MRL, & UCSD 3

First: Need Clear Images >3 m 0 0 L Auto gain control: off Manual shutter control: off OP 50 Light source: 6 out of 10 Mirror: Adjusted for orthog. 50 m light incidence L Wait time: < 1 hour Jay Silver: GIT, INTEL MRL, & UCSD 4

Analyzing Images Numerically Images and video have many dimensions l Reduce dimensions (math and morphology) l Make decisions (probability and heuristics) l Starboard Port Head Curvature Starboard Curvature Outline Columns sit y Images In ten Rows Max The many dimensions of grayscale video Port Outline Centroid Covariances (Features are drawn by algorithm, annotations are drawn by hand) Jay Silver: GIT, INTEL MRL, & UCSD 5

Image Analysis Overview GOAL: Go from original grayscale image to analyzed black and white image. Identify egg if present in black and white image. (Record the time of each egg laying event for statistical parameterization) Jay Silver: GIT, INTEL MRL, & UCSD 6

Threshold, Label, and Floodfill Threshold e. g. Label and Count Area e. g. Area = 8 If (gray. Scale. Value > mean + 2*s), Pixel = 1; Else, Pixel = 0; 25 40 18 7 11 13 247 254 210 5 190 18 36 50 225 12 255 20 28 13 5 32 13 It’s a “K” 0 0 0 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 0 1 1 0 0 0 0 1 1 0 0 0 0 0 1 1 1 1 Area = 4 0 0 0 0 0 2 2 0 0 0 3 3 0 0 0 0 0 Area = 5 Actual Threshold and Invert Label and Fill Holes within Worm Floodfill all objects smaller than worm Jay Silver: GIT, INTEL MRL, & UCSD 7

Thin to a Skeleton and Prune Structuring element Prune: Maximal distance criterion using graphs, or iteratively remove points that have one neighbor Erosion: each. Pixel=min{pixel. Neigborhood} Jay Silver: GIT, INTEL MRL, & UCSD 8

Keep Head and Tail Consistent If(std. Dev(Fwdi) > std. Dev(Revi)) {reverse. Direction. Of. Backbone(); } Fwdi Fwd 1 Fwd 2 Fwd 3 Revi l R ev 1 v 2 e R v 3 e R Jay Silver: GIT, INTEL MRL, & UCSD 9

Outline of Worm (White lines and annotations drawn by algorithm, other colors drawn by hand) For(all. Pixels) If(has 8 neigbors), delete. Pixel() Thin. Outline() 1 Notice extra pixels The X is on its way up to “run in” to the outline 2 Extra pixels removed by thinning Remove corners (make 8 -connected) Follow perpendicular line to backbone to find median 4 3 8 -connectivity Jay Silver: GIT, INTEL MRL, & UCSD 10

Extract Curvature from Outline Use as template While(traversing. Outline. Pixels){ M=Slope. Between. Pixels(i, i+10); A=Slope. To. Angle. In. Degrees(M); D=Diff. Angle(A, A+1); //assume |D|<90 C=accumulate(D); } // it is necessary to differentiate and then // sum to remove angle discontinuities // caused by the ambiguous angle of a line Jay Silver: GIT, INTEL MRL, & UCSD 11

Match Curvature with Egg Port (Left) Side Starboard (Right) Side Template MATCH (skewed to match horizontal scale) Concavity Covariance Enlarge Algorithm draws square at location where covariance exceeds threshold. Concavity Enlarge Covariance Jay Silver: GIT, INTEL MRL, & UCSD 12

Make Decisions BU FFE RH EG RIG HT G RE GI ON RI GH T BU FFE RH LEF T EG LE G R FT EG I ON If(curr. egg. Region. has. Egg) &(!prev. Buffer. Lost. Egg) &(prev. egg. Region!=egg) BUFFER T RIGHT {egg. Event=true; } BUFFER T LEFT Jay Silver: GIT, INTEL MRL, & UCSD 13

Future Improvements l l l Convert code to run on W 2 K and use PIV 1. 6 MHz Increase f. p. s. by using optimized code like Intel OSCV Lib Use several templates Add new measures to covariance Think of a way to deal with egg events in buffer region Variance(egg. Template - outline. Curvature) will be a good inverse measure of similarity. Possible measure to use in addition to covariance Starboard Curvature Port Head Starboard Curvature Outline Max Port Outline Centroid Covariances Jay Silver: GIT, INTEL MRL, & UCSD 14