Computational tools for linking form and function Charless

Computational tools for linking form and function Charless Fowlkes UC Irvine

Overview • Automatically linking shape and function • Learning shape transformations (with Ragib Morshed at UCI) • Comparing gene expression patterns between species (with Angela De. Pace at Harvard) (to apear in PLo. S Genomics)

Automatically Connecting Form and Function • Given shape measurements and known biology, tell me what aspects of the shape are “important” • Given some new shapes, suggest some interesting predictions about biology that one can go off and verify

PCA

Example Deformations

Principal Modes of Deformation

PCA on shape parameterization

PCA -> regression x = coordinate in shape space S y = functional prediction Problem: Build up a model for P(y|x) e. g. in linear regression y = Cx + e

regression -> dimensionality reduction x = coordinate in shape space S F(x) = projection of X into some subspace/submanifold of S y = functional/phylogenic prediction Problem: Find F so that P(y|x, F(x)) ≈ P(y|F(x)) y is independent of x given F(x)

- Simulated scattering from incoming plane wave using MATLABs PDE toolbox

magnitude theta Compute acoustical energy scattered in different directions far from the object

Shape parameterization

Shape parameterization

Shapes in “function space” - Distance measured by Euclidian distance between acoustic profiles - Shown here is 3 D visualization of those distances (via MDS)

Define distance on “function space”

Difficulties 1. Simulation is computationally expensive 2. Parameterization of shape high dimensional and sparsely sampled hard to visualize/understand the embedding

Idea #1 • Replace simulation with regression (interpolate between simulated results) Gaussian Process Regression (GPR) - general non-parametric Bayesian technique - can incorporate prior information about measurement noise - train using simulation to generate samples Yi=S(Xi) and then fit predictor Ŝ

Forward simulation/prediction X = 12 numbers specifying coordinates of hexagon Y = 400 measurements of amplitude as a function of θ 100 shape/sound training pairs, parameters set using maximum likelihood

Forward simulation/prediction

Forward prediction/simulation

Inverse Simulation X = 400 measurements of amplitude as a function of θ Y = 12 numbers specifying coordinates of hexagon 100 random shape/sound training pairs, parameters set using maximum likelihood

Idea #2 • Use inverse simulation to pull metric from output (function) space back to input (shape) space – e. g. find principal modes of variation in acoustical response, use inverse regression to map back to variations in shape.

Backscatter only

Backscatter only

Topscatter only

Difficulties… -PCA is not a good model for this manifold …many other techniques to test - Making this work on real data (much sparser)

Generally applicable? Bat Ear Shape Beam Pattern Heart Shape Timeseries Fluid Flow Pollen Grain Shape Phylogenetic Distance Mitochondria Shape Motility

Overview • Automatically linking shape and function • Learning shape transformations • Comparing gene expression patterns between species

Learning Transformation Groups • Suppose we have a collection of shapes which are related by action of some transformation group • Classic examples: – rigid object subject to rotation in 3 D – images of an object lit by a point light source placed at different locations in space

Learning transformations Can we learn the set of transformations and how they act on the shape from examples of the transformed shape? If so, we can 1. predict how other new shapes might be transformed 2. measure distances between shapes in a way which is invariant to transformations

Toy example • shape = x = a point in 2 D • transformation = rotation around origin dx/dt = Ax x(t) = C exp(At)x 0

Toy example • shape = x = a point in 2 D • transformation = rotation around origin dx/dt = Ax x(t) = C exp(At)x 0 *matrix exponential

Modeling continuous transformations Transformation from x 0 to x 1 modeled by: Replace single time t with a vector c Linearization appropriate for nearby point pairs x 0, x 1

Learning transformations Given collection of point pairs {x 0, x 1} find transformation matrices Psi In practice, use regularization…

Linearized Version Given training vectors: Optimize: Objective is bilinear in c and psi so can just alternate between two least squares problems.

Learning vector fields (1 D transformation groups)

2 D transformation group

Learned tangent spaces

Extension to non-linear case What if vector at x 0 isn’t a linear function of coordinates of x 0? Trick: apply non-linear mapping to lift points to a higher-dimensional space

Iter 5 Iter 15 Iter 20

Iter 1 Iter 5 Iter 10

Iter 30

Other ideas: How to use prior knowledge about transformations? e. g. we know heart shapes are cyclic Enforcing that collection of Psi’s are closed under bracket => foliation of shape space