Markov Random Fields Conditional Random Fields John Winn

Markov Random Fields & Conditional Random Fields John Winn MSR Cambridge

Road map n Markov Random Fields What they are q Uses in vision/object recognition q Advantages q Difficulties q n Conditional Random Fields What they are q Further difficulties q

Markov Random Fields X 1 12 X 2 23 234 X 3

Examples of use in vision n n Grid-shaped MRFs for pixel labelling e. g. segmentation MRFs (e. g. stars) over part positions for pictorial structures/constellation models.

Advantages n Probabilistic model: q q n Undirected model q n Captures uncertainty No ‘irreversible’ decisions Iterative reasoning Principled fusing of different cues Allows ‘non-causal’ relationships (soft constraints) Efficient algorithms: inference now practical for MRFs with millions variables – can be applied to raw pixels.

Maximum Likelihood Learning Sufficient statistics of data Expected model sufficient statistics

Difficulty I: Inference n n Exact inference intractable except in a few cases e. g. small models Must resort to approximate methods q q q Loopy belief propagation MCMC sampling Alpha expansion (MAP solution only)

Difficulty II: Learning n n n Gradient descent – vulnerable to local minima Slow – must perform expensive inference at each iteration. Can stop inference early… q q n Contrastive divergence Piecewise training + variants Need fast + accurate methods

Difficulty III: Large cliques n n n For images, we want to look at patches not pairs of pixels. Therefore would like to use large cliques. Cost of inference (memory and CPU) typically exponential in clique size. Example: Field of Experts, Black + Roth q Training: contrastive divergence over a week on a cluster of 50+ machines q Test: Gibbs sampling very slow?

Other MRF issues… n n Local minima when performing inference in highdimensional latent spaces MRF models often require making inaccurate independence assumptions about the observations.

Conditional Random Fields Lafferty et al. , 2001 X 1 12 X 2 23 234 I X 4 X 3

Examples of use in vision n Grid-shaped CRFs for pixel labelling (e. g. segmentation), using boosted classifiers.

Difficulty IV: CRF Learning Sufficient statistics Expected sufficient of labels given the image statistics given the image

Difficulty V: Scarcity of labels n n n CRF is a conditional model – needs labels. Labels are expensive + increasingly hard to define. Labels are also inherently lower dimensional than the data and hence support learning fewer parameters than generative models.