Outline Texture modeling continued Markov Random Field models

Outline • Texture modeling - continued – Markov Random Field models – Fractals 10/26/2020 Visual Perception Modeling 1

Some Texture Examples 10/26/2020 Visual Perception Modeling 2

Texture Modeling • Texture modeling is to find feature statistics that characterize perceptual appearance of textures • There are two major computational issues – What kinds of feature statistics shall we use? – How to verify the sufficiency or goodness of chosen feature statistics? 10/26/2020 Visual Perception Modeling 3

Texture Modeling – cont. • The structures of images – The structures in images are due to the inter-pixel relationships – The key issue is how to characterize the relationships 10/26/2020 Visual Perception Modeling 4

Co-occurrence Matrices • Gray-level co-occurrence matrix – One of the early texture models – Was widely used – Suppose that there are G different gray values in a texture image I – For a given displacement vector (dx, dy), the entry (i, j) of the co-occurrence matrix Pd is 10/26/2020 Visual Perception Modeling 5

Autocorrelation Features • Autocorrelation features – Many textures have repetitive nature of texture elements – The autocorrelation function can be used to assess the amount of regularity as well as the fineness/coarseness of the texture present in the image 10/26/2020 Visual Perception Modeling 6

Geometrical Models • Geometrical models – Applies to textures with texture elements – First texture elements are extracted – Then one can compute the statistics of local elements or extract the placement rule that describes the texture – Voronoi tessellation features – Structural methods 10/26/2020 Visual Perception Modeling 7

Markov Random Fields • Markov random fields – Have been popular for image modeling, including textures – Able to capture the local contextual information in an image 10/26/2020 Visual Perception Modeling 8

Markov Random Fields – cont. • Sites – Let S index a discrete set of m sites S = {1, . . , m} – A site represents a point or a region in the Euclidean space • Such as an image pixel • Labels – A label is an event that may happen to a site • Such as pixel values 10/26/2020 Visual Perception Modeling 9

Markov Random Fields – cont. • Labeling problem – Assign a label from the label set L to each of the sites in S – Also a mapping from S L – A labeling is called a configuration • In texture modeling, a configuration is a texture image – The set of all possible configurations is called the configuration space 10/26/2020 Visual Perception Modeling 10

Markov Random Fields – cont. • Neighborhood systems – The sites in S are related to one another via a neighborhood – A neighborhood system for S is defined as – The neighborhood relationship has the following properties • A site is not a neighbor to itself • The neighborhood relationship is mutual 10/26/2020 Visual Perception Modeling 11

Markov Random Fields – cont. • Markov random fields – Let F={F 1, . . , Fm} be a family of random variables defined on the set S in which each random variable Fi takes a value from L – F is said to be a Markov random field on S with respect to a neighborhood system N if an only if the following two conditions are satisfied: 10/26/2020 Visual Perception Modeling 12

Markov Random Fields – cont. • Homogenous MRFs – If P(fi | f. Ni) is regardless of the relative position of site i in S • How to specify a Markov random field – Conditional probabilities P(fi | f. Ni) – Joint probability P(f) 10/26/2020 Visual Perception Modeling 13

Markov Random Fields – cont. • Gibbs random fields – A set of random variables F is said to be a Gibbs random field on S with respect to N if and only if its configurations obey a Gibbs distribution – and 10/26/2020 Visual Perception Modeling 14

Markov Random Fields – cont. • Cliques – A clique c for (S, N) is defined as a subset of sites in S and it consists of • • 10/26/2020 A single site A pair of neighboring sites A triple of neighboring sites. . . . Visual Perception Modeling 15

Markov Random Fields – cont. • Markov-Gibbs equivalence – Hammersley-Clifford theorm • F is an Markov random field on S respect to N if and only if F is a Gibbs random field on S with respect to N • Practical value of theorem – It provides a simple way to specify the joint probability by specifying the clique potentials 10/26/2020 Visual Perception Modeling 16

Markov Random Fields – cont. • Markov random field models for textures – Homogeneity of Markov random fields is assumed – A texture type is characterized by a set of parameters associated with clique types – Texture images can be generated (synthesized) by sampling from the Markov random field model 10/26/2020 Visual Perception Modeling 17

Markov Random Fields – cont. • The -model – The energy function is of the form – with 10/26/2020 Visual Perception Modeling 18

Markov Random Fields – cont. • Parameter estimation – Parameters are generally estimated using Maximum-Likelihood estimator or Maximum-APosterior estimator • Computationally, the partition function can not be evaluated • Markov chain Monte Carlo is often used to estimate the partition function by generating typical samples from the distribution 10/26/2020 Visual Perception Modeling 19

Markov Random Fields – cont. • Pseudo-likelihood – Instead of maximizing P(f), the joint probability, we maximize the products of conditional probabilities 10/26/2020 Visual Perception Modeling 20

Markov Random Fields – cont. • Texture synthesis – Generate samples from the Gibbs distributions – Two sampling techniques • Metropolis sampler • Gibbs sampler 10/26/2020 Visual Perception Modeling 21

Markov Random Fields – cont. 10/26/2020 Visual Perception Modeling 22

Fractals • Fractals – Many natural surfaces have a statistical quality of roughness and self-similarity at different scales – Fractals are very useful in modeling selfsimilarity • Texture features based on fractals – Fractal dimension – Lacunarity 10/26/2020 Visual Perception Modeling 23

Fractals – An Example 10/26/2020 Visual Perception Modeling 24