Outline S C Zhu X Liu and Y

Outline • S. C. Zhu, X. Liu, and Y. Wu, “Exploring Texture Ensembles by Efficient Markov Chain Monte Carlo”, IEEE Transactions On Pattern Analysis And Machine Intelligence, Vol. 22, No. 6, pp. 554 -569, 2000 10/27/2020 Computer Vision 1

Limitations of Linear Representations • Linear representations do not depend on the spatial relationships among pixels – For example, if we shuffle the pixels and corresponding representations, then the classification results will remain the same • But in images spatial relationships are important 10/27/2020 Computer Vision 2

Image Features 10/27/2020 Computer Vision 3

Spectral Representation of Images • Spectral histogram – Given a bank of filters F(a), a = 1, …, K, a spectral histogram is defined as the marginal distribution of filter responses 10/27/2020 Computer Vision 4

Spectral Representation of Images - continued • An example of spectral histogram 10/27/2020 Computer Vision 5

Image Modeling - continued • Given observed feature statistics {H(a)obs}, we associate an energy with any image I as • Then the corresponding Gibbs distribution is – The q(I) can be sampled using a Gibbs sampler or other Markov chain Monte-Carlo algorithms 10/27/2020 Computer Vision 6

Image Modeling - continued Image Synthesis Algorithm • Compute {Hobs} from an observed texture image • Initialize Isyn as any image, and T as T 0 • Repeat Randomly pick a pixel v in Isyn Calculate the conditional probability q(Isyn(v)| Isyn(-v)) Choose new Isyn(v) under q(Isyn(v)| Isyn(-v)) Reduce T gradually • Until E(I) < e 10/27/2020 Computer Vision 7

A Texture Synthesis Example Observed image 10/27/2020 Initial synthesized image Computer Vision 8

A Texture Synthesis Example Temperature Image patch Energy Conditional probability • 10/27/2020 Energy and conditional. Computer probability of the marked Vision pixel 9

A Texture Synthesis Example - continued • A white noise image was transformed to a perceptually similar texture by matching the spectral histogram 10/27/2020 Computer Vision 10 Average spectral histogram error

A Texture Synthesis Example - continued • Synthesized images from different initial conditions 10/27/2020 Computer Vision 11

Texture Synthesis Examples - continued Observed image Synthesized image • A random texture image 10/27/2020 Computer Vision 12

Texture Synthesis Examples - continued Observed image Synthesized image • An image with periodic structures 10/27/2020 Computer Vision 13

Texture Synthesis Examples - continued Mud image Synthesized image • A mud image with some animal foot prints 10/27/2020 Computer Vision 14

Texture Synthesis Examples - continued Observed image Synthesized image • A random texture image with elements 10/27/2020 Computer Vision 15

Texture Synthesis Examples - continued Observed image Synthesized image • An image consisting of two regions – Note that wrap-around boundary conditions were used 10/27/2020 Computer Vision 16

Texture Synthesis Examples - continued Synthesized image Original cheetah skin patch • A cheetah skin image 10/27/2020 Computer Vision 17

Texture Synthesis Examples - continued Observed image Synthesized image • An image consisting of circles 10/27/2020 Computer Vision 18

Texture Synthesis Examples - continued Observed image Synthesized image • An image consisting of crosses 10/27/2020 Computer Vision 19

Texture Synthesis Examples - continued Observed image Synthesized image • A pattern with long-range structures 10/27/2020 Computer Vision 20

Object Synthesis Examples • As in texture synthesis, we start from a random image • In addition, similar object images are used as boundary conditions in that the corresponding pixel values are not updated during sampling process 10/27/2020 Computer Vision 21

Object Synthesis Examples - continued 10/27/2020 Computer Vision 22

Object Synthesis Examples - continued 10/27/2020 Computer Vision 23

Linear Transformations of Images • Linear transformations include – – – Principal component analysis Independent component analysis Fisher discriminant analysis Optimal component analysis They have been widely used to reduce dimension of images for appearance-based recognition applications • Each image is viewed as a long vector and projected into a set of bases that have certain properties 10/27/2020 Computer Vision 24

Principal Component Analysis • Defined with respect to a training set such that the average reconstruction error is minimized 10/27/2020 Computer Vision 25

Principal Component Analysis - continued 10/27/2020 Computer Vision 26

Eigen Values of 400 Eigen Vectors 10/27/2020 Computer Vision 27

Principal Component Analysis - continued Original Image 10/27/2020 Reconstructed using 50 PCs Computer Vision Reconstructed using 200 PCs 28

Principal Component Analysis - continued • Is PCA representation a good representation of images for recognition in that images that have similar principal representations are similar? – Image generation through sampling – Roughly speaking, we try to generate images that have the given coefficients along PCs 10/27/2020 Computer Vision 29

Principal Component Analysis - continued 10/27/2020 Computer Vision 30

Principal Component Analysis - continued 10/27/2020 Computer Vision 31

Difference Between Reconstruction and Sampling Reconstruction is not sufficient to show the adequacy of a representation and sampling from the set of images with same representation is more informational 10/27/2020 Computer Vision 32

Object Recognition Experiments • We compare linear methods in the methods including – – Principal component analysis (PCA) Independent component analysis (ICA) Fisher discriminant analysis (FDA) Random component analysis (RCA) • For fun and to show the actual gain of using different bases is relatively small • Corresponding linear methods in the spectral histogram space including – SPCA, SICA, SFDA, and SRCA 10/27/2020 Computer Vision 33

COIL Dataset 10/27/2020 Computer Vision 34

3 D Recognition Results 10/27/2020 Computer Vision 35

Experimental Results - continued • To further demonstrate the effectiveness of our method for different types of images, we create a dataset of combining the texture dataset, face dataset, and COIL dataset, resulting in a dataset of 180 categories with 10160 images in total 10/27/2020 Computer Vision 36

Linear Subspaces of Spectral Representation 10/27/2020 Computer Vision 37

Experimental Results - continued • Combined dataset – continued – Not only the recognition rate is very good, but also it is very reliable and robust, as the average entropy of the p 0(i|I) is 0. 60 bit (The corresponding uniform distribution’s entropy is 7. 49 bits) 10/27/2020 Computer Vision 38

Experimental Results - continued • Combined dataset – continued – Not only the recognition rate is very good, but also it is very reliable and robust, as the average entropy of the p 0(i|I) is 0. 60 bit (The corresponding uniform distribution’s entropy is 7. 49 bits) Entropy=6. 78 bits Entropy=0. 60 bit 10/27/2020 Computer Vision 39