Simulation of the matrix Binghamvon Mises Fisher distribution

Simulation of the matrix Bingham-von Mises. Fisher distribution, with applications to multivariate and relational data Peter D. Hoff to appear in Journal of Computational and Graphical Statistics Discussion led by Chunping Wang ECE, Duke University July 10, 2009

Outline • Introduction and Motivations • Sampling from the Vector Von Mises-Fisher (v. MF) Distribution (existing method) • Sampling from the Matrix Von Mises-Fisher (m. MF) Distribution • Sampling from the Bingham-Von Mises-Fisher (BMF) Distribution • One Example • Conclusions 1/21

Introduction Stiefel manifold: set of rank- orthonormal matrices, denoted The matrix Bingham-von Mises-Fisher distribution The matrix von Mises-Fisher distribution – linear term The matrix Bingham distribution – quadratic term 2/21

Motivations Sampling orthonormal matrices from distributions is useful for many applications. Examples: • Factor analysis observed matrix latent Given uniform priors over Stiefel manifold, 3/21

Motivations • Principal components observed matrix, with each row Eigen-value decomposition with Likelihood Posterior with respect to uniform prior 4/21

Motivations • Network data , symmetric binary observed matrix, with the 0 -1 indicator of a link between nodes i and j. E: symmetric matrix of independent standard normal noise Posterior with respect to uniform prior 5/21

Sampling from the v. MF Distribution (wood, 1994) the modal vector; , concentration parameter A distribution on the -sphere in constant distribution for any given angle defines the modal direction. 6/21

Sampling from the v. MF Distribution (wood, 1994) (1) A simple direction ( Proposal envelope ) (2) An arbitrary direction For a fixed orthogonal matrix , 7/21

Sampling from the m. MF Distribution Rejection sampling scheme 1: uniform envelope rejection region a bound Sample Acceptance region when accept Extremely inefficient 8/21

Sampling from the m. MF Distribution Rejection sampling scheme 2: based on sampling from v. MF Proposal samples are drawn from v. MF density functions with parameter , constrained to be orthogonal to other columns of. Y Y Y 9/21

Sampling from the m. MF Distribution Rejection sampling scheme 2: based on sampling from v. MF Proposal samples are drawn from v. MF density functions with parameter , constrained to be orthogonal to other columns of. Y Y Y 9/21

Sampling from the m. MF Distribution Rejection sampling scheme 2: based on sampling from v. MF Proposal samples are drawn from v. MF density functions with parameter , constrained to be orthogonal to other columns of. Y Y Rotate the modal direction Y 9/21

Sampling from the m. MF Distribution Rejection sampling scheme 2: based on sampling from v. MF Proposal samples are drawn from v. MF density functions with parameter , constrained to be orthogonal to other columns of. Y Y Y Rotate the sample to be orthogonal to the previous columns 9/21

Sampling from the m. MF Distribution Rejection sampling scheme 2: based on sampling from v. MF Proposal samples are drawn from v. MF density functions with parameter , constrained to be orthogonal to other columns of. Y Y Y Proposal distribution 9/21

Sampling from the m. MF Distribution Rejection sampling scheme 2: based on sampling from v. MF Sample scheme: 10/21

Sampling from the m. MF Distribution A Gibbs sampling scheme Sample iteratively • Note that. When. remedy: sampling two columns at a time • Non-orthogonality among the columns of add to the autocorrelation in the Gibbs sampler. remedy: performing the Gibbs sampler on 11/21

Sampling from the BMF Distribution The vector Bingham distribution 12/21

Sampling from the BMF Distribution The vector Bingham distribution 12/21

Sampling from the BMF Distribution The vector Bingham distribution Better mixing 12/21

Sampling from the BMF Distribution The vector Bingham distribution From variable substitution, rejection sampling or grid sampling 12/21

Sampling from the BMF Distribution The vector Bingham distribution The density is symmetric about zero 12/21

Sampling from the BMF Distribution The vector Bingham-von Mises-Fisher distribution The density is not symmetric about zero any more, is no longer uniformly distributed on. The update of and should be done jointly. The modified step 2(b) and 2(c) are: 13/21

Sampling from the BMF Distribution The matrix Bingham-von Mises-Fisher distribution Rewrite 14/21

Sampling from the BMF Distribution The matrix Bingham-von Mises-Fisher distribution Sample two columns at a time Parameterize 2 -dimensional orthonormal matrices as Uniform pairs on the circle Uniform 15/21

Sampling from the BMF Distribution The matrix Bingham-von Mises-Fisher distribution 16/21

Example: Eigenmodel estimation for network data 17/21

Example: Eigenmodel estimation for network data , symmetric binary observed matrix, with the 0 -1 indicator of a link between nodes i and j. E: symmetric matrix of independent standard normal noise Posterior with respect to uniform prior BMF distribution with 18/21

Example: Eigenmodel estimation for network data Samples from two independent Markov chains with different starting values 19/21

Example: Eigenmodel estimation for network data 20/21

Conclusions • The sampling scheme of a family of exponential distributions over the Stiefel manifold was developed; • This enables us to make Bayesian inference for those orthonormal matrices and incorporate prior information during the inference; • The author mentioned several application and implemented the sampling scheme on a network data set. 21/21

References • Andrew T. A. Wood. Simulation of the von Mises Fisher distribution. Comm. Statist. Simulation Comput. , 23: 157164, 1994 • G. Ulrich. Computer generation of distributions on the msphere. Appl. Statist. , 33, 158 -163, 1984 • J. G. Saw. A family of distributions on the m-sphere and some hypothesis tests. Biometrika, 65, 69 -74, 1978