kazusmapip is tohoku ac jp http www smapip

Introduction Factor Graph and Loopy Belief Propagation F. R. Kschischang, B. J. Frey and

Generalized Belief Propagation Cluster: Set of nodes Every subcluster of the element of C

Generalized Belief Propagation KL Divergence Sum of all the Basic Clusters and their Sub-Clusters

Selection of Basic Cluster in LBP and GBP LBP (Bethe Approx. ） 1 4

Generalized Belief Propagation 5 2 1 3 4 5 2 1 1 4 4

Generalized Belief Propagation 5 5 2 2 1 1 5 2 1 1 3

Generalized Belief Propagation Marginal Probabilities are determined so as to minimize the Kikuchi Free

Generalized Belief Propagation 5 5 1 4 4 16 December, 2008 2 1 1

Generalized Belief Propagation 5 2 1 5 1 4 1 1 1 4 4

Generalized Belief Propagation 5 2 5 1 2 1 4 3 5 2 1

Generalized Belief Propagation 5 2 5 1 1 4 3 5 2 1 1

Simplified Cluster Variation Method Marginal Probabilities are determined so as to minimize the Kikuchi

Simplified Cluster Variation Method 5 5 1 4 4 16 December, 2008 2 1

Simplified Cluster Variation Method 5 2 1 5 1 4 1 1 1 4

Simplified Cluster Variation Method 5 2 5 1 1 4 3 5 2 1

Simplified Cluster Variation Method 5 2 5 1 2 1 4 3 5 2

Simplified Cluster Variation Method and Factor Graph 5 5 2 1 1 3 4

Simplified Cluster Variation Method 5 2 1 1 4 4 16 December, 2008 1

Simplified Cluster Variation Method and Factor Graph 5 5 2 2 1 1 3

Selection of Basic Clusters and their Sub-Clusters in Generalized Belief Propagation on Square Grid

Critical Points for Ising Model (V, E)=Cube Lattice (V, E)=Square Lattice TC/J Exact (Onsager)

Summary We reviewed the generalized belief propagation based on cluster variation method. Some extensions

Slides: 33

Download presentation

一般化された確率伝搬法の数学的構造東北大学大学院情報科学研究科田中和之 kazu@smapip. is. tohoku. ac. jp http: //www. smapip. is. tohoku. ac. jp/~kazu/ 16 December, 2008 DEX-SMI 2008 1

Introduction Factor Graph and Loopy Belief Propagation F. R. Kschischang, B. J. Frey and H. -A. Loeliger: IEEE Transactions on information theory, vol. 47, no. 2, 2001. Generalized Belief Propagation J. S. Yedidia, W. T. Freeman and Y. Weiss: IEEE Transactions on Information Theory, vol. 51, no. 7, pp. 22822312, 2005. Simplified Cluster Variation Method T. Morita: Physica A, vol. 155, no. 1, pp. 73 -83, 1989. Factor Graph による確率伝搬法の表現を一般化された確率伝搬法(Generalized Belief Propagation)とSimplified Cluster Variation Methodの定式化の立場から導出する． 16 December, 2008 DEX-SMI 2008 2

Generalized Belief Propagation Cluster: Set of nodes Every subcluster of the element of C does not belong to C. Example: System consisting of 4 nodes 1 2 1 3 4 16 December, 2008 3 3 4 2 4 DEX-SMI 2008 3

Generalized Belief Propagation KL Divergence Sum of all the Basic Clusters and their Sub-Clusters Example 1 2 1 3 4 3 Subclusters 16 December, 2008 1 2 3 4 2 4 Basic Clusters DEX-SMI 2008 4

Selection of Basic Cluster in LBP and GBP LBP (Bethe Approx. ） 1 4 2 5 2 3 5 6 8 8 8 9 1 2 3 4 4 5 6 7 7 7 8 9 1 2 2 3 4 4 5 5 6 6 7 8 8 9 GBP (Square Approx. in CVM; Kikuchi Approx. ) 16 December, 2008 DEX-SMI 2008 5 3 6 6 9 5

Generalized Belief Propagation 5 2 1 3 4 5 2 1 1 4 4 16 December, 2008 1 5 1 1 1 3 3 2 4 DEX-SMI 2008 1 1 1 3 7

Generalized Belief Propagation 5 5 2 2 1 1 5 2 1 1 3 2 4 5 2 3 4 1 1 3 4 2 5 2 1 16 December, 2008 1 5 2 1 DEX-SMI 2008 8

Generalized Belief Propagation Marginal Probabilities are determined so as to minimize the Kikuchi Free Energy under some Reducibility Conditions. 5 2 1 3 4 ~ = 5 1 + + 1 4 3 1 1 -1 - 5 2 5 1 1 16 December, 2008 2 1 1 +4 + 1 3 Kikuchi Free Energy 2 5 2 1 1 1 4 2 5 2 2 Reducibility Conditions DEX-SMI 2008 9

Generalized Belief Propagation 5 5 1 4 4 16 December, 2008 2 1 1 1 4 5 2 1 1 4 3 3 DEX-SMI 2008 10

Generalized Belief Propagation 5 2 1 5 1 4 1 1 1 4 4 16 December, 2008 1 2 1 1 3 5 2 1 4 3 3 3 DEX-SMI 2008 11

Generalized Belief Propagation 5 2 5 1 2 1 4 3 5 2 1 4 16 December, 2008 1 3 4 DEX-SMI 2008 3 12

Generalized Belief Propagation 5 2 5 1 2 1 4 3 5 2 1 4 16 December, 2008 1 3 4 DEX-SMI 2008 3 13

Generalized Belief Propagation 5 2 5 1 2 1 4 3 5 2 1 4 16 December, 2008 1 3 4 DEX-SMI 2008 3 14

Generalized Belief Propagation 5 2 5 1 1 4 3 5 2 1 1 3 2 4 5 5 1 3 4 DEX-SMI 2008 3 2 4 1 16 December, 2008 2 4 3 15

Generalized Belief Propagation Marginal Probabilities are determined so as to minimize the Kikuchi Free Energy under some Reducibility Conditions. 5 2 1 3 4 ~ = 5 1 5 2 2 + + 1 4 3 1 1 -1 - 5 2 1 +4 + 1 3 Kikuchi Free Energy 5 2 2 1 1 2 1 Reducibility Conditions 1 4 16 December, 2008 DEX-SMI 2008 16

Simplified Cluster Variation Method Marginal Probabilities are determined so as to minimize the Kikuchi Free Energy under some Reducibility Conditions. 5 2 1 3 4 ~ = 5 1 5 2 2 + 4 3 1 1 -1 - - 3 5 5 5 2 4 2 1 1 Reducibility Conditions DEX-SMI 2008 +4 + 3 1 2 1 1 16 December, 2008 1 + 1 1 2 1 17

Simplified Cluster Variation Method 5 5 1 4 4 16 December, 2008 2 1 1 1 4 5 2 1 1 4 3 3 DEX-SMI 2008 18

Simplified Cluster Variation Method 5 2 1 5 1 4 1 1 1 4 4 16 December, 2008 1 2 1 1 3 5 2 1 4 3 3 3 DEX-SMI 2008 19

Simplified Cluster Variation Method 5 2 5 1 1 4 3 5 2 1 4 16 December, 2008 2 1 3 4 DEX-SMI 2008 3 20

Simplified Cluster Variation Method 5 2 5 1 2 1 4 3 5 2 1 1 3 4 5 3 4 2 5 1 4 16 December, 2008 DEX-SMI 2008 2 1 3 3 4 21

Simplified Cluster Variation Method and Factor Graph 5 5 2 1 1 3 4 5 2 1 1 4 4 16 December, 2008 1 5 1 2 1 1 3 3 3 4 2 5 2 4 DEX-SMI 2008 1 1 1 3 22

Generalized Belief Propagation 5 2 1 3 4 5 2 1 1 4 4 16 December, 2008 1 5 1 1 1 3 3 2 4 DEX-SMI 2008 1 1 1 3 23

Generalized Belief Propagation 5 2 1 3 4 5 2 1 1 4 4 16 December, 2008 1 5 1 1 1 3 3 2 4 DEX-SMI 2008 1 1 1 3 24

Generalized Belief Propagation Marginal probabilities are determined so as to minimize the Kikuchi Free Energy under some Reducibility Conditions. 5 2 1 ~ = 5 3 4 1 + 1 1 5 5 2 2 1 3 2 + 1 4 3 +4 1 4 3 5 Kikuchi Free Energy 5 2 2 2 1 1 Reducibility Conditions 16 December, 2008 DEX-SMI 2008 26

Simplified Cluster Variation Method 5 2 1 1 4 4 16 December, 2008 1 5 2 1 1 4 3 3 3 DEX-SMI 2008 27

Simplified Cluster Variation Method 5 2 5 1 1 4 3 5 2 1 1 3 4 16 December, 2008 2 DEX-SMI 2008 4 3 28

Simplified Cluster Variation Method and Factor Graph 5 5 2 2 1 1 3 4 5 4 2 5 2 1 1 4 4 16 December, 2008 1 5 3 2 1 1 3 4 3 3 DEX-SMI 2008 29

Selection of Basic Clusters and their Sub-Clusters in Generalized Belief Propagation on Square Grid The set of Basic Clusters LBP (Bethe Approx. ) 16 December, 2008 DEX-SMI 2008 The Set of Basic Clusters and Their Sub-Clusters 30

Selection of Basic Clusters and their Sub-Clusters in Generalized Belief Propagation on Square Grid The set of Basic Clusters GBP (Square Approximation in CVM） 16 December, 2008 DEX-SMI 2008 The Set of Basic Clusters and Their Sub-Clusters 31

Critical Points for Ising Model (V, E)=Cube Lattice (V, E)=Square Lattice TC/J Exact (Onsager) 2. 2692 High Temperature Series 4. 5103 GBP 2. 4257 GBP 4. 6097 SCVM 2. 6253 SCVM 4. 7611 Square Cactus CVM 2. 7708 Cube Cactus CVM 4. 8396 Bethe Approx. 2. 8854 Bethe Approx. 4. 9326 Factor Graph 3. 4659 Mean Field Approx. 6. 0 Mean Field Approx. 4. 0000 Factor Graph 8. 6181 16 December, 2008 DEX-SMI 2008 32

Summary We reviewed the generalized belief propagation based on cluster variation method. Some extensions of the generalized belief propagation have been given by using the simplified cluster variation method. The relationship between the factor graph and the present extensions by means of the simplified cluster variation method are clarified. 16 December, 2008 DEX-SMI 2008 33