Section 2 Appendix Tensor Notation for BP 1

Tensor Notation for BP In section 2, BP was introduced with a notation which defined messages and beliefs as functions. This Appendix includes an alternate (and very concise) notation for the Belief Propagation algorithm using tensors.

Tensor Notation • Tensor multiplication: • Tensor marginalization: 3

Tensor Notation A rank-r tensor is… A real function with r keyword arguments = Axis-labeled array with arbitrary indices = Database with column headers Tensor multiplication: (vector outer product) X 1 3 2 5 Y 4 X value Y value blue 6 1 3 red 4 2 5 blue 6 red Y X red blue 1 12 18 2 20 30 X Y value 1 red 12 2 red 20 1 blue 18 2 blue 30 4

Tensor Notation A rank-r tensor is… A real function with r keyword arguments = Axis-labeled array with arbitrary indices = Database with column headers Tensor multiplication: (vector pointwise product) X a 3 X b 5 X a 4 X value b 6 a 3 a 4 b 5 b 6 X value a 12 b 30 5

Tensor Notation A rank-r tensor is… A real function with r keyword arguments = Axis-labeled array with arbitrary indices = Database with column headers Tensor multiplication: (matrix-vector product) X Y value 1 red 1 7 2 2 8 1 3 4 X 2 5 6 X value 3 1 7 red 5 2 8 1 blue 4 2 blue 6 X Y value red 21 blue Y X red X blue red Y 1 1 21 28 2 red 40 2 40 48 1 blue 28 2 blue 48 6

Tensor Notation A rank-r tensor is… A real function with r keyword arguments = Axis-labeled array with arbitrary indices = Database with column headers Tensor marginalization: Y X X Y value red blue 1 red 3 1 3 4 2 red 5 2 5 6 1 blue 4 2 blue 6 Y Y value red blue red 8 8 10 blue 10 7

Sum-Product Belief Propagation Input: a factor graph with no cycles Output: exact marginals for each variable and factor Algorithm: 1. Initialize the messages to the uniform distribution. 1. 2. Choose a root node. Send messages from the leaves to the root. Send messages from the root to the leaves. 1. Compute the beliefs (unnormalized marginals). 2. Normalize beliefs and return the exact marginals. 8

Sum-Product Belief Propagation Variables Factors Beliefs ψ2 ψ1 X 2 ψ3 X 1 Messages ψ2 ψ1 X 1 ψ1 X 3 X 2 ψ3 X 1 ψ1 X 3 9

Sum-Product Belief Propagation Variables Factors Beliefs ψ2 ψ1 X 2 ψ3 X 1 Messages ψ2 ψ1 X 1 ψ1 X 3 X 2 ψ3 X 1 ψ1 X 3 10

Sum-Product Belief Propagation Variable Belief ψ2 v 0. 1 n 3 p 1 ψ1 v n p 1 2 2 v n p 4 1 0 ψ3 X 1 v n p . 4 6 0 11

Sum-Product Belief Propagation Variable Message ψ2 v 0. 1 n 3 p 1 ψ1 X 1 v n p 1 2 2 v 0. 1 n 6 p 2 ψ3 12

Sum-Product Belief Propagation Factor Belief p d n X 1 4 1 0 v p 0. 1 d 3 n 1 n 8 0 1 ψ1 v p 3. 2 d 18 n 0 v 8 n 0. 2 X 3 n 6. 4 0 0 13

Sum-Product Belief Propagation Factor Message p 0. 8 + 0. 16 24 + 0 d 8 + 0. 2 n X 1 v p 0. 1 d 3 n 1 ψ1 n 8 0 1 v 8 n 0. 2 X 3 14

Loopy Belief Propagation Input: a factor graph with cycles Output: approximate marginals for each variable and factor Algorithm: 1. Initialize the messages to the uniform distribution. 1. Send messages until convergence. Normalize them when they grow too large. 1. Compute the beliefs (unnormalized marginals). 2. Normalize beliefs and return the approximate marginals. 15