Probabilistic Graphical Models Recitation 2 Markov Networks Yaniv

Probabilistic Graphical Models – Recitation 2 Markov Networks Yaniv Tenzer - Huji

Definitions • Factor- a (nonnegative) function whose scope is a set of random variables. For example: • A markov network is an undirected graph, whose nodes are random variables. If two nodes in the graph appear together in the scope of some factor, then they are connected by an edge. (Or in other words, the scope of each factor is a clique in G) • one can think of a Markov network as a pair of undirected graph and a set of factors.

Parameterization of joint distribution • The set of factors defines a valid distribution over the r. v. by the following: • This distribution is called “Gibbs distribution”. • Let P be a Gibbs distribution parameterized by a set of factors and H be a markov network. P is said to factorizes according to H, if the scope of each factor is a complete sub-graph of H.

Global Independencies in MN • Separation: Let X, Y, Z be disjoints sets in G. We say X is Hseparated Y given Z, if there is no active trail from any node in X to any node in Y, given Z. • We denote this separation by • The set of these separations is denoted by

Soundness and Completeness of Global Independencies in MN • Soundness: Let P be a distribution over X, and H a markov network structure over X. If P is a Gibbs distribution that factorizes according to H, then I(H) holds in P. (Or in other words, H is an I-map for P) • The other direction is true only for positive distribution. i. e. , if I(H) holds in P, then P factorizes according to G.

GM independencies in MN • We want to show that assuming I(H) holds in P, then positivity of P is necessary for factorization.

Example continue

Additional Independencies encoded • Pair wise independencies: • Local markov independencies: • Exercise: (optional) Show that for any Markov network H, and any distribution P:

Additional Independencies • Let P be a positive distribution, then: • A counter example:

Counter example - continue

Counter example continue

From Bayesian networks to MN – Moral graph • Let G be a directed acyclic graph. The moral graph of G, denoted by , is an undirected graph, were two nodes are connected if they were connected in G, or they share a child.

The Moral Graph

Soundness of d-separation in BN Revisited • Now that we have introduced the structure of moral-graph, we have a tool of making BN into an MN. • The moral graph is not enough, since we want an undirected graph such that the set of d-separation properties will be kept under this transformation. • The moral graph, basically doesn’t keep independencies assertions. For example – the independencies assertions associated with an immoral v-structure are lost.

Soundness of d-separation in BN – Revisited • Thus we need to choose sum subset of nodes in the graph, before applying the moral transformation. Intuitively, the relevant set consists on nodes that have an observed descendents.

Examples

Examples continue