Probabilistic Graphical Models Representation Bayesian Networks Semantics Factorization

Probabilistic Graphical Models Representation Bayesian Networks Semantics & Factorization Daphne Koller

• • • Grade Course Difficulty Student Intelligence Student SAT Reference Letter Daphne Koller

The Student Network d 0 d 1 0. 6 0. 4 i 0 0. 7 Difficulty i 0, d 0 i 0, d 1 i 1, d 0 i 1, d 1 g 1 0. 3 0. 05 0. 9 0. 5 g 2 0. 4 0. 25 0. 08 0. 3 g 3 0. 7 0. 02 0. 2 i 1 0. 3 Intelligence Grade SAT i 0 i 1 s 0 0. 95 0. 2 s 1 0. 05 0. 8 Letter g 1 g 2 g 3 l 0 0. 1 0. 4 0. 99 l 1 0. 9 0. 6 0. 01 Daphne Koller

The Chain Rule for Bayesian Nets Difficulty Intelligence Grade SAT Letter Daphne Koller

Bayesian Network • A Bayesian network is: Daphne Koller

BN Is a Legal Distribution Daphne Koller

P Factorizes over G • Let G be a graph over X 1, …, Xn. • P factorizes over G if P(X 1, …, Xn) = Daphne Koller

Genetic Inheritance Clancy Marge Homer Bart Lisa Jackie Selma Maggie Daphne Koller

BNs for Genetic Inheritance GClancy GJackie BClancy GHomer BJackie GMarge GSelma BMarge BHomer GBart GLisa GMaggie BBart BLisa BMaggie BSelma Daphne Koller

END END Daphne Koller

Daphne Koller

Daphne Koller

Daphne Koller

Daphne Koller

Daphne Koller

The Chain Rule for Bayesian Nets d 0 d 1 i 0 i 1 0. 6 0. 4 0. 7 0. 3 Difficulty i 0, d 0 i 0, d 1 i 1, d 0 i 1, d 1 g 1 0. 3 0. 05 0. 9 0. 5 g 2 0. 4 0. 25 0. 08 0. 3 g 3 0. 7 0. 02 0. 2 Intelligence Grade Letter g 1 g 2 g 3 l 0 0. 1 0. 4 0. 99 P(D, I, G, S, L) = P(D) P(I) P(G | I, D) P(L | G) P(S | I) l 1 0. 9 0. 6 0. 01 SAT i 0 i 1 s 0 s 1 0. 95 0. 2 0. 05 0. 8 Daphne Koller

Suppose q is at a local minimum of a function. What will one iteration of gradient descent do? Leave q unchanged. Change q in a random direction. Move q towards the global minimum of J(q). Decrease q.

Consider the weight update: Which of these is a correct vectorized implementation?

Fig. A corresponds to a=0. 01, Fig. B to a=0. 1, Fig. C to a=1. Fig. A corresponds to a=0. 1, Fig. B to a=0. 01, Fig. C to a=1. Fig. A corresponds to a=1, Fig. B to a=0. 01, Fig. C to a=0. 1. Fig. A corresponds to a=1, Fig. B to a=0. 1, Fig. C to a=0. 01.