Probabilistic Graphical Models Representation Markov Networks Conditional Random
Probabilistic Graphical Models Representation Markov Networks Conditional Random Fields Daphne Koller
Motivation • Observed variables X • Target variables Y Daphne Koller
CRF Representation Daphne Koller
CRFs and Logistic Model Daphne Koller
CRFs for Language Features: word capitalized, word in atlas or name list, previous word is “Mrs”, next word is “Times”, … Daphne Koller
More CRFs for Language Daphne Koller
Summary • A CRF is parameterized the same as a Gibbs distribution, but normalized differently • Don’t need to model distribution over variables we don’t care about • Allows models with highly expressive features, without worrying about wrong independencies Daphne Koller
END END Daphne Koller
Daphne Koller
Daphne Koller
Daphne Koller
Daphne Koller
Daphne Koller
The Chain Rule for Bayesian Nets d 0 d 1 i 0 i 1 0. 6 0. 4 0. 7 0. 3 Difficulty i 0, d 0 i 0, d 1 i 1, d 0 i 1, d 1 g 1 0. 3 0. 05 0. 9 0. 5 g 2 0. 4 0. 25 0. 08 0. 3 g 3 0. 7 0. 02 0. 2 Intelligence Grade Letter g 1 g 2 g 3 l 0 0. 1 0. 4 0. 99 P(D, I, G, S, L) = P(D) P(I) P(G | I, D) P(L | G) P(S | I) l 1 0. 9 0. 6 0. 01 SAT i 0 i 1 s 0 s 1 0. 95 0. 2 0. 05 0. 8 Daphne Koller
Suppose q is at a local minimum of a function. What will one iteration of gradient descent do? Leave q unchanged. Change q in a random direction. Move q towards the global minimum of J(q). Decrease q.
Fig. A corresponds to a=0. 01, Fig. B to a=0. 1, Fig. C to a=1. Fig. A corresponds to a=0. 1, Fig. B to a=0. 01, Fig. C to a=1. Fig. A corresponds to a=1, Fig. B to a=0. 01, Fig. C to a=0. 1. Fig. A corresponds to a=1, Fig. B to a=0. 1, Fig. C to a=0. 01.
- Slides: 18