Probabilistic Reasoning Networkbased reasoning Set 7 ICS 179
Probabilistic Reasoning; Network-based reasoning Set 7 ICS 179, Spring 2010
Propositional Reasoning = B = A Example: party problem n If Chris goes, then Alex goes: n Question: = If Alex goes, then Becky goes: = n A C Is it possible that Chris goes to the party but Becky does not? Chavurah 5/8/2010
Probabilistic Reasoning Party example: the weather effect n n n Alex is-likely-to-go in bad weather Chris rarely-goes in bad weather Becky is indifferent but unpredictable W A P(A|W=bad)=. 9 W C P(C|W=bad)=. 1 W B P(B|W=bad)=. 5 Questions: n n Given bad weather, which group of individuals is most likely to show up at the party? What is the probability that Chris goes to the party but Becky does not? P(W) W A P(A|W) good 0 . 01 good 1 . 99 bad 0 . 1 bad 1 . 9 W P(W, A, C, B) = P(B|W) · P(C|W) · P(A|W) · P(W) P(A, C, B|W=bad) = 0. 9 · 0. 1 · 0. 5 Chavurah 5/8/2010 B C P(B|W) P(C|W) A P(A|W)
Mixed Probabilistic and Deterministic networks PN CN P(W) W P(B|W) P(C|W) A→B P(A|W) B Algorithms? Chavurah 5/8/2010 C C→A C Query: Semantics? A Is it likely that Chris goes to the party if Becky does not but the weather is bad?
The problem All men are mortal All penguins are birds T T … Socrates is a man Men are kind Birds fly T looks like a penguin Turn key –> car starts p 1 p 2 True propositions Uncertain propositions P_n Q: Does T fly? Logic? . . but how we handle exceptions P(Q)? Probability: astronomical 5
6
12
13
Alpha and beta are events
15
Burglary is independent of Earthquake
Earthquake is independent of burglary
20
26
27
28
29
30
31
32
33
34
Bayesian Networks: Representation Smoking P(S) P(C|S) P(B|S) lung Cancer Bronchitis P(X|C, S) X-ray P(D|C, B) Dyspnoea CPD: C 0 0 1 1 B D=0 D=1 0 0. 1 0. 9 1 0. 7 0. 3 0 0. 8 0. 2 1 0. 9 0. 1 P(S, C, B, X, D) = P(S) P(C|S) P(B|S) P(X|C, S) P(D|C, B) Conditional Independencies Efficient Representation 35
Bayesian networks Chapter 14 , Russel and Norvig Section 1 – 2
Outline n n Syntax Semantics
Example n n n Topology of network encodes conditional independence assertions: Weather is independent of the other variables Toothache and Catch are conditionally independent given Cavity
Example n I'm at work, neighbor John calls to say my alarm is ringing, but neighbor Mary doesn't call. Sometimes it's set off by minor earthquakes. Is there a burglar? n Variables: Burglary, Earthquake, Alarm, John. Calls, Mary. Calls n Network topology reflects "causal" knowledge: n n A burglar can set the alarm off An earthquake can set the alarm off The alarm can cause Mary to call The alarm can cause John to call
Example contd.
Compactness n n n A CPT for Boolean Xi with k Boolean parents has 2 k rows for the combinations of parent values Each row requires one number p for Xi = true (the number for Xi = false is just 1 -p) If each variable has no more than k parents, the complete network requires O(n · 2 k) numbers n I. e. , grows linearly with n, vs. O(2 n) for the full joint distribution n For burglary net, 1 + 4 + 2 = 10 numbers (vs. 2 5 -1 = 31)
Semantics The full joint distribution is defined as the product of the local n conditional distributions: P (X 1, … , Xn) = πi = 1 P (Xi | Parents(Xi)) e. g. , P(j m a b e) = P (j | a) P (m | a) P (a | b, e) P ( b) P ( e)
Constructing Bayesian networks n n 1. Choose an ordering of variables X 1, … , Xn 2. For i = 1 to n n add Xi to the network n select parents from X 1, … , Xi-1 such that P (Xi | Parents(Xi)) = P (Xi | X 1, . . . Xi-1) This choice of parents guarantees: P (X 1, … , Xn) (chain rule) = πi =1 P (Xi | X 1, … , Xi-1) = πi =1 P (Xi | Parents(Xi)) (by construction)
Example n Suppose we choose the ordering M, J, A, B, E P(J | M) = P(J)?
Example n Suppose we choose the ordering M, J, A, B, E P(J | M) = P(J)? No P(A | J, M) = P(A | J)? P(A | J, M) = P(A)?
Example n Suppose we choose the ordering M, J, A, B, E P(J | M) = P(J)? No P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? No P(B | A, J, M) = P(B | A)? P(B | A, J, M) = P(B)?
Example n Suppose we choose the ordering M, J, A, B, E P(J | M) = P(J)? No P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? No P(B | A, J, M) = P(B | A)? Yes P(B | A, J, M) = P(B)? No P(E | B, A , J, M) = P(E | A)? P(E | B, A, J, M) = P(E | A, B)?
Example n Suppose we choose the ordering M, J, A, B, E P(J | M) = P(J)? No P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? No P(B | A, J, M) = P(B | A)? Yes P(B | A, J, M) = P(B)? No P(E | B, A , J, M) = P(E | A)? No P(E | B, A, J, M) = P(E | A, B)? Yes
Example contd. Deciding conditional independence is hard in noncausal directions n n (Causal models and conditional independence seem hardwired for humans!) Network is less compact: 1 + 2 + 4 = 13 numbers needed
- Slides: 70