1118 Everything is fine Everything is fine Prob
11/18 • Everything is fine. . Everything is fine…
Prob. Prop logic: The Game plan • We will review elementary “discrete variable” probability • We will recall that joint probability distribution is all we need to answer any probabilistic query over a set of discrete variables. • We will recognize that the hardest part here is not the cost of inference (which is really only O(2 n) –no worse than the (deterministic) prop logic • The real problem is assessing probabilities. – You could need as many as 2 n numbers (if all variables are dependent on all other variables); or just n numbers if each variable is independent of all other variables. Generally, you are likely to need somewhere between these two extremes. – The challenge is to • Recognize the “conditional independences” between the variables, and exploit them to get by with as few input probabilities as possible and • Use the assessed probabilities to compute the probabilities of the user queries efficiently.
Propositional Probabilistic Logic
t, ry n oi ue j q ll u Y f N e th r A w we o n ns k u na o y a If u c Yo
P(CA & TA) = P(CA) = P(TA) = P(CA V TA) = P(CA|~TA) = TA ~TA CA 0. 04 0. 06 ~CA 0. 01 0. 89
P(CA & TA) = 0. 04 P(CA) = 0. 04+0. 06 = 0. 1 (conditioning over TA) TA ~TA CA 0. 04 0. 06 ~CA 0. 01 0. 89 P(TA) = 0. 04+0. 01= 0. 05 P(CA V TA) = P(CA) + P(TA) – P(CA&TA) = 0. 1+0. 05 -0. 04 = 0. 11 P(CA|~TA) = P(CA&~TA)/P(~TA) = 0. 06/(0. 06+. 89) =. 06/. 95=. 0631
ny : a m om s e r l e o b t b Pro eed ers… num ess --N umb ded o ass n nee er t e ard h T -- re h a
Relative ease/utility of Assessing various types of probabilities • • • Joint distribution requires us to assess probabilities of type P(x 1, ~x 2, x 3, …. ~xn) This means we have to look at all entities in the world and see which fraction of them have x 1, ~x 2, x 3…. ~xm true Difficult experiment to setup. . • Conditional probabilities of type P(A|B) are relatively much easier to assess – You just need to look at the set of entities having B true, and look at the fraction of them that also have A true • Among the conditional probabilities, causal probabilities of the form P(effect|cause) are better to assess than diagnostic probabilities of the form P(cause|effect) – Causal probabilities tend to me more stable compared to diagnostic probabilities – (for example, a text book in dentistry can publish P(TA|Cavity) and hope that it will hold in a variety of places. In contrast, P(Cavity|TA) may depend on other fortuitous factors—e. g. in areas where people tend to eat a lot of icecream, many tooth aches may be prevalent, and few of them may be actually due to cavities.
Get by with easier to assess numbers A be Anthrax; Rn be Runny Nose P(A|Rn) = P(Rn|A) P(A)/ P(Rn)
Can we avoid assessing P(S)? P(M|S) = P(S|M) P(M)/P(S) P(~M|S) = P(S|~M) P(~M)/P(S) --------------------------------1 = 1/P(S) [ P(S|M) P(M) + P(S|~M) P(~M) ] So, if we assess P(S|~M), then we don’t need to assess P(S) “Normalization”
What happens if there are multiple symptoms…? Patient walked in and complained of toothache You assess P(Cavity|Toothache) Conditional independence To the rescue Suppose P(TA, Catch|cavity) = P(TA|Cavity)*P(Catch|Cavity) Now you try to probe the patients mouth with that steel thingie, and it catches… How do we update our belief in Cavity? P(Cavity|TA, Catch) = P(TA, Catch| Cavity) * P(Cavity) P(TA, Catch) = a P(TA, Catch|Cavity) * P(Cavity) Need to know this! If n evidence variables, We will need 2 n probabilities!
Generalized bayes rule P(A|B, e) = P(B|A, e) P(A|e) P(B|e)
20 th November
R w e i ev
R w e i ev
In N dep G on- en iv de d en sc en ju ed ce st an fr th ts om ep h ar old en s ts Markov Blanket Each node is conditionally independent of all others given its Markov Blanket: Parents+Children’s parents
Inefficient (redundant) computations in Enumeration Repeated multiplications
Alarm Burglary Earthquake P(A|J, M) =P(A)? How many probabilities are needed? 12 for the new; 10 for the old Is this the worst?
Constructing Belief Networks: Summary • [[Decide on what sorts of queries you are interested in answering – This in turn dictates what factors to model in the network • Decide on a vocabulary of the variables and their domains for the problem – Introduce “Hidden” variables into the network as needed to make the network “sparse” • Decide on an order of introduction of variables into the network – Introducing variables in causal direction leads to fewer connections (sparse structure) AND easier to assess probabilities • Try to use canonical distributions to specify the CPTs – Noisy-OR – Parameterized discrete/continuous distributions • Such as Poisson, Normal (Gaussian) etc
Constructing Belief Networks: Summary • [[Decide on what sorts of queries you are interested in answering – This in turn dictates what factors to model in the network • Decide on a vocabulary of the variables and their domains for the problem – Introduce “Hidden” variables into the network as needed to make the network “sparse” • Decide on an order of introduction of variables into the network – Introducing variables in causal direction leads to fewer connections (sparse structure) AND easier to assess probabilities • Try to use canonical distributions to specify the CPTs – Noisy-OR – Parameterized discrete/continuous distributions • Such as Poisson, Normal (Gaussian) etc
Case Study: Pathfinder System • Domain: Lymph node diseases – Deals with 60 diseases and 100 disease findings • Versions: – – – Pathfinder I: A rule-based system with logical reasoning Pathfinder II: Tried a variety of approaches for uncertainity • Simple bayes reasoning outperformed Pathfinder III: Simple bayes reasoning, but reassessed probabilities – Parthfinder IV: Bayesian network was used to handle a variety of conditional dependencies. • Deciding vocabulary: 8 hours • Devising the topology of the network: 35 hours • Assessing the (14, 000) probabilities: 40 hours – Physician experts liked assessing causal probabilites • Evaluation: 53 “referral” cases – Pathfinder III: 7. 9/10 – Pathfinder IV: 8. 9/10 [Saves one additional life in every 1000 cases!] – A more recent comparison shows that Pathfinder now outperforms experts who helped design it!!
- Slides: 39