Artificial Intelligence Representation and Problem Solving Probabilistic Reasoning

Artificial Intelligence: Representation and Problem Solving Probabilistic Reasoning (2): Bayesian Networks 15 -381 / 681 Instructors: Fei Fang (This Lecture) and Dave Touretzky feifang@cmu. edu Wean Hall 4126

Recap �Probability � Joint Models probability distribution of random variables �Probabilistic � Compute �Chain Inference Marginal probability or Conditional probability Rule, Independence, Bayes’ Rule �Full joint distribution is hard to estimate and too big to represent explicitly 2 Fei Fang 9/19/2021

Outline �Bayesian Networks �Independence in Bayes’ Net �Construct a Bayes’ Net �Exact Inference in Bayes’ Net �Applications of Bayes’ Net 3 Fei Fang 9/19/2021

Bayesian Network � (≈ “directly influences”) Way to Commute to Work Weather Sleeping Quality 4 Fei Fang 9/19/2021

Example: Alarm �I’m at work, both of my neighbors John and Mary call to say my alarm for burglary is ringing. Sometimes it’s set off by minor earthquakes. Is there a burglary? �How do we model this scenario? How can we represent our knowledge in such a domain with uncertainty? 5 Fei Fang 9/19/2021

Example: Alarm � Joint Probability 6 T T T T T F T … … … Fei Fang … 9/19/2021

Example: Alarm � Joint Probability 8 T T T T T F T … … … Fei Fang … 9/19/2021

Example: Alarm � However, there are some intuitive independence relationships based on our causal knowledge! � Causal knowledge – �A burglary can set the alarm off � An earthquake can set the alarm off � The alarm can cause Mary to call � The alarm can cause John to call Burglar y Earthquak e Alarm Mary. Call s John. Call s 10 Fei Fang 9/19/2021

Example: Alarm � 11 Burglary (B), Earthquake (E), Alarm (A), John. Calls (J), Mary. Calls 9/19/2021

Example: Alarm �Given these independence relationships, �We don’t need fill the full joint probability table anymore to represent our knowledge! � Only need to provide these conditional probabilities � Is this better or worse? 15 Fei Fang 9/19/2021

Example: Alarm How many numbers we need here? 17 burglary (B), Earthquake (E), Alarm (A), John. Calls (J), Mary. Calls 9/19/2021

Example: Alarm Enrich the network with more links: more realistic, less compact 18 Fei Fang 9/19/2021

Bayesian Network �Bayesian Network: A compact way to represent knowledge in an uncertain domain Joint Probability 20 T T T T T F T … … … Fei Fang … 9/19/2021

Bayesian Network � Global semantics Joint Probability T T T T T F T … … … 21 … Have to be equivalent! Fei Fang 9/19/2021

Bayesian Network � Joint Probability T T T T T F T … … … 22 … Have to be equivalent! Fei Fang 9/19/2021

Bayesian Network � A Bayes’ Net = topology (graph) + local conditional probabilities 24 Fei Fang 9/19/2021

Quiz 1 �At Rainy/Snow 0. 2 y Sunny 0. 3 Other 0. 5 Weather Way to Commute to Work Sleeping Quality least how many entries are needed for a general CPT (conditional probability table) for the node “Way to Commute to Work”? � A: High 0. 2 Low 0. 3 25 18 � B: 12 � C: 6 � D: 3 Fei Fang 9/19/2021

Quiz 1 Rainy/Snow 0. 2 y R/S H Bus R/S H Walk 0. 25 Sunny 0. 3 R/S L Bus Other 0. 5 R/S L Walk 0. 15 Weather Way to Commute to Work Sleeping Quality 0. 7 0. 8 Sunny H Bus 0. 2 Sunny H Walk 0. 5 Sunny L Bus Sunny L Walk 0. 3 Other H Bus Other H Walk 0. 2 0. 3 0. 4 High 0. 2 Other L Bus Low 0. 3 Other L Walk 0. 2 26 Fei Fang 0. 6 9/19/2021

Another Perspective of Bayes’ Net � 27 Fei Fang 9/19/2021

Outline �Bayesian Networks �Independence in Bayes’ Net �Construct a Bayes’ Net �Exact Inference in Bayes’ Net �Applications of Bayes’ Net 28 Fei Fang 9/19/2021

Independence in Bayes’ Net �Given a Bayes’ Net, which variables are independent? Local semantics �Each node is conditionally independent of its non -descendants given its parents 29 Fei Fang 9/19/2021

Independence in Bayes’ Net �Each node is conditionally independent of all others given its Markov blanket: parents + children’s parents 30 Fei Fang 9/19/2021

Example � Local Semantics: Each node is conditionally independent of its nondescendants given its parents Each node is conditionally independent of all others given its Markov blanket: parents + children’s parents 31 Fei Fang 9/19/2021

Quiz 2 � B C A D F Local Semantics: Each node is conditionally independent of its non-descendants given its parents Each node is conditionally independent of all others given its Markov blanket: parents + children’s parents 32 Fei Fang E G H 9/19/2021

Outline �Bayesian Networks �Independence in Bayes’ Net �Construct a Bayes’ Net �Exact Inference in Bayes’ Net �Applications of Bayes’ Net 34 Fei Fang 9/19/2021

Is Bayes’ Net Expressive Enough? �Any full joint probability table can be represented by a Bayes’ Net 35 Fei Fang 9/19/2021

Is Bayes’ Net Unique? �One (full joint probability distribution)-to-many (Bayes’ Net) mapping 37 Fei Fang 9/19/2021

Construct a Bayes’ Net �Construct a (ideally simple) Bayes’ Net systematically As a knowledge engineer or domain expert 38 Fei Fang 9/19/2021

Construct a Bayes’ Net � Joint Probability T T T T T F T … … … 40 … Fei Fang 9/19/2021

Outline �Bayesian Networks �Independence in Bayes’ Net �Construct a Bayes’ Net �Exact Inference in Bayes’ Net �Applications of Bayes’ Net 41 Fei Fang 9/19/2021

Probabilistic Inference in Bayes’ Net � 42 Fei Fang 9/19/2021

General Inference Procedure � 43 Fei Fang 9/19/2021

Inference in Bayes’ Net �Inference with full joint probability distribution table available: Read the joint probability from the table �Inference in Bayes’ Net: compute joint probability through conditional probability table 45 Fei Fang 9/19/2021

Example: Alert �I’m at work, both of my neighbors John and Mary call to say my alarm for burglary is ringing. Sometimes it’s set off by minor earthquakes. Is there a burglary? 46 Fei Fang 9/19/2021

Example: Alert �Evaluate through depth-first recursion of the following expression tree Top-down DF evaluation: × Values along each path + at the branching nodes 48 Fei Fang 9/19/2021

Example: Alert �Normalize 49 Fei Fang 9/19/2021

Exact Inference in Bayes’ Net: Enumeration 51 Fei Fang 9/19/2021

Exact Inference in Bayes’ Net: Variable Elimination �Avoid repeated computation of subexpressions in the enumeration algorithm �Similar to dynamic programming 52 Fei Fang 9/19/2021

Outline �Bayesian Networks �Independence in Bayes’ Net �Construct a Bayes’ Net �Exact Inference in Bayes’ Net �Applications of Bayes’ Net 53 Fei Fang 9/19/2021

Bayes’ Net as a Model of Real World �Bayes’ Net represents knowledge in an uncertain domain �View it as a way to model the real world based on domain knowledge �Is your model (Bayes’ Net) for a real-world problem correct? Not necessarily. 54 Fei Fang 9/19/2021

Bayes’ Net as a Model of Real World �"All models are wrong“ �A common aphorism in statistics � Generally attributed to the statistician George Box "Essentially, all models are wrong, but some are useful". https: //en. wikipedia. org/wiki/All_models_are_wrong 55 Fei Fang 9/19/2021

Use of Bayes’ Net � 56 Fei Fang 9/19/2021

Use of Bayes’ Net Russel and Novig 57 Fei Fang 9/19/2021

Summary � Bayes’ Net � Graphical model � Decompose full joint probability distributions into interpretable, simple, local distributions � Independence in Bayes’ Net � Local semantics � Markov Blanket � Construct a Bayes Net � Exact Inference in Bayes’ Net � Applications of Bayes’ Net 58 Fei Fang 9/19/2021