Artificial Intelligence Representation and Problem Solving Probabilistic Reasoning
Artificial Intelligence: Representation and Problem Solving Probabilistic Reasoning (1): Probability Model 15 -381 / 681 Instructors: Fei Fang (This Lecture) and Dave Touretzky feifang@cmu. edu Wean Hall 4126
Recap �What we have learned so far… � Search & Satisfiability in discrete space � Basic optimization in continuous and discrete space � Deterministic / Symbolic Reasoning �All focusing on deterministic settings! � Known environment � Full observability � Deterministic change 7 Fei Fang 10/2/2020
This section �Reason about uncertainty � Signature of real-world scenarios Will it rain tomorrow? https: //www. flickr. com/photos/blackplastic/4520500 953 8 Fei Fang 10/2/2020
This section �Reason about uncertainty � Signature of real-world scenarios � Partial observability Is it raining in DC now? 9 Fei Fang 10/2/2020
This section �Reason about uncertainty � Signature of real-world scenarios � Partial observability � Sometimes, introduce uncertainty to the model to abstract how the world really works https: //pixabay. com/en/atom-science-research-physics-1013638/ 10 Fei Fang https: //en. wikipedia. org/wiki/Universe 10/2/2020
This section �Reason about uncertainty � Signature of real-world scenarios � Partial observability � Sometimes, introduce uncertainty to the model to abstract how the world really works � Need probabilistic models for knowledge representation & reasoning � Provide a way of summarizing the uncertainty that comes from our laziness and ignorance 11 Fei Fang 10/2/2020
Outline �Probability Model and Probabilistic Inference �Chain Rule, Independence, Bayes’ Rule 12 Fei Fang 10/2/2020
Probability Statement � What is the knowledge state in this statement? 13 Fei Fang 10/2/2020
Quiz 1 � I will roll a die four times; I win if I get a 1 14 Fei Fang 10/2/2020
From Gambling to Probability Theory � I will roll two dice 24 times; I win if I get a double 1 16 Fei Fang 10/2/2020
Basic Probability Notation � I will roll a die four times; I win if I get a 1 17 Fei Fang 10/2/2020
Basic Probability Notation � I roll two dice one by one, the first die shows 1. What is the probability that I get a double 1? 18 Fei Fang 10/2/2020
Basic Probability Notation � I roll two dice one by one, the first die shows 1. What is the probability that I get a double 1? 20 Fei Fang 10/2/2020
Quiz 2 � Bag 1: two gold coins. Bag 2: two pennies. Bag 3: one of each. � Bag is chosen at random, and one coin from it is selected at random; the coin is gold � What is the probability that the other coin is gold given the observation? � A: 1/6 � B: 1/3 � C: 2/3 � D: 1/2 21 Fei Fang 10/2/2020
Basic Rules � Sample space 24 Fei Fang 10/2/2020
Basic Rules � 26 Fei Fang 10/2/2020
Probability Model with Factored Representation � I will roll a die; I win if I get a 1 28 Fei Fang 10/2/2020
Probability Model with Factored Representation � I will roll a die four times; I win if I get a 1 30 Fei Fang 10/2/2020
Probability Model with Factored Representation �A possible world is defined to be an assignment of values to all the random variables under consideration �A probability model is completely determined by the full joint probability distribution – the joint distribution of all the random variables under consideration 31 Fei Fang 10/2/2020
Joint Probability Distribution 32 . 05 . 2 0 . 1 0 0 . 1 . 05 . 1 0 Fei Fang 10/2/2020
Probabilistic Inference � May be none 34 Fei Fang 10/2/2020
Compute Marginal Probability � 35 Fei Fang . 05 . 2 0 . 1 0 0 . 1 . 05 . 1 0 10/2/2020
Marginal Probability Distribution � 37 Fei Fang 10/2/2020
Marginal Probability Distribution � 39 Fei Fang 10/2/2020
Marginal Probability Distribution 41 . 05 . 2 0 . 1 0 0 . 1 . 05 . 1 0 Fei Fang 10/2/2020
Compute Conditional Probability � 43 Fei Fang . 05 . 2 0 . 1 0 0 . 1 . 05 . 1 0 10/2/2020
Conditional Probability Distribution � Note: This is not matrix division! It represents a set of equations! 45 Fei Fang 10/2/2020
Conditional Probability Distribution � 47 Fei Fang 10/2/2020
Revisit Quiz 2 � A B 48 Fei Fang C D E F 10/2/2020
Revisit Quiz 2 �What is the probability that the other coin is gold given the observation? A B 49 Fei Fang C D E F 10/2/2020
Outline �Probability Model and Probabilistic Inference �Chain Rule, Independence, Bayes’ Rule 50 Fei Fang 10/2/2020
Chain Rule � Note: This is not matrix multiplication! It represents a set of equations! 51 Fei Fang 10/2/2020
Independence � I will roll a die tomorrow; What is the probability of getting a 1 and there is no rain? I will roll two dice one by one; What is the probability of first getting a 1 and then getting a 6? 53 Fei Fang 10/2/2020
Independence � 55 Fei Fang 10/2/2020
Quiz 1 revisited �Antoine Gombaud (1607 -1684) I will roll a die four times; I win if I get a 1 �What is the probability of winning? 56 Fei Fang 10/2/2020
Conditional Independence � 58 Fei Fang 10/2/2020
Bayes’ Rule � 62 Fei Fang 10/2/2020
Bayes’ Rule � 64 Fei Fang 10/2/2020
Monty Hall Problem �Behind one door is a car; behind the other two are goats. Which door has the car is randomly assigned. You pick a door. Then announcer randomly opens another door that has a goat behind it and asks you whether you want to switch before he opens the door you pick. Should you stay with your choice or switch? Steve Selvin 66 Fei Fang 10/2/2020
Monty Hall Problem �Stay or switch? 67 Fei Fang 10/2/2020
Monty Hall Problem � 68 Fei Fang 10/2/2020
Quiz 3 � Behind one door is a car; behind the other four are goats. Which door has the car is randomly assigned. You pick a door. Then announcer randomly opens another door that has a goat behind it and asks you whether you want to switch before he opens the door you pick. 70 Fei Fang 10/2/2020
Summary � Probability Model Random variable, Domains � Joint probability distribution � � Probabilistic Inference Use full joint probability distribution as the “knowledge base” � Compute � Marginal probability � Conditional probability � � Chain Rule, Independence, Bayes’ Rule � Challenge? � 72 Full joint distribution is hard to estimate and too big to represent explicitly Fei Fang 10/2/2020
Acknowledgment �Some slides are borrowed from previous slides made by Tai Sing Lee 73 Fei Fang 10/2/2020
Backup Slides Fei Fang
Outline �Basics in probability theory 75 Fei Fang 10/2/2020
Concepts and Definitions � 76 Fei Fang 10/2/2020
Concepts and Definitions � 77 Fei Fang 10/2/2020
Concepts and Definitions � 78 Fei Fang 10/2/2020
The Sum Rule and Marginal Probability � 79 Fei Fang 10/2/2020
Conditional Probability � 80 Fei Fang 10/2/2020
The Chain Rule � 81 Fei Fang 10/2/2020
Continuous Variables � 82 Fei Fang 10/2/2020
Gaussian Distribution / Normal Distribution � 83 Fei Fang 10/2/2020
Expectation and Variance � 84 Fei Fang 10/2/2020
Covariance � 85 Fei Fang 10/2/2020
Covariance Matrix � 86 Fei Fang 10/2/2020
Central Limit Theorem � 87 Fei Fang 10/2/2020
- Slides: 58