STATISTICS Probability Professor KeSheng Cheng Department of Bioenvironmental

STATISTICS Probability Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering/Master Program in Statistics National Taiwan University 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 1

Random Experiment and Sample Space • An experiment that can be repeated under the same (or uniform) conditions, but whose outcome cannot be predicted in advance, even when the same experiment has been performed many times, is called a random experiment. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 2

• Examples of random experiments • Tossing a coin. • Rolling a die. • The selection of a numbered ball (1 -50) in an urn. (selection with replacement) • Occurrences of earthquakes • The time interval between the occurrences of two consecutive higherthan-scale 6 earthquakes. • Occurrences of typhoons • The amount of rainfalls produced by typhoons in one year (yearly typhoon rainfalls). 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 3

• The following items are always associated with a random experiment: • Sample space. The set of all possible outcomes, denoted by . • Outcomes. Elements of the sample space, denoted by . These are also referred to as sample points or realizations. • Events. An event is a subsets of for which the probability is defined. Events are denoted by capital Latin letters (e. g. , A, B, C). 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 4

Definition of Probability • Classical probability • Frequency probability • Probability model 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 5

Classical (or a priori) probability • If a random experiment can result in n mutually exclusive and equally likely outcomes and if n. A of these outcomes have an attribute A, then the probability of A is the fraction n. A/n. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 6

• Example 1. Compute the probability of getting two heads if a fair coin is tossed twice. (1/4) • Example 2. The probability that a card drawn from an ordinary well-shuffled deck will be an ace or a spade. (16/52) 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 7

Remarks • The probabilities determined by the classical definition are called “a priori” probabilities since they can be derived purely by deductive reasoning. • The “equally likely” assumption requires the experiment to be carried out in such a way that the assumption is realistic; such as, using a balanced coin, using a die that is not loaded, using a well-shuffled deck of cards, using random sampling, and so forth. This assumption also requires that the sample space is appropriately defined. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 8

• Troublesome limitations in the classical definition of probability: • If the number of possible outcomes is infinite; • If possible outcomes are not equally likely. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 9

Relative frequency (or a posteriori) probability • We observe outcomes of a random experiment which is repeated many times. We postulate a number p which is the probability of an event, and approximate p by the relative frequency f with which the repeated observations satisfy the event. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 10

• Suppose a random experiment is repeated n times under uniform conditions, and if event A occurred n. A times, then the relative frequency for which A occurs is fn(A) = n. A/n. If the limit of fn(A) as n approaches infinity exists then one can assign the probability of A by: P(A)=. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 11

• This method requires the existence of the limit of the relative frequencies. This property is known as statistical regularity. This property will be satisfied if the trials are independent and are performed under uniform conditions. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 12

• Example 3 A fair coin was tossed 100 times with 54 occurrences of head. The probability of head occurrence for each toss is estimated to be 0. 54. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 13

• The chain of probability definition Random experiment 1/8/2022 Sample space Event space Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University Probability space 14

Probability Model Each outcome can be thought of as a sample point, or an element, in the sample space. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 15

• Event and event space • An event is a subset of the sample space. The class of all events associated with a given random experiment is defined to be the event space. • An event will always be a subset of the sample space, but for sufficiently large sample spaces not all subsets will be events. Thus the class of all subsets of the sample space will not necessarily correspond to the event space. • If the sample space consists of only a finite number of points, then the corresponding event space will be the class of all subsets of the sample space. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 16

1) (the empty set) and (the sure event) are both subsets of . 2) An event A is said to occur if the experiment at hand results in an outcome that belongs to A. 3) An event space is usually denoted by a script Latin letter such as A and B. 4) Two events A and B are said to be mutually exclusive if and only if. Events are mutually exclusive if and only if. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 17

Event space and algebra of events • Let A denote an event space, the following properties are called the Boolean algebra, or algebra of events: 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 18

Probability function • Let denote the sample space and A denote an algebra of events for some random experiment. Then, a probability function P is a set function with domain A (an algebra of events) and counter domain the interval [0, 1] which satisfies the following axioms: 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 19

• Probability is a mapping (function) of sets to numbers. • Probability is not a mapping of the sample space to numbers. • The expression is not defined. However, for a singleton event , is defined. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 20

Probability space • A probability space is the triplet ( , A, P[ ]), where is a sample space, A is an event space, and P[ ] is a probability function with domain A. • A probability space constitutes a complete probabilistic description of a random experiment. • The sample space defines all of the possible outcomes, the event space A defines all possible things that could be observed as a result of an experiment, and the probability P defines the degree of belief or evidential support associated with the experiment. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 21

Finite Sample Space • A random experiment can result in a finite number of possible outcomes. A sample space with only a finite number of elements (points) is called a finite sample space. • Finite sample space with equally likely points – simple sample space • Finite sample space without equally likely points 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 22

Conditional probability 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 23

Bayes’ theorem 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 24

Multiplication rule 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 25

Independent events 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 26

• The property of independence of two events A and B and the property that A and B are mutually exclusive are distinct, though related, properties. • If A and B are mutually exclusive events then AB=. Therefore, P(AB) = 0. Whereas, if A and B are independent events then P(AB) = P(A)P(B). Events A and B will be mutually exclusive and independent events only if P(AB)=P(A)P(B)=0, that is, at least one of A or B has zero probability. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 27

• But if A and B are mutually exclusive events and both have nonzero probabilities then it is impossible for them to be independent events. • Likewise, if A and B are independent events and both have nonzero probabilities then it is impossible for them to be mutually exclusive. 1/8/2022 Dept. of Bioenvironmental Systems Engineering/Master Program in Statistics, National Taiwan University 28