Bayesian Methods and Subjectiv Probability Daniel Thorburn Stockholm

Bayesian Methods and Subjectiv Probability Daniel Thorburn Stockholm University 2011 -01 -10

Outline 1. 2. 3. 4. 5. 6. 7. 8. 9. Background to Bayesian statistics Two simple rules Why not design-based? Bayes, Public statististics and sampling De Finetti theorem, Bayesian Bootstrap Comparisons between paradigms Preposterior analysis Statistics in science Complementary Bayesian meethods 2

1. Background • Mathematically: – Probability is a positive, finite, normed, (s)-additive measure defined on a (s)algebra • But what does that correspond to in real life?

What is the probability of heads in the following sequence? Does it change? And when? – This is a fair coin – I am now going to toss it in the corner – I have tossed it but noone has seen the result – I have got a glimpse of it but you have not – I know the result but you don´t – I tell you the result 4

• Laplace definition. ”All outcomes are equally probable if there is no information to the contrary”. (number of positive elementary events/number of possible elementary events) • Choose heads and bet on it, with your neighbour. You get one krona if you are right and lose one if you are wrong. When should you change from indifference? • Frequency interpretation. (LLN). If there is an infinite sequence of independent experiments then the relative frequency converges a. s. towards the true value. Cannot be used as a definition for two reasons – It is a vicious circle. Independence is defined in terms of probability – It is logically impossible to define over-countably many different quantities by a countable procedure. 5

Probabilities do not exist (de Finetti) • They only describe your lack of knowledge • If there is a God almighty, he knows everything now, in the past and in the future. (God does not play dice, (Einstein)) • But lack of knowledge is personal, thus probability is subjective • Kolmogorovs axioms only does not say anything about the relation to reality 6

• Probability is the language which describes uncertainty • If you do not know a quantity you should describe your opinion in terms of probability • Probability is subjective and varies between persons and over time, depending on the background information. 7

Rational behaviour – one person • Axiomatic foundation of probability. Type: – For any two events A and B exactly one of the following must hold A < B, A > B or A v B (pronounce A as more likely than B, B more likely than A, equally likely) – If A 1, A 2, B 1 and B 2 are four events such that A 1 A 2 = B 1 B 2 is empty and A 1 > B 1 and A 2 > B 2 then A 1 U A 2 > B 1 U B 2. If further either A 1 > B 1 or A 2 > B 2 then A 1 U A 2 > B 1 U B 2 – … • If these axioms hold all events can be assigned probabilities, which obey Kolmogorovs axioms (Villegas, Annals Math Stat, 1964), • Axioms for behaviour. Type … – If you prefer A to B, and B to C then you must also prefer A to C – … • If you want to behave rationally, then you must behave as if all events were assigned probabilities (Anscombe and Aumann, Annals Math Stat, 1963)

• Axioms for probability (these six are enough to prove that a probability following Kolmogorovs axioms can be defined plus the definition of conditional probability) – For any two events A and B exactly one of the following must hold A < B, A > B or A v B (pronounce A as more likely than B, B more likely than A, equally likely) – If A 1, A 2, B 1 and B 2 are four events such that A 1 A 2 = B 1 B 2 is empty and A 1 > B 1 and A 2 > B 2 then A 1 U A 2 > B 1 U B 2. If further either A 1 > B 1 or A 2 > B 2 then A 1 U A 2 > B 1 U B 2 – If A is any event then A > (the impossible (empty) event) – If Ai is an strictly decreasing sequence of events and B a fixed event such that Ai > B for all i then (the intersection of all A ii) > B – There exists one random variable which has a uniform distribution – For any events A, B and D, (A|D) < (B|D) if and only if AD < BD • Then one needs some axioms about comparing outcomes, (utilities) in order to be able to prove rationality… 9

• Further one needs some axioms about comparing outcomes, (utilities) in order to be able to prove rationality – For any two outcomes, A and B, one either prefers A to B or B to A or is indifferent – If you prefer A to B, and B to C then you must also prefer A to C – If P 1 and P 2 are two distributions over outcomes they may be compared and you are indifferent between A and the distribution with P(A)=1 – Two measurability axioms like • If A is any outcome and P a distribution the event that P gives an outcome preferred two A can be compared to other events (more likely …) – If P 1 is preferred to P 2 and A is an event, A > 0, then the game giving P 1 if A occurs is preferred to the game giving P 2 under A if the results under the not-A are the same. – If you prefer P 1 to P and P to P 2, then there exists numbers a>0 and b>0 such that P 1 with probability 1 -a and P 2 with probability a is preferred to P, which is preferred to P 1 with probability b and P 2 with probability 1 -b. 10

There is only one type of numbers, which may be known or unknown. • Classical inference has a mess of different types of numbers e. g. – – – – Parameters Latent variables like in factor analysis Random variables Observations Independent (explaining) variables Dependent variables Constants a. s. o. • Superstition!

2. Two simple requirements for rational inference 12

Rule 1 • What you know/believe in advance + The information in the data = What you know/believe afterwards 13

Rule 1 • What you know/believe in advance + The information in the data = What you know/believe afterwards • This is described by Bayes’ Formula: • P(q|K) * P(X|q, K) a P(q|X, K) 14

Rule 1 • What you know/believe in advance + The information in the data = What you know/believe afterwards • This is described by Bayes’ Formula: • P(q|K) * P(X|q, K) a P(q|X, K) • or in terms of the likelihood • P(q|K) * L(q|X) a P(q|X, K) 15

Rule 1 corrolarium • What you believe afterwards + information in a new study = you believe after both studies the What 16

Rule 1 corrolarium • What you believe afterwards + information in a new study = you believe after both studies the What • The result of the inference should be possible to use as an input to the next study • It should thus be of the same form! • Note that hypothesis testing and confidence intervals can never appear on the left hand side so they do not follow rule 1 17

Rule 2 • Your knowledge must be given in a form that can be used for deciding actions. (At least in a well-formulated problem with well-defined losses/utility). 18

Rule 2 • Your knowledge must be given in a form that can be used for deciding actions. (At least in a well-formulated problem with well-defined losses/utility). • If you are rational, you must use the rule which minimizes expected ”losses” (maximizes utility) • Dopt = argmin E(Loss(D, q )|X, K) = argmin X Loss(D, q) P(q |X, K) dq 19

Rule 2 • Your knowledge must be given in a form that can be used. (At least in a well-formulated problem with well-defined losses/utility. • If you are rational, you must use the rule which minimizes expected ”losses” (maximizes utility) • Dopt = argmin E(Loss(D, q )|X, K) = argmin X Loss(D, q) P(q |X, K) dq • Note that classical design-based inference has no interface with decisions. 20

Statistical tests are useless • They cannot be used to combine with new data. • They cannot be used even in simple decision problems. • They can be compared to the blunt plastic knife given to a three year old child – He cannot do much sensible with it – But he cannot harm himself either

3. An example of the stupidity of frequency-based (design-based) methods N=4, n=2, SRS. Dichotomous data, black or white. The variable is known to come in pairs, i. e. the total is T=0, 2 or 4. Probabilities: Populationoutcome 0 white 1 white No white T=0 1 2 white T=2 1/6 All white T=4 2/3 2 white 1/6 1 If you observe 1 white you know for sure that the population contains 2 white. If you observe 0 or 2 you white the only unbiased estimate is T*= 0 resp. 4 The variance of this estimate is 4/3 if T=2 (=1/6*4+4/6*0+1/6*4) and 0 if T=0 or 4 So if you know the true value the design-based variance is 4/3 and if you are uncertain the design-based variance is 0. (Standard unbiased variance 22 estimates are 2 resp. 0)

Bayesian analysis works OK – We saw the Bayesian analysis when t=1, (T*=2). – If all possibilities are equally likely à priori, the posterior estimates of T when t = 0 (2) is T* = 2/7 (26/7) and the posterior variance is 24/49. 23

Always stupid? • It is always stupid to believe that the variance of an estimator is a measure of precision in one particular case. (It is defined as a long run property for many repetitions) • But it is not always so obvious and so stupid as in this example. • Is this a consequence of the unusual prior with T must be even? 24

Example without the prior info Still stupid but not quite as much outcome 0 population 0 1 2 3 4 Var(q|X) 6/6 3/6 1/6 9/20 1 3/6 4/6 3/6 6/10 2 1/6 3/6 6/6 9/20 Var(q*|q)= Var(2 T|q) 0 1 4/3 1 0 25

Example without the prior info Still stupid but not quite as much outcome 0 population 0 1 2 3 4 Var(q|X) 6/6 3/6 1/6 9/20 1 3/6 4/6 3/6 6/10 2 1/6 3/6 6/6 9/20 Var(q*|q)= Var(2 T|q) 0 1 4/3 1 0 If you observe 1, the true error is never larger than 1, but the standard deviation is always larger than 1 for all possible parameter values. 26

Always stupid? • It is always stupid to assume that the variance of an estimator is a measure of precision in one particular case. (It is defined as a long run property for many repetitions) • But it is not always so obvious and stupid as in these examples. • Under suitable regularity conditions designbased methods are asymptotically as efficient as Bayesian methods 27

• Many people say that one should choose the approach that is best for the problem at hand. Classical or Bayesian. 28

• Many people say that one should choose the approach that is best for the problem at hand. Classical or Bayesian. • So do Bayesians. • But they also draw the conclusion: 29

• Many people say that one should choose the approach that is best for the problem at hand. Classical or Bayesian. • So do Bayesians. • But they also draw the conclusion: • Always use Bayesian methods! 30

• Many people say that one should choose the approach that is best for the problem at hand. Classical or Bayesian. • So do Bayesians. • But they also draw the conclusion: • Always use Bayesian methods! • Classical methods can sometimes be seen as quick and dirty approximations to Bayesian methods. • Then you may use them. 31

4. What is special for many statistical surveys, e. g. public statistics? 32

3. What is special for many statistical surveys, e. g. public statistics? • Answer 1 The producer of the survey is not the user. – Often many readers and many users. – The producer has no interest in the figures par se • P( q |Kuser) is not known to the producer and not unique P(q| Kuser) * L(q|X) a P(q|X, Kuser) • Solution? 33

3. What is special for many statistical surveys, e. g. public statistics? • Answer 1 The producer of the survey is not the user. – Often many readers and many users. – The producer has no interest in the figures par se • P( q |Kuser) not known to the producer and not unique P(q| Kuser) * L(q|X) a P(q|X, Kuser) • Publish L( q |X) so that any reader can plug in his prior • Usually given in the form of the posterior with a vague, uninformative (often = constant) prior L(q|X) t P(q|K 0) * L(q|X) a P(q|X, K 0) 34

Describing the likelihood • Estimates are often asymptotically normal. Then it is enough to give the posterior mean and variance or a (symmetric) 95% prediction interval (for large samples) • When the maximum likelihood estimator is approximately efficient and normal the MLestimate and inverse Fisher information are enough. (t standard confidence interval) • Asymptotically efficient t for large samples almost as good as Bayesian estimates, which are known to be admissible also for finite samples

What is special for many statistical surveys, e. g. public statistics? • Answer 2 There is no parameter or more exactly: The parameter consists of all the N values of all the units in the population. 36

What is special for many statistical surveys, e. g. public statistics? • Answer 2 There is no parameter or more exactly The parameter consists of all the N values of all the units in the population. • Use this vector as the parameter q in Bayes’ formula. • If you are interested in a certain function, e. g. the total T, integrate out all nuisance parameters in the posterior, to get the marginal of interest P(YT|X, K) = X…XS qi = YT P(q 1, … , q. N |X, K) P 1 N-1 dqi 37

5. De Finetti’s theorem • Random variables are said to be exchangeable if there is no information in the ordering. This is for instance the case with SRS • If a sequence of random variables is infinitely exchangeable than they can be described as independent variables given q, where q is a latent random variable. (The proof is simple but needs some knowledge of random processes. Formally q is defined on the tail s-algebra) • Latent means in this case that it does not exist but can be useful when desscribing the distribution. 38

• This imaginary random variable can take the place of a parameter • But note that it does not exist (is not defined) until the full infinite sequence has been defined and the full sequence will never be observed. • Note also that most sequences in the real world are not independent but only exchangeable. If you toss a coin 1000 times and get 800 heads it is more likely that the next toss will be heads (compared to the case with 200 heads). • So obviously there is a dependence between the first 1000 tosses and the 1001 st 39

Dichotomous variables or The Polya Urn scheme – – In an urn there is one white and one black ball. Draw one ball at random. Note its colour. Put it back together with one more ball of the same colour Draw one at random … • This sequence can be shown to be exchangeable and it can by de Finetti’s theorem be described as – Take q € U(0, 1) = Beta(1, 1) – Draw balls independently with this probability of being white • There is no way to see the difference between a Bernoulli sequence (binomial distribution) with an unknown p and a Polya urn scheme. Since the outcomes follow the same distribution there cannot exist any test to differentiate between them. 40

Dichotomous variables or The Polya Urn scheme • We could have started with another number of balls. This had given other parameters in the prior Beta-distribution • Beta(a, b) a white balls and b black balls • E(Beta) = a /(a + b) • Var(Beta) = ab /((a + b)2(a + b+1)) 41

Dichotomous variables or The Polya Urn scheme • This can be used to derive the posterior distribution of the number (y. T) of balls/persons with a certain property (white) in a population, given an observed SRS-sample of size n with y. S white balls/persons. • Use a prior with parameters so that the expected value is your best guess of the unknown proportion and the standard deviation describes your uncertainty about it. 42

Properties • The posterior distribution can be shown to be • With both parameters set to 0, the expected value is Np* and the variance p*(1 -p*)N(N-n)/(n+1). • The designbased estimate and variance estimator are good approximations to this (equal apart from n in place of n+1) 43

Simulation • It is often easier to simulate from the posterior than to give its exact form. • In this case the urn scheme gives a simple way to simulate the full population. Just continue with the Polya sampling scheme starting from the sample • If you repeat this 1000 times, say, and plot the 1000 simulated population totals in a histogram, you will get a good description of the distribution of the unknown quantity 44

Dirichlet-multinomial • If the distribution is discrete with a finite number of categories, a similar procedure is possible • Just draw from the set of all observations and put it back together with a similar observation. Continue until N • Repeat and you get a number of populations which are drawn from the posterior distribution. • For each population compute the parameter of interest, e. g. the mean or median, and plot the values in a histogram • If this is described as in de Finetti’s Theorem, the parameter comes from a Dirichlet distribution and the observations are conditionally independent multinomial. 45

The Bayesian Bootstrap • This procedure is called the Bayesian bootstrap (if an uninformative prior i. e. all parameters = 0 is used) • This can be generalised to variables measured on a continuous scale • The design-based estimate gives the same mean estimate as this (for polytomous populations). • The design-based variance estimator is also close to the true variance apart from a factor n/(n+1) • Note, that if the distribution is skew, this method does not work well, since it does not use the prior information of skewness (nor does the designbased methods) • Note also that with many categories it may be better to use even smaller parameters e. g. – 0. 9. 46

Other Bayesian models • There are many other models/methods within Bayesian survey sampling, than the Bayesian Bootstrap • Another approach starts with a normal-gamma model – Given m, s 2, data come from an iid normal(m, s 2) model – The variance s 2 follows a priori an inverse gamma – The mean m follows a priori a normal model with mean m and variance ks 2 • and later relaxes the normality assumption • but I have not enough time here. 47

6. Properties of some different paradigms within survey sampling Design-based Model-based Bayesian Uncertainty, Randomness Home-made Given by nature, Subjective, ratfrequencybased ionality axioms Main focus Population Parameters Population values Unknown unobservable Do not exist, but useful, de Finetti Inference Long run properties based This case, probability based Output Point estimates intervals Point estimates confidence intervals Full posterior distributions, means, variance Possible use Not my problem Interface with decisions 48

7. Preposterior analysis Study/experimental design • In the design of a survey one must take into account the posterior distribution. • You may e. g. want to – Get a small posterior variance – Get a short 95 % prediction interval – Make a good decision • This analysis of the possible posterior consequences before the experiment is carried out, is called preposterior analysis 49

Preposterior analys with public statistics • Usually when you make a survey for your own benefit you should use your own prior both in the preposterior and the posterior analysis • With public statistics you should have a flat prior in the posterior analysis, – e. g. the posterior variance is Var(q |X, K 0). • But the design decision is yours and you should use all your information for that decision – e. g. find the design, which minimizes E(Var(q |X, K 0)| KYou ) 50

Example: Neyman allocation; Dichotomous data • M strata with Nm elements. Unknown proportion in stratum m is pm. • How many elements should be drawn from each stratum in order to estimate the average proportion best? • Neyman: Chose nm a Nm(pm(1 -pm))1/2. • But pm is unknown. Classical people: Use your best subjective guess pm 0 • Neyman: Chose nm a Nm (pm 0(1 -pm 0))1/2.

Bayesians: Do not use a one point prior. It is to subjective! Take also your prior uncertainty into account! • Chose e. g. the prior Beta(am, bm). m<M – where pm 0 = am/(am+bm) – and Var(pm) = ambm/((am+bm)2(am+bm+1))

Example: Optimal allocation • M strata, size Nm , dichotomous data, independent priors (am, bm) (as we saw above). Posterior variance: • The expected value of this is • Minimising this gives approximately the sample sizes • The terms (am +bm) on the left hand side should not be there in the case of public statistics (c cost) • This differs from Neyman-allocation since it takes the prior uncertainty of the proportions into account 53

8. Statistics in science Science is more complicated • One may divide it into (at least) three phases 1. Exploratory 2. Trying to get a good picture, convincing yourself 3. Proving the fact - convincing others • These phases may require different approaches and priors 1. Your own prior but critical 2. . 1. During work: Your own prior and often informative based on theory, arguments, experience or the exploration. 2. In the presentation: Usually vague prior. 3. Other possible priors (but use also vague)

8. 1 Exploratory • Sometimes called hypothesis generating • Most theories are false. Most substances are useless against cancer. • Use your own priors, which most often say that all facts are most unlikely. Some examples – Screening – all substances have a probability of 0. 001 to have some effect – Regression situations with an abundance (M) of explaining variables. If all variables are ordered after importance all M! orderings are equally likely. Given the ordering the m: th regression coefficient is N(0, 1/(m-1)2) (after standardisation of X) (another possibility is that reduction in unexplained variance 1 -R 2 is Beta(1, m 2)) • When there is a support from theory or previous experiences other priors may be used

8. 2 Getting a good picture yourself (Assuming that you are the scientist) • In classical terms this is the phase when you can formulate the hypotheses that you want to test. (Your prior is strong enough to formulate hypotheses) • (In classical theory there is no description of the first phase. Mostly one said that: if you can formulate a hypothesis you may test it, as long as you do not formulate to many) • Your priors should still be your own.

• The reporting in this phase is quite similar to what was said about official statistics. I. e. try to give a good picture of the likelihood function • But, contrary to public statistics, in the design of experiments it is your posterior precision that should be maximised. (It is assumed that you are an expert in the field and it is no reason to believe that your opinion is far from the present state of knowledge)

8. 3 Proving scientific facts • It is very easy to convince people who believe in the fact from the beginning. • It is often fairly simple to convince yourself even if you are broadminded • But to prove a scientific fact you must convince also those that have reasonable doubts.

Proving scientific facts • P(q|X, K) a P(X|q, K) *P(q|K) • A person is convinced of a fact when his posterior probability is close to one for the fact. • But to prove the fact scientifically this must hold for all reasonable priors including those describing reasonable doubt. • Even if there is no such person this must hold also for that prior as long as it is reasonable • I. e. a result is ”proved” if inf (P(q|X, K); K reasonable) > 1 – a for some a.

• Reporting: Use vague priors, but also show what the consequences are for some priors with (un-) reasonable doubt. • When you prove something all available data should be used. Type: Meta analysis. In some fields one study is ususall not enough to convince people • Designing experiments: Design your experiments so that you maximise E(inf (P(q|X, K); K reasonable) | KYOU) (if you are convinced).

What is reasonable doubt? Convincing others • You have to contemplate what is meant by reasonable doubt. • Depends on the importance of the subject. • It can be just putting very small prior probability on the fact to be proven • But you must also try to find the possible flaws in your theory and designing your experiments to counterprove them.

Priors with reasonable doubt • Use priors with reasonable doubt – In an experiment to prove telepathic effects you could e. g. use priors like P(logodds ratio = 0) = 0, 9999. If the logodds ratio is different from 0 it may be modelled as N(0, s 2), where s 2 may be moderate or large. – If the posterior e. g. says that P(p > 0, 5) > 0. 95, and may consider the fact as proved. (Roughly this means that you need about 20 successes in a row, where a random guess has probability ½ to be correct). – Never use the prior P=1, since you must always be open-minded (only fundamentalists do so. They will never change their opinion whatever the evidence). • In more standard situations you will probably not need quite so negative priors

Modelling flaws in theory/study – Example: Several studies needed • An argument often met in medical studies is that no effect is proven until it is corroborated by at least three independent studies. • This means that different conclusions must be drawn with different prior knowledge. (No, one or two previous studies). – People arguing like that violate the Neyman-Pearson theory • How can this be modelled in Bayesian terms?

Some type of multilevel model • Where m is the worldwide mean, ai is the unknown methodology bias of study number i, bi is the site-specific bias and ei is the precision of the experiment (usually with a known (posterior) distribution) • m has an uninformative prior • bi can often be assumed to follow normal distributions with common mean and variance following Normal-c-2 distributions. • ai has probably prior distributions with much longer tails. • The prior distributions for ai and bi might be estimated from other studies.

• If the prior for ai is chosen so that two similar outliers out of two trials is not impossible but three similar outliers is unlikely, we would end up in requiring three independent trials with similar results. • There should probably also be included a selection bias. This can be done but that is too complicated for this short talk. • In the same way the distribution of m may depend on how strict the inclusion criteria are. • One may argue that the k trials are not independent. Studies following the same protocol may get the same bias.

. Some situations which designbased sampling cannot handle • Many people say that one should choose the approach that is best for the problem at hand. Many problems are more difficult than others to handle design-based • For instance – – – – – Missing data Multiple imputation Small area estimation Outlier detection Editing Meta-analysis Synthetic estimation Coding and classification Total survey design … 66

Missing data • A model for the missingness property is needed. The following Bayesian notions are commonly used, but not everyone realises that they are Bayesian • Missing completely at random (MCAR): F(x, y, z, qxyz)= F(x, y, qxy)F(z, qz) • Missing at Random (MAR): ”Given what you know the response mechanism is independent of the other variables” F(x, y, z, qxyz)=F(y, qy|X=x)F(z, qz|X=x)F(X, qx) (where x is known, for unit non-response) (or F(y, z, qyz|y, y€R)=F(y, qy|y€R)F(z, qz|y€R)) (with item non-response) • Not Missing at Random (NMAR) • X auxiliary variables, Y study variables, Z missingness indicator, parameter indexed with the variable for which it contains information. 67

Multiple imputation • Many different situations. We only look at one situation with two y-variables (but use an MCMCtechnique). • For some respondents one of the y-variables is missing, but which one differs between respondents • We assume MAR! • We also assume just now that (y 1 i, y 2 i) comes from a normal super-population with unknown mean q and unknown variance S. 68

Multiple imputation – MCMC-procedure 1. Impute starting values for all missing values 2. Put b=1 3. Find true posterior of q and S (assuming vague normal inverse-gamma prior) 4. Draw possible qb and Sb from this distribution 5. Find the distribution of the missing values assuming these parameter values 6. Draw new random numbers from this distribution and impute them 7. Draw a random value from the conditional distribution of the sum of the non-sampled units (given parameters and imputed values. 8. Add all values to get an estimate of the totals YT 1 b YT 2 b 9. Save them 10. If b < B 0 + B set b = b+1 and go to 3 else stop 69

Multiple imputation cont. • This is an ergodic Markov chain. The distribution converges to the true distribution of T, q and S. • Choose a burn-in period B 0 so large that convergence is reached. • The remaining B observations are thus drawn from the true posterior • This distribution may be plotted 70

Multiple imputation – not fully Bayesian 1 -6 As before to get an imputed data set nr b 7 Estimate the total Tb (or whatever) from the sample, with standard methods and its variance, Sb 8 Save them 9 If b < B 0 + B set b = b+1 and go to 3 else stop • • • Compute the mean of the last B estimates (T). Use this as the estimate of the total Compute the mean of the last B variances (S) Compute the variance of the last B estimates (T) Use the sum of these two values as a variance estimator of the estimate of the total Note that this is a mixture of Bayesian and design-based variances. But it works as a classical estimate.

Multiple imputation cont. • Posterior under normality • But what if the distribution is not normal? 72

Multiple imputation cont. • Posterior under normality • But what if the distribution is not normal? • The means are still a BLUE estimators of the parameter and the total and a consistent estimator of the variance • But if the distribution is skew it will not be particularly good. This is not a defect with multiple imputation, but a problem with skew distributions in general 73

Conclusions • Always use Bayesian methods – – You will get new tools (e. g. full posterior distributions) You will produce something useful You will be logically consistent You will be able to tackle many more problems within theory • You may use design-based methods as ”quick and dirty methods” when you know that the result will be almost equivalent to the Bayesian approach. 74

Thank you for your attention! 75