Determining the Size of a Sample Sample Accuracy

  • Slides: 56
Download presentation
Determining the Size of a Sample

Determining the Size of a Sample

Sample Accuracy • Sample accuracy: refers to how close a random sample’s statistic is

Sample Accuracy • Sample accuracy: refers to how close a random sample’s statistic is to the true population’s value it represents • Important points: – Sample size is not related to representativeness – Sample size is related to accuracy Ch 13 2

Sample Size and Accuracy • Intuition: Which is more accurate: a large probability sample

Sample Size and Accuracy • Intuition: Which is more accurate: a large probability sample or a small probability sample? • The larger a probability sample is, the more accurate it is (less sample error). Ch 13 3

± A Picture Says 1, 000 Words n 550 - 2000 = 1, 450

± A Picture Says 1, 000 Words n 550 - 2000 = 1, 450 4% - 2% = ± 2% Ch 13 Probability sample accuracy (error) can be calculated with a simple formula, and expressed as a ± % number. 4

How to Interpret Sample Accuracy • From a report… – The sample is accurate

How to Interpret Sample Accuracy • From a report… – The sample is accurate ± 7% at the 95% level of confidence… • From a news article – The accuracy of this survey is ± 7%… Ch 13 5

How to Interpret Sample Accuracy • Interpretation – Finding: 60% are aware of our

How to Interpret Sample Accuracy • Interpretation – Finding: 60% are aware of our brand – So… between 53% (60%-7%) and 67% (60%+7%) of the entire population is aware of our brand Ch 13 6

Sample Size Axioms • To properly understand how to determine sample size, it helps

Sample Size Axioms • To properly understand how to determine sample size, it helps to understand the following axioms… Ch 13 7

Sample Size Axioms • The only perfectly accurate sample is a census. • A

Sample Size Axioms • The only perfectly accurate sample is a census. • A probability sample will always have some inaccuracy (sample error). • The larger a probability sample is, the more accurate it is (less sample error). • Probability sample accuracy (error) can be calculated with a simple formula, and expressed as a +- % number. Ch 13 8

Sample Size Axioms • You can take any finding in the survey, replicate the

Sample Size Axioms • You can take any finding in the survey, replicate the survey with the same probability sample size, and you will be “very likely” to find the same finding within the +- range of the original finding. • In almost all cases, the accuracy (sample error) of a probability sample is independent of the size of the Ch 13 population. 9

Sample Size Axioms • A probability sample can be a very tiny percentage of

Sample Size Axioms • A probability sample can be a very tiny percentage of the population size and still be very accurate (have little sample error). Ch 13 10

Sample Size and Population Size • Where is N (size of the population) in

Sample Size and Population Size • Where is N (size of the population) in the sample size determination formula? Population Size 10, 000 100, 000 1, 000 e=± 3% Sample Size ____ 1, 067 e=± 4% Sample Size ____ 600 100, 00 ____ 1, 067 600 0 In almost all cases, the accuracy (sample error) of a sample is independent of the size of the Ch probability 13 population. 11

Sample Size and Population Size • Does the size of the population, N, affect

Sample Size and Population Size • Does the size of the population, N, affect sample size or sample error? Population Size e=± 3% Sample Size Sample as % of the Population 10, 000 100, 000 1, 000 ____ 1, 067 ___% 1% ___%. 01% 100, 000 ____ 1, 067 ___%. 0001% A probability sample size can be a very tiny percentage of the population size and still be very accurate (have 12 Ch 13 little sample error).

Sample Size Axiom • The size of the probability sample depends on the client’s

Sample Size Axiom • The size of the probability sample depends on the client’s desired accuracy (acceptable sample error) balanced against the cost of data collection for that sample size. Ch 13 13

Putting It All Together • • • Ch 13 MR – What level accuracy

Putting It All Together • • • Ch 13 MR – What level accuracy do you want? MM – I don’t have a clue. MR – National opinion polls use 3. 5%. MM – Sounds me. sample Thegood size of ato probability MR – Okay, depends that means we need on the client’s desired a sample of accuracy (acceptable sample 1, 200. error) balanced against the cost MM – Gee Whiz. That small? of data collection for that MR – Yup, and atsize. a cost of $20 per completion, it sample will be $24, 000. MM – Holy Cow! That much? MR – I could do 500 for $10, 000, and that would be 4. 4% accurate, or 300 for $6, 000 at 5. 7%. MM – 500 sounds good to me. 14

 • There is only one method of determining sample size that allows the

• There is only one method of determining sample size that allows the researcher to PREDETERMINE the accuracy of the sample results… The Confidence Interval Method of Determining Sample Size Ch 13 15

The Confidence Interval Method of Determining Sample Size • This method is based upon

The Confidence Interval Method of Determining Sample Size • This method is based upon the Confidence Interval and the Central Limit Theorem… • Confidence interval: range whose endpoints define a certain percentage of the response to a question Ch 13 16

The Confidence Interval Method of Determining Sample Size • Confidence interval approach: applies the

The Confidence Interval Method of Determining Sample Size • Confidence interval approach: applies the concepts of accuracy, variability, and confidence interval to create a “correct” sample size • Two types of error: – Nonsampling error: pertains to all sources of error other than sample selection method and sample size – Sampling error: involves sample selection and sample size Ch 13 17

The Confidence Interval Method of Determining Sample Size • Sample error formula: Ch 13

The Confidence Interval Method of Determining Sample Size • Sample error formula: Ch 13 18

The Confidence Interval Method of Determining Sample Size • The relationship between sample size

The Confidence Interval Method of Determining Sample Size • The relationship between sample size and sample error: Ch 13 19

Computations Help Page 1. 96 50 times 50 Let’s try 3 n’s 1000 Answers

Computations Help Page 1. 96 50 times 50 Let’s try 3 n’s 1000 Answers this way… 500 Ch 13 100 20

And the answers are… 1. 96 50 times 50 Let’s try 3 n’s Ch

And the answers are… 1. 96 50 times 50 Let’s try 3 n’s Ch 13 1000 ± 3. 1% 500 ± 4. 4% 100 ± 9. 8% 21

Review: What does sample accuracy mean? • 95% Accuracy – Calculate your sample’s finding,

Review: What does sample accuracy mean? • 95% Accuracy – Calculate your sample’s finding, p% – Calculate your sample’s accuracy, ± e% – You will be 95% confident that the population percentage (π) lies between p% ± e% Ch 13 22

Review: What does sample accuracy mean? • Example – Sample size of 1, 000

Review: What does sample accuracy mean? • Example – Sample size of 1, 000 – Finding: 40% of respondents like our brand – Sample accuracy is ± 3% (via our formula) – So 37% - 43% like our brand Ch 13 23

The Confidence Interval Method of Determining Sample Size • Variability: refers to how similar

The Confidence Interval Method of Determining Sample Size • Variability: refers to how similar or dissimilar responses are to a given question • P: percent • Q: 100%-P • Important point: the more variability in the population being studied, the higher the sample size needed to achieve a stated level of accuracy. Ch 13 24

 • With nominal data (i. e. yes, no), we can conceptualize variability with

• With nominal data (i. e. yes, no), we can conceptualize variability with bar charts…the highest variability is 50/50 Ch 13 25

Confidence Interval Approach • The confidence interval approach is based upon the normal curve

Confidence Interval Approach • The confidence interval approach is based upon the normal curve distribution. • We can use the normal distribution because of the CENTRAL LIMITS THEOREM…regardless of the shape of the population’s distribution, the distribution of samples (of n at least =30) drawn from that population will form a normal distribution. Ch 13 26

Central Limits Theorem • The central limits theorem allows us to use the logic

Central Limits Theorem • The central limits theorem allows us to use the logic of the normal curve distribution. • Since 95% of samples drawn from a population will fall + or – 1. 96 x sample error (this logic is based upon our understanding of the normal curve) we can make the following statement… Ch 13 27

 • If we conducted our study over and over, 1, 000 times, we

• If we conducted our study over and over, 1, 000 times, we would expect our result to fall within a known range. Based upon this, we say that we are 95% confident that the true population range value falls within this range. Ch 13 28

The Confidence Interval Method of Determining Sample Size • 1. 96 x s. d.

The Confidence Interval Method of Determining Sample Size • 1. 96 x s. d. defines the endpoints of the distribution. Ch 13 29

 • We also know that, given the amount of variability in the population,

• We also know that, given the amount of variability in the population, the sample size will affect the size of the confidence interval. Ch 13 30

So, what have we learned thus far? • There is a relationship between: Ch

So, what have we learned thus far? • There is a relationship between: Ch 13 – The level of confidence we wish to have that our results would be repeated within some known range if we were to conduct the study again, and… – Variability in the population and… – The amount of acceptable sample error (desired accuracy) we wish to have and… 31 – The size of the sample!

Sample Size Formula • Fortunately, statisticians have given us a formula which is based

Sample Size Formula • Fortunately, statisticians have given us a formula which is based upon these relationships. – The formula requires that we • Specify the amount of confidence we wish • Estimate the variance in the population • Specify the amount of desired accuracy we want. Ch 13 – When we specify the above, the formula tells us what sample we need to use…n 32

Sample Size Formula • Standard sample size formula for estimating a percentage: Ch 13

Sample Size Formula • Standard sample size formula for estimating a percentage: Ch 13 33

Practical Considerations in Sample Size Determination • How to estimate variability (p times q)

Practical Considerations in Sample Size Determination • How to estimate variability (p times q) in the population – Expect the worst cast (p=50; q=50) – Estimate variability: Previous studies? Conduct a pilot study? Ch 13 34

Practical Considerations in Sample Size Determination • How to determine the amount of desired

Practical Considerations in Sample Size Determination • How to determine the amount of desired sample error – Researchers should work with managers to make this decision. How much error is the manager willing to tolerate? – Convention is + or – 5% – The more important the decision, the more (smaller number) the Ch 13 sample error. 35

Practical Considerations in Sample Size Determination • How to decide on the level of

Practical Considerations in Sample Size Determination • How to decide on the level of confidence desired – Researchers should work with managers to make this decision. The more confidence, the larger the sample size. – Convention is 95% (z=1. 96) – The more important the decision, the more likely the manager will want more confidence. 99% Ch 13 confidence, z=2. 58. 36

Example Estimating a Percentage in the Population • What is the required sample size?

Example Estimating a Percentage in the Population • What is the required sample size? – Five years ago a survey showed that 42% of consumers were aware of the company’s brand (Consumers were either “aware” or “not aware”) – After an intense ad campaign, management wants to conduct another survey and they want to be 65% confident that the survey estimate will be within ± 5% of the true percentage of “aware” consumers in the population. – What is n? Ch 13 37

Estimating a Percentage: What is n? • • • Ch 13 Z=1. 96 (95%

Estimating a Percentage: What is n? • • • Ch 13 Z=1. 96 (95% confidence) p=42 q=100 -p=58 e=5 What is n? 38

Estimating a Percentage: What is n? N=374 • What does this mean? Ch 13

Estimating a Percentage: What is n? N=374 • What does this mean? Ch 13 – It means that if we use a sample size of 374, after the survey, we can say the following of the results: (assume results show that 55% are aware) – “Our most likely estimate of the percentage of consumers that are ‘aware’ of our brand name is 55%. In addition, we are 95% confident that the true percentage of ‘aware’ customers in the population falls between 50% and 60%. ” 39

Estimating a Mean • Estimating a mean requires a different formula (See MRI 13.

Estimating a Mean • Estimating a mean requires a different formula (See MRI 13. 2, p. 378) • Z is determined the same way (1. 96 or 2. 58) • E is expressed in terms of the units we are estimating (i. e. , if we are measuring attitudes on a 1 -7 scale, we may want error to be no more than ±. 5 scale units • S is a little more difficult to estimate… Ch 13 40

Estimating s • Since we are estimating a mean, we can assume that our

Estimating s • Since we are estimating a mean, we can assume that our data are either interval or ratio. When we have interval or ratio data, the standard deviation, s, may be used as a measure of variance. Ch 13 41

Estimating s • How to estimate s? – Use standard deviation from a previous

Estimating s • How to estimate s? – Use standard deviation from a previous study on the target population. – Conduct a pilot study of a few members of the target population and calculate s. – Estimate the range the value you are estimating can take on (minimum and maximum value) and divide the range Ch 13 42 by 6.

Estimating s – Why divide the range by 6? • The range covers the

Estimating s – Why divide the range by 6? • The range covers the entire distribution and ± 3 (or 6) standard deviations cover 99. 9% of the area under the normal curve. Since we are estimating one standard deviation, we divide the range by 6. Ch 13 43

Example Estimating the Mean of a Population • What is the required sample size?

Example Estimating the Mean of a Population • What is the required sample size? Ch 13 – Management wants to know customers’ level of satisfaction with their service. They propose conducting a survey and asking for satisfaction on a scale from 1 to 10. (since there are 10 possible answers, the range=10). – Management wants to be 99% confident in the results and they do not wan the allowed error to be more than ±. 5 scale points. 44 – What is n?

Estimating a Mean: What is n? • • Ch 13 S=10/6 or 1. 7

Estimating a Mean: What is n? • • Ch 13 S=10/6 or 1. 7 Z=2. 58 (99% confidence) e=. 5 scale points What is n? 45

Estimating a Percentage: What is n? N=77 • What does this mean? Ch 13

Estimating a Percentage: What is n? N=77 • What does this mean? Ch 13 – After the survey, management may make the following statement: (assume satisfaction mean is 7. 3) – “Our most likely estimate of the level of consumer satisfaction is 7. 3 on a 10 -point scale. In addition, we are 99% confident that the true level of satisfaction in our consumer population falls between 6. 8 46 and 7. 8 on a 10 -point scale”

Other Methods of Sample Size Determination • Arbitrary “percentage of thumb” sample size: –

Other Methods of Sample Size Determination • Arbitrary “percentage of thumb” sample size: – Arbitrary sample size approaches rely on erroneous rules of thumb. – Arbitrary sample sizes are simple and easy to apply, but they are neither efficient nor economical. Ch 13 47

Other Methods of Sample Size Determination • Conventional sample size specification: – Conventional approach

Other Methods of Sample Size Determination • Conventional sample size specification: – Conventional approach follows some convention: or number believed somehow to be the right sample size. – Using conventional sample size can result in a sample that may be too large or too small. – Conventional sample sizes ignore the special circumstances of the survey at Ch 13 48 hand.

Other Methods of Sample Size Determination • Statistical analysis requirements of sample size specification:

Other Methods of Sample Size Determination • Statistical analysis requirements of sample size specification: – Sometimes the researcher’s desire to use particular statistical technique influences sample size Ch 13 49

Other Methods of Sample Size Determination • Cost basis of sample size specification: –

Other Methods of Sample Size Determination • Cost basis of sample size specification: – “All you can afford” method – Instead of the value of the information to be gained from the survey being primary consideration in the sample size, the sample size is determined by budget factors that usually ignore the value of the Ch 13 survey’s results to management. 50

Special Sample Size Determination Situations • Sampling from small populations: – Small population: sample

Special Sample Size Determination Situations • Sampling from small populations: – Small population: sample exceeds 5% of total population size – Finite multiplier: adjustment factor for sample size formula – Appropriate use of the finite multiplier formula will reduce a calculated sample size and save money when performing research on small populations. Ch 13 51

Special Sample Size Determination Situations • Sample size using nonprobability sampling: – When using

Special Sample Size Determination Situations • Sample size using nonprobability sampling: – When using nonprobability sampling, sample size is unrelated to accuracy, so costbenefit considerations must be used. Ch 13 52

Practice Examples • We will do some examples from the questions and exercises at

Practice Examples • We will do some examples from the questions and exercises at the end of the chapter on sample size…question 5 on page 386. Ch 13 53

Practice Examples • 5 a. Using the formula provided in your text, determine the

Practice Examples • 5 a. Using the formula provided in your text, determine the approximate sample sizes for each of the following cases, all with precision (allowable error) of ± 5%: – Variability of 30%, confidence level of 95% Ch 13 54

Practice Examples • 5 b. Using the formula provided in your text, determine the

Practice Examples • 5 b. Using the formula provided in your text, determine the approximate sample sizes for each of the following cases, all with precision (allowable error) of ± 5%: – Variability of 60%, confidence level of 99% Ch 13 55

Practice Examples • 5 c. Using the formula provided in your text, determine the

Practice Examples • 5 c. Using the formula provided in your text, determine the approximate sample sizes for each of the following cases, all with precision (allowable error) of ± 5%: – Unknown variability, confidence level of 95% Ch 13 56