Sample Size Determination Bandit Thinkhamrop Ph D Statistics

Sample Size Determination Bandit Thinkhamrop, Ph. D (Statistics) Dept. of Biostatistics & Demography Khon Kaen University

Essential of sample size calculation No one accept any “magic number” n Too large vs Too small n To justify with the sponsor and the Ethics Committee n To ensure: n – adequate power to test a hypothesis – desired precision to obtain an estimate

Two main approaches n Hypothesis-based sample size calculation – Involve “power” or beta error – Ensure a significant finding but may not be conclusive clinically – Easy and widely available n Confidence interval methods of sample size calculation – Involve precision of the estimation – Ensure a conclusive finding clinically as this method is directly estimate the magnitude of effect – Difficult and not widely available

Overall steps n n n Identify the primary outcome Identify and review the magnitude of effect and its variability that will be used as the basis of the conclusion of the research. Identify what statistical method that will be used to obtain the main magnitude of effect. Calculate the sample size Describe how the sample size is calculated with sufficient details that allow explicability.

Steps in the calculation n n Base sample size calculation Design effect (for correlated outcome) Contingency (increase to account for nonresponses or dropout) Rounding up to a nearest (and comfortable) number Evaluate if this sample size would provide a precise and conclusive answer to the research question by analyze the data as if it is as expected.

Suggested approaches n n For unknown parameters in the formula, try to find existing evidences or use your best “GUESTIMATE”, a. k. a. educated guest. Do not use only one scenario or based on only one reference for the calculation. It is highly recommended that all key parameters should be varied to see how they effect on the sample size. Always evaluate its sufficiency by estimate the main magnitude of effect and its 95% CI and see if it provide a conclusive finding. Consult with the statistician early

Common pitfalls n n n Unjustified sample size by specifying a “magic” number Based on a simplify formula or a sample size table without understanding its limitations "A previous study in this area recruited 50 subjects and found highly significant results (p=0. 001), and therefore a similar sample size should be sufficient. " – never do it like this Inconsistent with the protocol Too much rely on the previous findings in sample size calculation

Examples of common calculations Mean – one group n Mean – two independent groups n Proportion – one group n Proportion – two independent groups n Get some idea from those n Practice with your own research n

Mean – one group : Formula Where: n = The sample size Z /2 = The standard normal coefficient, typically 1. 96 for 95% CI s =The standard deviation. d = The desired precision level expressed as half of the maximum acceptable confidence interval width.

Mean – one group : Calculations (fix = 0. 05) Expected Standard deviation 25 Precision (half width) n 5 99 30 5 141 25 10 27 30 10 38

Mean – one group : Descriptions n A sample size of 38 would be able to estimate a mean with a precision of 10 assuming a standard deviation of 30 according to a study by <Reference>. That is, based on the expected mean of 55 <Reference>, the 95% confidence interval of the estimated mean would be between 45 and 65.

Mean – two independent group : Formula Sample size in each group (assumes equal sized groups) Represents the desired power (typically. 84 for 80% power). Represents the desired level of statistical significance (typically 1. 96 for = 0. 05). A measure of variability (This is a variance or a square of the standard deviation) Minimum meaningful difference or Effect Size

Mean – two independent groups : Calculations (fix = 0. 05) H 0: M 1 -M 2=0. H 1: M 1 -M 2=D 1<>0. Test Statistic: Z test with pooled variance (SD 1 = 20; SD 2 = 25) Power Mean in Control grp. Minimum and meaningful difference n 1 n 2 90% 30 10 109 80% 30 10 82 82 90% 30 20 28 28 80% 30 20 22 22 90% 35 5 432 80% 35 5 322 90% 35 15 49 49 80% 35 15 37 37

Mean – two independent groups : Descriptions n n A total sample size of 37 in group one and 37 in group two would have a power of 80% to detect a difference between group of 15 assuming a mean of 35 in control group with estimated group standard deviations of 20 and 25, respectively, according to a study by <Reference>. The test statistic used is the two-sided two sample t-test. The significance level of the test was targeted at 0. 05.

Proportion – one group : Formula Where: n = The sample size Z /2 = The standard normal coefficient, , typically 1. 96 for 95% CI p = The value of the proportion as a decimal percent (e. g. , 0. 45). d = The desired precision level expressed as half of the maximum acceptable confidence interval width.

Proportion – one group : Calculations (fix = 0. 05) Expected Prevalence 15% Precision (half width) 2% n 1, 225 20% 2% 1, 537 15% 4% 307 20% 4% 385

Proportion – one group : Descriptions n A sample size of 400 would have a 95% confidence interval of 16% to 24% assuming a prevalence of 20% according to a study by <Reference. <

Proportion – two independent group : Formula Sample size in each group (assumes equal sized groups) Represents the desired power (typically. 84 for 80% power). Represents the desired level of statistical significance (typically 1. 96 for = 0. 05). A measure of variability (similar to standard deviation) Minimum meaningful difference or Effect Size

Proportion – two independent groups : Calculations (fix = 0. 05) H 0: P 1 -P 2=0. H 1: P 1 -P 2=D 1<>0. Test Statistic: Z test with pooled variance Power Proportion in Control grp. Minimum and meaningful difference n 1 n 2 90% 40% 5% 2, 053 80% 40% 5% 1, 534 90% 5% 2, 095 80% 5% 1, 565 90% 40% 10% 519 80% 40% 10% 388 90% 50% 10% 519 80% 50% 10% 388

Proportion – two independent groups : Descriptions n n A total sample size of 388 in group one and 388 in group two would have a power of 80% to detect a difference between group of 10% assuming a prevalence of 50% in control group according to a study by <Reference>. The test statistic used is the two-sided Z test. The significance level of the test was targeted at 0. 0500.

Other considerations n Sampling design affects the calculation of sample size n Complex study designs affects the calculation of sample size n – Simple random sampling / assignment – Stratified random sampling / assignment – Clustered random sampling / assignment – – – Matching Multiple stages of sampling Repeated measures – – – Correlation, Agreement, Diagnostic performance Z-test Regression – multiple linear, logistic Multivariate analyses such as principle component or factor analysis Survival analyses Multilevel models Usually the sample size calculation is based on method of analysis

Other considerations n Demonstrate superiority – Sample size sufficient to detect difference between treatments – Require to specify “minimum meaningful” difference n Demonstrate non-inferiority or equally effective – Sample size required to demonstrate equivalence larger than required to demonstrate superiority – Require to specify “non-inferiority margin or equivalence range”

Precision or Power Estimation n n Equivalence to sample size calculation – do it in the planning phase of the study Do it when the number of available sample is known Wrong: “There around 50 patients per year, of whom 10% may refuse to take part in the study. Therefore over the 2 years of the study, the sample size will be 90 patients. “ Correct: “It is estimated that there will be 90 patients in the clinic. This will give a precision of the prevalence estimation of 20% assuming a prevalence of 65%. ”

Suggested learning resources n WWW: Statistics Guide for Research Grant Applicants at St George’s University of London (maintained by Martin Bland): – http: //www-users. york. ac. uk/~mb 55/guide/size. htm n Software: PASS 2008, n. Query, Epi. Table, Seq. Trial, PS, etc.

Q&A