Significance in Strategic Management Research Steve Werner Department

Overview n The problems of statistical significance. n The solutions. n Conclusions.

The Problems of Statistical Significance 1. 2. 3. Statistical significance is not practical significance.

Statistical Significance is not Practical Significance a. b. c. Practical significance is effect size

Conventional Cut-offs are Arbitrary and Misleading Convention is: a. i. iii. iv. b. c.

Frequently Occurring Type II Errors a. b. c. Studies conclude no relationship because findings

The Solutions 1. 2. 3. Solutions to the practical significance problem. Solutions to the

Solutions to the Practical Significance Problem a. b. c. d. e. Report confidence intervals.

Solutions to the Arbitrary Cut-off Problem a. b. c. d. Go back to classical

Solutions to the Type II Error Problem Increase Power: a. i. iii. b. c.

Conclusion n n The problems of statistical significance. The solutions. Report actual p values.

Slides: 12

Download presentation

“Significance” in Strategic Management Research Steve Werner Department of Management Bauer College of Business University of Houston

Overview n The problems of statistical significance. n The solutions. n Conclusions.

The Problems of Statistical Significance 1. 2. 3. Statistical significance is not practical significance. The conventional cut-offs are arbitrary and misleading. Frequently occurring Type II errors.

Statistical Significance is not Practical Significance a. b. c. Practical significance is effect size and importance. Related but not the same. Statistical significance largely affected by n.

Conventional Cut-offs are Arbitrary and Misleading Convention is: a. i. iii. iv. b. c. d. p<. 10 = marginally significant; p<. 05 = significant; p<. 01 = highly significant; p<. 001=very highly significant. p<. 05 from Fisher (1925). Counters classical view of predetermining alpha. Simplifies hypothesis testing.

Frequently Occurring Type II Errors a. b. c. Studies conclude no relationship because findings not significant. Power issues. This occurs in 60% of all studies (Hunter, 1997).

The Solutions 1. 2. 3. Solutions to the practical significance problem. Solutions to the conventional cut-offs problem. Solutions to the frequently occurring Type II errors problem.

Solutions to the Practical Significance Problem a. b. c. d. e. Report confidence intervals. Report effect sizes. Avoiding using “significance”. Many journals still do not report effect sizes. SMJ: 98. 5% of studies report effect sizes.

Solutions to the Arbitrary Cut-off Problem a. b. c. d. Go back to classical Neyman. Pearson hypothesis testing with researcher pre-determined alphas. Report actual p values (Fisher, 1956). Easier to accomplish now. SMJ: 13. 4% (35/262) of studies report some actual p values.

Solutions to the Type II Error Problem Increase Power: a. i. iii. b. c. d. Conduct power analyses; Increase sample size; Increase effect size. Relax alpha. SMJ: 65. 6% (172/262) of studies use p<. 10 as cut-off. Report actual p values.

Conclusion n n The problems of statistical significance. The solutions. Report actual p values. Consider p, power, effect size, sample size, and importance when evaluating research findings.