Schmidt Hunter Approach to r Bare Bones Statistical

Schmidt & Hunter Approach to r Bare Bones

Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction Computational error Dichotomization of variables *addressed in the (bare-bones) analysis

Bare Bones r Find weighted mean and variance: Note sample size weight. Note that for unit weights, the weighted variance estimator is the sample, not population, estimate.

Confidence Interval for Mean There are k studies, with Ni observations. This is not the only formula they use, but it’s the best one IMHO.

Estimated Sampling Error Variance The variance of r Estimated variance for a study. Estimated sampling variance for a meta-analysis. Note mean r is constant. This is the variance of sampling error we expect if all the studies have a common effect estimated by r-bar.

Variance of Rho Classical Test Theory Sampling Error A definition

Estimated Variance of rho Note that the variance of rho will be called tausquared by Hedges To find the variance of infinite-sample correlations, find the variance of r in the meta-analysis and subtract expected sampling error variance. Schmidt would be quick to add that part of the estimated variance of infinite-sample correlations is artifactual.

Credibility Interval The credibility interval and the confidence interval are quite different things. The CI is a standard statistical estimate (intended to contain rho, or average of rho). The CR is intended to contain a percentage of the values of a random variable – infinite-sample effect sizes. The S&H value forgets that there is also uncertainty in the mean value; the two should be added. There are Bayesian programs that will do this; there is also an approximation called the prediction interval described in Borenstein et al.

Bare-Bones Example (1) Study Ni r 1 200 . 20 2 100 . 20 3 150 . 40 4 80 . 40 Mean 132. 5. 30 <- Unit weighted mean

Bare-Bones Example (2) r Ni r. Ni . 20 200 40 . 20 100 20 . 40 150 60 . 40 80 32 sum 530 152

BB Example (3) Recall unwighted or unit weighted mean =. 30. Why are they different?

BB Example (4) r Ni . 20 200 1. 507 . 20 100 . 753 . 40 150 1. 922 . 40 80 1. 025 Sum 530 5. 208

BB Example (5)

Interpretation Schmidt says this is a random-effects metaanalysis. It uses a sample of studies to represent a larger population of studies. People interpret the Credibility Interval, but typically do not recognize that it is poorly estimated.