Not Helpful Somewhat Helpful Very Helpful but not
Not Helpful Somewhat Helpful Very Helpful but not Essential Somewhat Essential Very Essential # of Experts Very Essential Somewhat Essential # of Experts Somewhat Essential Very Helpful but not Essential # of Experts Very Essential Very Helpful but not Essential Somewhat Helpful Very Helpful but not Essential Not Helpful Somewhat Helpful 2 3 4 Not Helpful Somewhat Helpful Very Helpful but not Essential Somewhat Essential Very Essential 4 Not Helpful 1 3 3 3 40 0 2 2 2 4 Mean: 3. 2 Median: 3. 0 60 Mean: 2. 4 Median: 2. 0 40 1 1 1 3 Lack of Ceiling/Floor Effects 60 0 2 0 0 Not Helpful Somewhat Helpful Very Helpful but not Essential Somewhat Essential Very Essential 0 0 1 20 20 0 4 40 4 3 Mean: 3. 3 Median: 3. 5 40 3 2 Lack of Practice Effects 2 2 1 Construct Validity Adequate Number of Trials for Robust Estimate 60 60 4 3 1 0 0 0 # of Experts 20 20 0 40 40 40 0 0 Median: 2. 0 40 # of Experts Internal Consistency 60 Mean: 2. 2 Mean: 2. 4 Median: 2. 0 60 Mean: 3. 5 Median: 4. 0 60 Mean: 2. 9 Median: 3. 0 Short Length Alternate Form Reliability Test-Retest Reliability 60 Mean: 2. 9 Median: 3. 0 # of Experts Small Scale Clinical Trials
Very Essential 4 # of Experts Very Essential Somewhat Essential Very Essential 3 Somewhat Essential Very Helpful but not Essential Somewhat Essential 2 Very Helpful but not Essential Somewhat Helpful Very Helpful but not Essential 1 Somewhat Helpful Not Helpful Somewhat Helpful 0 Not Helpful 4 Very Essential 4 3 Somewhat Essential 3 4 2 Very Helpful but not Essential Very Essential 2 3 1 Somewhat Helpful Somewhat Essential 1 2 0 Not Helpful Very Helpful but not Essential 0 1 0 0 Not Helpful Somewhat Helpful Very Helpful but not Essential Somewhat Essential Very Essential 0 20 20 Somewhat Helpful 40 Not Helpful 40 Mean: 2. 5 Median: 2. 0 40 0 0 40 Mean: 3. 2 Median: 3. 0 60 Lack of Ceiling/Floor Effects 60 Mean: 3. 2 Median: 3. 0 4 4 Lack of Practice Effects 2 3 3 4 Construct Validity Adequate Number of Trials for Robust Estimate 60 60 4 3 1 2 2 3 # of Experts 60 0 1 0 0 2 20 # of Experts 20 20 0 1 40 40 0 0 Mean: 2. 8 Median: 3. 0 # of Experts Internal Consistency Mean: 2. 6 Median: 3. 0 60 Mean: 3. 3 Median: 4. 0 60 Mean: 3. 0 Median: 3. 0 Short Length Alternate Form Reliability Test-Retest Reliability 60 Mean: 2. 9 Median: 3. 0 # of Experts Large Scale Clinical Trials
Small Scale - Top Two By Area • Clinical Trials in Schizophrenia – Test-Retest Reliability – Lack of Ceiling/Floor Effects • Cognitive Neuroscience of Schizophrenia – Construct Validity – Test-Retest Reliability • Cognitive Rehab of Schizophrenia – Construct Validity – Test-Retest Reliability • Human Cognitive Neuroscience – Construct Validity – Lack of Ceiling/Floor Effects • Animal Cognitive Neuroscience – Test-Retest Reliability – Construct Validity
Large Scale - Top Two By Area • Clinical Trials in Schizophrenia – Test-Retest Reliability – Lack of Ceiling/Floor Effects • Cognitive Neuroscience of Schizophrenia – Test-Retest Reliability – Lack of Ceiling/Floor Effects • Cognitive Rehab of Schizophrenia – Construct Validity – Test-Retest Reliability • Human Cognitive Neuroscience – Construct Validity – Lack of Ceiling/Floor Effects • Animal Cognitive Neuroscience – Internal Consistency – Lack of Ceiling/Floor Effects
Benchmark Values - Entire Sample Characteristic Internal Consistency Test-Retest Alternate Form Length (in minutes) Minimum # of Trials Practice Effect (in SD units) Floor Effect (% diff from 0) Ceiling Effect (% diff from 100) Optimal. 85. 90 15 min 30 trials. 20 SD 20% 25% “Worst Acceptable”. 60. 70. 65 30 min 15 trials. 50 SD 10% 15% Benchmark Values - Higher Confidence Characteristic Internal Consistency Test-Retest Alternate Form Length (in minutes) Minimum # of Trials Practice Effect (in SD units) Floor Effect (% diff from 0) Ceiling Effect (% diff from 100) Optimal. 875. 90 10 min 25 trials. 20 SD 25% 37. 5% “Worst Acceptable”. 60. 70 30 min 10 trials. 50 SD 10% 12. 5%
- Slides: 5