Examining the Sensitivity and Incremental Validity of an

Mixed Responding A type of non-content based responding that includes: • Random Responding •

Combined Response Inconsistency Scale (CRIN) • CRIN was developed on the MMPI-A-RF (Archer, Handel,

CRIN Components: VRIN-r • 53 item pairs • A point is assigned when an

CRIN Components: TRIN-r • 26 pairs • A point is assigned when an examinee

CRIN Calculation on the MMPI-2 -RF VRIN-r True TRIN-r False 53 Pairs 15 Pairs

Previous Research: • Whitney et al. (2018 a): • Calculated raw scores and Linear

Previous Research • Given that CRIN scores uniquely detect cases, Whitney et al. (2018

Current Study • Replicate and extend the work of Whitney et al. (2018 b)

Hypotheses 1. Similar to Whitney et al. (2018 b), CRIN scores will demonstrate elevations

Methods Participants: • Stringent exclusionary criteria were used to exclude all invalid protocols (Burchett

Methods - Procedure • Replicated the methodology of Whitney et al. (2018) • Used

Methods - Procedure • This entire process was complete 6 different times to account

Hypothesis # 1: CRIN scores will demonstrate elevation in the presence of mixed responding.

40% Mixed Responding 60% Mixed Responding 120 110 100 90 90 80 80 70

100% Mixed Responding 80% Mixed Responding 120 110 100 90 90 80 80 70

Hypothesis # 2: Scores on CRIN will incrementally add to scores on VRIN-r and

40% Mixed Responding 100% 90% CASES NOT DETECTED 80% 70% 24% 13% 10% [VALUE]

60% Mixed Responding 100% 4% 3% 90% 0% 80% 14% 4% 0% 31% 10%

80% Mixed Responding 100% 90% CASES NOT DETECTED 80% 12% 5% 0% 6% 2%

100% Mixed Responding 100% CASES NOT DETECTED 1% 90% 19% 80% 3% 1% 3%

Hypothesis # 3: CRIN scores’ incremental utility for detecting mixed responding beyond VRIN-r and

80% Mixed Responding 100% Mixed Responding 120 110 100 90 90 80 80 70

40% Mixed Responding 100% 90% 24% 26% 70% 36% 38% 80% 6% 58% <1%

60% Mixed Responding 100% 90% 7% 0% 29% 13% 80% 70% 60% 50% 40%

80% Mixed Responding 100% 90% 80% 70% 8% 14% 13% 3% 15% 10% 5%

100% Mixed Responding 100% 90% 80% 70% 60% 12% 13% 0% 7% 3% 2%

Hypothesis 2: Scores on CRIN will incrementally add to scores on VRIN-r and TRIN-r

Hypothesis 3: Splitting TRIN-r into TRIN-r – True and TRIN-r – False will negatively

Conclusions • In the presence of mixed responding, scores on CRIN appear to add

Limitation and Future Directions LIMITATIONS • Simulation design • Operationalization of mixed responding FUTURE

Acknowledgments: University of Minnesota Press & MMPI Lab at Kent State University CONTACT INFORMATION:

Slides: 56

Download presentation

Examining the Sensitivity and Incremental Validity of an MMPI-2 -RF Combined Response Inconsistency (CRIN) Scale for Detecting Mixed Responding Nicole M. Lemaster - Ball State University Kendall Whitney & Danielle Burchett - California State University, Monterey Bay Tayla T. C. Lee – Ball State University

Mixed Responding A type of non-content based responding that includes: • Random Responding • Acquiescent Responding • Counter-Acquiescent Responding

Combined Response Inconsistency Scale (CRIN) • CRIN was developed on the MMPI-A-RF (Archer, Handel, Ben. Porath, & Tellegen, 2016) to: • Augment the shortened VRIN-r and TRIN-r validity scales • Serves as a global measure of non-content based responding • Minimal research has been conducted for the use of CRIN on the MMPI-2 -RF

CRIN Components: VRIN-r • 53 item pairs • A point is assigned when an examinee inconsistently answers a pair of items that were written in the same direction • Total score = number of pairs answered inconsistently

CRIN Components: TRIN-r • 26 pairs • A point is assigned when an examinee inconsistently answers a pair of items that were written in the opposite direction • “Tug-o-War” scoring TRIN-r True TRIN-r False 11 TRIN-r

CRIN Calculation on the MMPI-2 -RF VRIN-r True TRIN-r False 53 Pairs 15 Pairs 11 Pairs CRIN

Previous Research: • Whitney et al. (2018 a): • Calculated raw scores and Linear T Scores using the MMPI-2 -RF normative sample • Examined the basic properties of CRIN scale scores on the MMPI-2 -RF in a forensic inpatient sample • Found that scores on CRIN flagged an extra 3% of cases

Previous Research • Given that CRIN scores uniquely detect cases, Whitney et al. (2018 b) investigated CRIN scores’ ability to measure mixed responding • Used computer-generated mixed responding • Investigated 40% mixed responding

Findings for Whitney et al. (2018 b)

Current Study • Replicate and extend the work of Whitney et al. (2018 b) by: • Investigate CRIN scores incremental validity to VRIN-r and TRIN-r scores in detecting mixed responding at a rate of 40 -100% • Investigate how splitting scores on TRIN-r into TRIN-r True and TRIN-r false impacts the incremental validity of CRIN score in detecting mixed responding alongside VRIN-r, TRIN-r True, and TRIN-r False

Hypotheses 1. Similar to Whitney et al. (2018 b), CRIN scores will demonstrate elevations in the presences of mixed responding. 2. Scores on CRIN will incrementally add to scores on VRIN-r and TRIN-r in detecting mixed responding across all of the rates of mixed responding. 3. CRIN scores’ incremental utility for detecting mixed responding beyond VRIN-r and TRIN-r scores will be negatively impacted by splitting TRIN-r into TRIN-r – True and TRIN-r – False

Method

Methods Participants: • Stringent exclusionary criteria were used to exclude all invalid protocols (Burchett et al. , 2016): • CNS ≥ 15; VRIN-r ≥ 70; TRIN-r ≥ 70; F-r ≥ 79; Fp-r ≥ 70; Fs ≥ 80; FBS ≥ 80; RBS ≥ 80; L-r ≥ 65; K ≥ 60 3, 298 College Students Exclusion Criteria 1, 781 College Students • Four samples of college students were combined to form this large sample: • All samples were collected at Midwestern universities between the years 2007 -2018 • Age: M = 19. 30; SD = 2. 44; Gender: 538 men, 1195 women • Ethnicity: 85. 9% White; 8. 3% African American, 2. 6% Hispanic, 1. 6% Asian, 2. 6% another

Methods - Procedure • Replicated the methodology of Whitney et al. (2018) • Used computer-generated responses to simulate mixed responding at various rates • Split MMPI-2 -RF into three sections • Section 1: Items 1 -113; Section 2: 114 -226; Section 3: 227 -338 Items 1 -113 Items 114 -226 Items 227 -338 F T F T F T F T F T F T F

Methods - Procedure • Replicated the methodology of Whitney et al. (2018) • Used computer-generated responses to simulate mixed responding at various rates • Selected designated percentage of items from each section (Ex. 40%) • Generate random, acquiescent, or counter acquiescent responding Items 1 -113 T Items 114 -226 F T F 40% Chosen F F F Items 227 -338 T F T F T F T T F T F

Methods - Procedure • Replicated the methodology of Whitney et al. (2018) • Used computer-generated responses to simulate mixed responding at various rates • Selected designated percentage of items from each section (Ex. 40%) • Generate random, acquiescent, or counter acquiescent responding Items 1 -113 T T F T Items 114 -226 Items 227 -338 F T F T T 40% Random F (True or False) F F F T F T F T F F F

Methods - Procedure • Replicated the methodology of Whitney et al. (2018) • Used computer-generated responses to simulate mixed responding at various rates • Selected designated percentage of items from each section (Ex. 40%) • Generate random, acquiescent, or counter acquiescent responding Items 1 -113 T T F T Items 114 -226 F T T T 40% Random F (True or False) F F F Items 227 -338 T 40%FChosen T F F T F T F

Methods - Procedure • Replicated the methodology of Whitney et al. (2018) • Used computer-generated responses to simulate mixed responding at various rates • Selected designated percentage of items from each section (Ex. 40%) • Generate random, acquiescent, or counter acquiescent responding Items 1 -113 T T F T Items 114 -226 F T T T 40% Random F (True or False) F F F Items 227 -338 T F T T T 40% Acquiescent F T (All True) F T T T F T F T T F F T

Methods - Procedure • Replicated the methodology of Whitney et al. (2018) • Used computer-generated responses to simulate mixed responding at various rates • Selected designated percentage of items from each section (Ex. 40%) • Generate random, acquiescent, or counter acquiescent responding Items 1 -113 T T F T Items 114 -226 F T T T 40% Random F (True or False) F F F Items 227 -338 T T 40% Acquiescent F T (All True) F T T T F T F F 40% TChosen T F

Methods - Procedure • Replicated the methodology of Whitney et al. (2018) • Used computer-generated responses to simulate mixed responding at various rates • Selected designated percentage of items from each section (Ex. 40%) • Generate random, acquiescent, or counter acquiescent responding Items 1 -113 T T F T Items 114 -226 F T T T 40% Random F (True or False) F F F Items 227 -338 T F T T T 40% Acquiescent F T (All True) F T T F F F 40% Counter. T F Acquiescent (All False) F T F F

Methods - Procedure • This entire process was complete 6 different times to account for six different variations of mixed responding • This was repeated across the 7 different variations in mixed responding rate (40 -100%) ACR ARC RCA RAC CAR CRA A = Acquiescent C = Counter-acquiescent R = Random

Results

Hypothesis # 1: CRIN scores will demonstrate elevation in the presence of mixed responding.

40% Mixed Responding 60% Mixed Responding 120 110 100 90 90 80 80 70 70 60 60 50 50 40 40 30 30 20 20 10 10 0 0 RAC RCA VRIN-r CAR CRA TRIN-r ARC CRIN ACR

100% Mixed Responding 80% Mixed Responding 120 110 100 90 90 80 80 70 70 60 60 50 50 40 40 30 30 20 20 10 10 0 0 RAC RCA VRIN-r CAR CRA TRIN-r ARC CRIN ACR

Hypothesis # 2: Scores on CRIN will incrementally add to scores on VRIN-r and TRIN-r in detecting mixed responding across all of the rates of mixed responding

40% Mixed Responding 100% 90% CASES NOT DETECTED 80% 70% 24% 13% 10% [VALUE] 62% 4% 2% 60% 11% 50% 16% 40% 2% 5% 30% 14% 59% 20% 6% 10% 17% RAC 16% 3% 3% 53% 2% 0% 37% 41% 16% 3% 36% 1% 43% 18% RCA CAR CRA ARC ACR 2%

60% Mixed Responding 100% 4% 3% 90% 0% 80% 14% 4% 0% 31% 10% 2% 6% 70% 60% 7% 7% 7% 30% CASES NOT DETECTED 10% 3% 24% 26% 50% 4% 5% 94% 86% 75% 66% 30% 20% 37% 4% 6% 32% 10% 0% RAC RCA CAR CRA 0% 1% ARC ACR

80% Mixed Responding 100% 90% CASES NOT DETECTED 80% 12% 5% 0% 6% 2% 6% 1% 2% 9% 1% 12% 24% 27% 5% 3% 7% 5% 70% 60% 5% 50% 94% 85% 40% 30% 93% 87% 55% 49% 20% 10% 0% RAC RCA CAR CRA ARC ACR 0%

100% Mixed Responding 100% CASES NOT DETECTED 1% 90% 19% 80% 3% 1% 3% 6% 2% 1% 24% 1% 70% 7% 4% 60% 50% 40% 1% 99% 97% 99% 93% 76% 62% 30% 20% 10% 0% RAC RCA CAR CRA ARC ACR

Hypothesis # 3: CRIN scores’ incremental utility for detecting mixed responding beyond VRIN-r and TRIN-r scores will be negatively impacted by splitting TRIN-r into TRIN-r – True and TRIN-r – False

40% Mixed Responding 60% Mixed Responding 120 110 100 90 90 80 80 70 70 60 60 50 50 40 40 30 30 20 20 10 10 0 0 RAC VRIN-r RCA CAR TRIN-r TRUE CRA ARC ACR TRIN-r FALSE CRIN RAC VRIN-r RCA CAR TRIN-r TRUE CRA ACR TRIN-r FALSE ARC CRIN

80% Mixed Responding 100% Mixed Responding 120 110 100 90 90 80 80 70 70 60 60 50 50 40 40 30 30 20 20 10 10 0 0 RAC VRIN-r RCA CAR TRIN-r TRUE CRA ARC TRIN-r FALSE ACR CRIN

40% Mixed Responding 100% 90% 24% 26% 70% 36% 38% 80% 6% 58% <1% 8% 2% 6% 60% 6% 9% 50% 40% 10% 4% 3% 7% <1% 3% 30% 20% 5% 1% 13% 3% 9% 7% 3% 10% 9% 9% <1% 6% 3% 1% 41% 28% 7% 14% 5% 14% RAC 58% 10% 47% 4% <1% 34% 11% RCA CAR CRA ARC ACR <1% 2%

60% Mixed Responding 100% 90% 7% 0% 29% 13% 80% 70% 60% 50% 40% 30% 0% 3% 7% 14% 10% 5% 1% 4% 0% 29% 19% 5% 1% 11% 27% 3% 2% 2% 32% 21% 53% 15% 38% 42% 23% RAC 0% 11% 17% 16% 13% 1% 3% 24% 7% 8% 10% 13% 1% 20% 10% 2% 40% 18% RCA CAR CRA ARC ACR

80% Mixed Responding 100% 90% 80% 70% 8% 14% 13% 3% 15% 10% 5% 0% <1% 1% 22% 8% 11% <1% [VALUE] 12% 2% 15% 0% <1% 0% 7% 19% 1% 9% 60% 50% 40% 35% 3% 13% 35% 31% 43% 25% 22% 30% 41% 20% 10% 0% 33% 31% CAR CRA 25% RAC RCA 41% 42% ARC ACR <1%

100% Mixed Responding 100% 90% 80% 70% 60% 12% 13% 0% 7% 3% 2% 27% 13% 9% 0% 0% 39% 19% 31% 38% 0% 26% 46% 23% 20% 0% 0% 3% 1% 25% 61% 30% 10% 3% 12% 0% 50% 40% 0% 5% 0% 15% 8% 9% 15% 36% 26% RAC 19% RCA CAR 25% 14% CRA ARC ACR

Discussion

Hypothesis 1: CRIN scores will demonstrate elevation in the presence of mixed responding • Successfully replicated and extended the findings observed by Whitney et al. (2018 b) • VRIN-r and CRIN scores were elevated across all rates of mixed responding • However, TRIN-r scores remained between 55 -70 across all the rates of mixed responding

Hypothesis 2: Scores on CRIN will incrementally add to scores on VRIN-r and TRIN-r in detecting mixed responding across all of the rates of mixed responding. • Scores on CRIN add incrementally to VRIN-r and TRIN-r score for detecting mixed responding across all of the different rates of mixed responding. • CRIN scores flagged 0 -27% unique cases across all conditions

Hypothesis 3: Splitting TRIN-r into TRIN-r – True and TRIN-r – False will negatively impact CRIN scores ability to incremental add to scores on VRIN-r, TRIN-r – True, and TRIN-r – False in the detection of mixed responding. • Allowed scores on TRIN-r – True and TRIN-r - False to elevate beyond the 60 -70 t-score range • Negatively impacted CRIN scores ability to incremental add to scores on VRIN-r, TRIN-r – True, and TRIN-r – False in the detection of mixed responding VRIN-r, TRIN-r, & CRIN VRIN-r, TRIN-r – True, TRIN-r False, & CRIN % Detected by CRIN 40% 10 – 16% 6 – 10% 50% 6 – 21% 3 – 10% 60% 3 – 27% 2 – 11% 70% 1 – 27% 0 – 11% 80% 0 – 27% 1 – 10% 90% 1 – 26% 0 – 9% 100% 0 – 24% 0 – 7% Mixed Responding Rate

Hypothesis 3: Splitting TRIN-r into TRIN-r – True and TRIN-r – False will negatively impact CRIN scores ability to incremental add to scores on VRIN-r, TRIN-r – True, and TRIN-r – False in the detection of mixed responding. • Allowed scores on TRIN-r – True and TRIN-r - False to elevate beyond the 60 -70 t-score range • Negatively impacted CRIN scores ability to incremental add to scores on VRIN-r, TRIN-r – True, and TRIN-r – False in the detection of mixed responding VRIN-r, TRIN-r, & CRIN VRIN-r, TRIN-r – True, TRIN-r - False, & CRIN % Detected by CRIN 40% 10 – 16% 6 – 10% 50% 6 – 21% 3 – 10% 60% 3 – 27% 2 – 11% 70% 1 – 27% 0 – 11% 80% 0 – 27% 1 – 10% 90% 1 – 26% 0 – 9% 100% 0 – 24% 0 – 7% Mixed Responding Rate

Hypothesis 3: Splitting TRIN-r into TRIN-r – True and TRIN-r – False will negatively impact CRIN scores ability to incremental add to scores on VRIN-r, TRIN-r – True, and TRIN-r – False in the detection of mixed responding. • However, by splitting TRIN-r into TRIN-r – True and TRIN-r – False, fewer invalid cases were detected than when simply using TRIN-r scores VRIN-r, TRIN-r, & CRIN VRIN-r, TRIN-r – True, TRIN-r - False, & CRIN % Not Detected 40% 39 – 80% 24 – 58% 50% 4 – 30% 10 – 42% 60% 4 – 31% 7 – 29% 70% 2 – 20% 6 – 21% 80% 0 – 12% 7 – 14% 90% 0 – 7% 9 – 28% 100% 0 – 2% 9 – 27% Mixed Responding Rate

Conclusions • In the presence of mixed responding, scores on CRIN appear to add incrementally to scores on VRIN-r and TRIN-r. • Though CRIN’s utility is limited when TRIN-r is split into TRIN-r – True and TRINr – False, splitting TRIN-r appears to limit the number of invalid cases detected. • Overall, adding a CRIN scale on the MMPI-2 -RF and future instruments may be more useful than using TRIN-r - True and TRIN-r - False in the detection of mixed responding.

Limitation and Future Directions LIMITATIONS • Simulation design • Operationalization of mixed responding FUTURE DIRECTIONS • Explore different method of operationalizing mixed responding • Investigate if/how mixed responding occurs naturally

Acknowledgments: University of Minnesota Press & MMPI Lab at Kent State University CONTACT INFORMATION: Nicole M. Lemaster Ball State University Department of Psychological Science North Quadrangle Building Muncie, Indiana 47306 Email: nmlemaster@bsu. edu