Creating Replicable Scales Generating Short Forms using Multiple

Creating Replicable Scales - Generating Short Forms using Multiple Samples Aurelio José Figueredo Evaluation Group for Analysis of Data Department of Psychology, School of Mind, Brain, and Behavior, College of Science, University of Arizona Tucson, Arizona, United States of America

Acknowledgements Rafael Antonio Garcia J. Michael Menke W. Jake Jacobs Paul Robert Gladden Jean. Marie Bianchi Emily Anne Patch Connie J. A. Beck Phillip S. Kavanagh Marcela Sotomayor-Peterson Norman P. Li Fann (Yunfan) Jiang Marco Del Giudice Romina Angeleri

The Construction of Short Formsempirical selection of variables � Often experience problems of replication when applied to independent samples, especially cross-cultural ones: ◦ model fitting to sample-specific characteristics ◦ capitalize on chance differences (stochastic fluctuations) among item characteristics that are purely due to sampling error Between-sample differences in cross-cultural research can be quite large!

The Construction of the K-SF-42 An alternative Short form for the 199 -item Arizona Life History Battery (ALHB) Secondary data analyses on a recently completed five-nation cross-cultural survey Objective: optimize K-SF-42 across all five cultures sampled Cross-sample Geometric Mean (CSGM) method was used for this cross-culturally valid short form

Outline: Psychometric Assessment/Evaluation � Internal consistency: short vs. long form � Part-whole correlations (unit-weighted factor structures) for each of the 7 subscales in ALHB and the short form � Simple bivariate correlations between short and long forms for each subscale of ALHB � Relative consistency reliabilities of the two alternative short forms (Mini-K and K-SF-42) � Relative incremental validities of the two short forms in multiple regression models � IRT Rasch modeling & generalizability theory

The ALHB and the K-SF-42: slower LH strategy vs. faster LH (lower) The Arizona Life History Battery (ALHB) contains seven subscales (not including the Mini-K, which was designed as domaingeneral) that sample the following domainspecific psychosocial resource allocations: 1. Insight, Planning, and Control 2. Mother/Father Relationship Quality 3. Family Social Contact and Support 4. Friends Social Contact and Support 5. Romantic Partner Attachment 6. General Altruism 7. Religiosity

The Mini-K and the K-SF-42 Mini-K, a highly successful 20 -item short form for a decade, but with a limited reliability (average alpha =. 75) K-SF-42: 42 items directly selected from ALHB

The K-SF-42 � Six items were selected from each of the 7 subscales of the ALHB � Novel item selection procedures were applied to obtain items and subscales that functioned at the highest levels of cross-cultural generality � Sample: 738 – Australia (N=131), Italy (N=172), Mexico (N=160), Singapore (N=115), US (N=160)

The K-SF-42 Mother/Father relationship quality subscale General Altruism scale- towards kin, friends, community First: calculated the part-whole correlation of each item score with the unit-weighted factor score Then computed the Geometric mean of these part-whole correlations across all five crosscultural replications rather than arithmetic mean: Cross-Sample Geometric Mean (CSGM)

Table 1. Internal Consistency Reliabilities for Convergent Indicators of Life History Strategy in Australia (AU), Italy (IT), Mexico (MX), Singapore (SG), and the United States of America (US).

Table 2. Unit-Weighted Factor Structures for Convergent Indicators of Life History Strategy in Australia (AU), Italy (IT), Mexico (MX), Singapore (SG), and the United States (US) of America.

Table 3. Short-Long Form Correlations for Convergent Indicators of Life History Strategy in Australia (AU), Italy (IT), Mexico (MX), Singapore (SG), and the United States (US) of America.

Table 4. Internal Consistency Reliabilities of Alternative Short Forms (Mini-K and K-SF-42) for Latent Multivariate Construct of Life History Strategy in Australia (AU), Italy (IT), Mexico (MX), Singapore (SG), and the United States of America (US).

Table 5. Incremental Validities of Alternative Short Forms (Mini-K and K-SF-42) for Latent Multivariate Construct of Life History Strategy in Australia (AU), Italy (IT), Mexico (MX), Singapore (SG), and the United States of America (US).

Follow-up analyses � IRT Rasch modeling of both individual items and aggregated subscales to ascertain whether our procedures performed as intended � We also performed a generalizability theory (GT) analysis (Figueredo & Olderbak, 2008) on the IRT-generated subscale difficulty estimates to test for the degree of crosscultural invariance that our procedures managed to achieve

IRT Rasch Modeling of Item and Subscale Difficulties One-parameter Rasch model Latent variable of interest is LH speed (K) The hierarchical item selection procedure that we used should therefore enable the K-SF-42 to measure the full range of LH speed (K) along the broader difficulty distribution of subscales

Table 6. Standard Deviations (SD) of Raw Item Difficulties within Standardized Subscale Infit/Outfit Mean Squares (MNSQ) and Standardized Subscale Infit/Outfit Standard Deviations (SD) for Convergent Indicators of Life History Strategy.

Generalizability Analyses of Item and Subscale Difficulties Subjected the individual-level subscale difficulty estimates to general linear modeling (GLM) and Generalizability Theory (GT) analyses (Figueredo & Olderbak, 2008) Restricted these analyses to aggregate subscale scores, as the cross-cultural function of the individual items within subscale was of lesser interest

Table 7 a. Generalizability Analysis of IRT-based Subscale Difficulties for the K-SF-42

Table 7 b. Generalizability Analysis of IRT-based Subscale Difficulties for the K-SF-42 Relative difficulties of the subscales are quite consistent across cultures, Indicating that the subscales function in similar ways in measuring the construct of interest

Figure 1. IRT Subscale Difficulties for K-SF-42

Conclusion and limitation K-SF-42 performed very well: adequate in terms of psychometric indices No data yet