The Effect of the Conscientious Responders Scale on

  • Slides: 1
Download presentation
The Effect of the Conscientious Responders Scale on Random Responding Rates in Psychological Questionnaires

The Effect of the Conscientious Responders Scale on Random Responding Rates in Psychological Questionnaires Noor Shubear, Zdravko Marjanovic, Lisa Bajkov, & Tsz Yin Fung Thompson Rivers University Introduction Design and Results The validity of questionnaire data is based on the expectation that responders are Results and Discussion Hypotheses conscientious responders (CR)—they answer items as honestly and accurately as 1. The CRS-embedded questionnaire will produce smaller ISD scores compared to the Table 1 they can. non-embedded questionnaire. Random Responding Differences between CRS-Embedded and Non-Embedded Questionnaires The problem is, for a variety of reasons such as disinterest and fatigue, some Random NEQ CEQ Responding Mean t-test p-value NEU_ISD 0. 93 -0. 02 . 985 Participants and Procedure EXT_ISD 0. 92 0. 89 0. 54 . 591 The CRS is a 5 -item validity scale (see below) that differentiates between CR and RR, A 109 undergraduate students were randomly divided into two groups. The first group- OPE_ISD 0. 98 0. 95 0. 56 . 579 which has been shown to be an effective tool in detecting RR with 93% accuracy rate. composed of 55 students- completed a 208 -item paper-and-pencil questionnaire that did AGR_ISD 0. 92 0. 94 -0. 64 . 526 (CRS; Marjanovic, Holden, et al. , 2015. Marjanovic, Struthers, et al. , 2014). not contain the CRS. The other group- composed of 54 student- completed a 218 -item CON_ISD 0. 92 0. 82 2. 14 . 035 However, a question remains to be answered is whether the presence of the CRS scale paper-and-pencil questionnaire which contained a double size CRS. For the purpose of _________________________________ in psychological questionnaires affects the rates of random responding. Does the this study, we only analyzed data from the 60 -item five-factor inventory (NEO-FFI) to Note. NEU=neuroticism, EXT=extraversion, OPE=openness to experience, AGR= presence of the CRS items in a questionnaire deter responders from answering determine the effect of CRS in psychological questionnaires. To test the effect of the agreeableness, CON=conscientiousness. randomly by alerting them that the validity of their data can be assessed? CRS on random responding rates, we used a statistical approach that does not alter the 2. Means and standard deviation will be about equal across both questionnaires. responders are random responders (RR)—they answer items carelessly or indiscriminately, without regard for what items are asking them to consider. Method look and/or the content of the questionnaire as CRS does. The inter-item standard The purpose of this study was to test whether the presence of CRS scale in a deviation (ISD; Marjanovic, Holden, et al. , 2015) is statistical technique that reflects how psychological questionnaire would (1) affect the rates of random responding as the responses are closely clustered around the mean score. CR produces a score that is compared to a questionnaire without the CRS scale, and (2) result in any significant closely clustered around the mean ( i. e. , small ISD score), while RR produces the Table 2 differences in the means and standard deviations. opposite effect. Mean-Scores Differences between CRS-Embedded and Non-Embedded Questionnaires NEQ CEQ Measures Mean t-test Measures NEU 3. 32 3. 12 1. 69 . 095 All items were answered on a 5 -point Likert scale EXT 3. 38 3. 42 -0. 42 . 676 Conscientious Responders Scale (CRS; Marjanovic, Struthers, et al. , 2014) OPE 3. 43 3. 50 -0. 59 . 559 Inter-Item Standard Deviation (ISD; Marjanovic, Holden, et al. , 2015) AGR 3. 57 3. 55 0. 251. 802 CON 3. 44 3. 63 -1. 84. 069 _________________________________ Note. NEU=neuroticism, EXT=extraversion, OPE=openness to experience, AGR= . The CRS utilizes instructional item content. Each CRS item, randomly embedded in a questionnaire, instructs responders exactly how to respond to that particular item. CRS 1. To answer this question, please choose number three, neither agree nor disagree. NEO Five Factor Inventory (NEO-FFI; Mc. Crae, Robert R. ; Costa, Paul T. , Jr. , 1978; 2010), Its 60 items measure five factors of personality; neuroticism, extraversion, openness to experience, agreeableness, and conscientiousness. 2. Choose the first option—strongly disagree—in answering this question. 3. To respond to this question, please choose number five, strongly agree. 4. Please answer this question by choosing number two, disagree. 5. In response to this question, please choose number four, agree. Compliant responses are scored as 1 s and incompliant responses are scored as 0 s. Items are summed to make a score that ranges from 0 (all incompliant responses) – 5 (all compliant responses). Because, on a 5 -point response scale, it is so statistically unlikely that RR can answer several CRS items compliantly by chance alone, high CRS sum scores (3 – 5) must reflect conscientious responding whereas low CRS sum scores (0– 2) probably reflect random responding. p-value agreeableness, CON=conscientiousness. Results All studied measures produced high estimates of reliability (Cronbach alpha >. 70). Hypothesis 1 statistics are presented in table 1. The results do not support the hypothesis. The presence of the CRS in a psychological questionnaires seems to have no influence on reducing or deterring rates of random responding as compared to questionnaires that do not contain CRS. Hypothesis 2 statistics are presented in table 2. The results do support the hypothesis. The presence of the CRS in psychological questionnaires seems to have no effect on means and standard deviations of psychological measures. Questionnaires with and without the CRS in it produced similar descriptive statistics. Conclusion 1. The presence of CRS items in a questionnaire had no effect of reducing or deterring rates of participant random responding. 2. A positive finding: the CRS had no effect on means and standard deviations across the CRS-embedded and non-embedded questionnaires.