Volunteers Responses to OpenEnded Survey Questions Evidence of

Volunteers' Responses to Open-Ended Survey Questions: Evidence of Negativity Bias? Kamila A. Gabka, Ashley S. Mc. Carthy, Kimberly T. Schneider, Aaron Whitely, Cassi M. Heider, Jacqueline C. Masso, Erin R. White, and Taylor Weismann Illinois State University Problem Results Table 1 Organizations have long incorporated the use of both close-ended and openended responses to surveys to better assess employees' satisfaction. In samples of paid employees, less satisfied employees are more likely to respond to optional open-ended items and comments tend to be relatively negative in tone (Borg & Zuell, 2012; Poncheri et al. , 2008). Previous research has focused on paid employees' responses; the aim of our study was to focus on samples of volunteers and distinguish between questions aimed at program strengths and weaknesses. It is important to understand volunteers' experiences and feedback given differences in motivations for volunteering versus engaging in paid work. Hypotheses We hypothesized that responders to optional open-ended survey items focused on strengths of the program would report higher levels of job satisfaction. We hypothesized that those commenting on weaknesses would be more likely to report feelings of burnout than those who had no comments to offer. Sample Method The study included two samples of volunteers who completed online surveys. • Sample 1 included 248 volunteers from three animal shelters in the Pacific Northwest • Sample 2 included 139 volunteers at a large hospital in the Midwest. Measures and Procedures Optional open-ended questions allowed volunteers to list three strengths and three weaknesses of the volunteer program. Independent raters coded whether volunteers commented on open-ended questions and assessed the tone of the comments on a 5 -point scale. Inter-rater reliability was. 8 -. 9 for these measures. Closed-ended scales assessed various dimensions of satisfaction (with tasks, with communication, voice), role ambiguity, commitment, and burnout. Example of Strength Comment: “The volunteer coordinators and staff members take the time to ‘catch people doing good work’ and praise volunteers weekly. ” Example of Weakness Comment: “The volunteer office is disorganized and loses records required for volunteering qualification. ” Group Comparisons between Responders and Non-Responders to both ‘Strengths’ and ‘Weaknesses’ of the Program Open-Ended Questions for Sample 1. Mean (SD) for Sample 1 (Animal Responders to Shelter volunteers, ‘Strengths’ of the Program Question N=248) (n=189) Perceptions of 3. 54 (. 88) Voice Satisfaction with 4. 62 (. 55) Work Mean (SD) for Responders to ‘Weaknesses’ of the Program Question (n=157) Perceptions of 3. 32 (. 83) Voice Satisfaction with 3. 97 (. 64) Communication Role Ambiguity 1. 88 (. 82) Burnout 1. 69 (. 67) *p <. 05 **p <. 01 ***p <. 001 Mean (SD) for Non. Responders to ‘Strengths’ of the Program Question (n=59) t Effect size (d) 3. 26 (. 90) -2. 12* 0. 32 4. 39 (. 78) -2. 51* 0. 38 t Effect size (d) Mean (SD) for Non. Responders to ‘Weaknesses’ of the Program Question (n=91) 3. 73 (. 92) 3. 53*** -0. 48 4. 28 (. 75) 3. 54*** -0. 46 1. 62 (. 72) -2. 53* 0. 33 1. 46 (. 62) -2. 53* 0. 35 Group Comparisons between Responders and Non-Responders to both ‘Strengths’ and ‘Weaknesses’ of the Program Open-Ended Questions for Sample 2. Satisfaction with Work Satisfaction with Communication Burnout 4. 46 (. 67) Mean (SD) for Responders to ‘Weaknesses’ of the Program Question (n=66) Mean (SD) for Non. Responders to ‘Strengths’ of the Program Question (n=44) 4. 38 (. 81) Mean (SD) for Non. Responders to ‘Weaknesses’ of the Program Question (n=73) These job attitudes should be of critical importance to volunteer coordinators as we also found that volunteers’ intentions to quit were related to perceptions of having less voice in the organization (r = -. 50**), lower commitment (r = -. 24**), and higher levels of burnout (r =. 43**) and role ambiguity (r =. 47**). Discussion Table 2 Mean (SD) for Sample 2 (Midwest Responders to hospital volunteers, ‘Strengths’ of the N=139) Program Question (n=93) For both samples, we found that those who wrote comments focused on the strengths of the volunteer program (76% in Sample 1 and 63% in Sample 2) were significantly more satisfied with volunteering and felt they had more voice (see Table 1). Those who wrote comments focused on the weaknesses of the program (63% in Sample 1 and 47% in Sample 2) were less committed, felt they had less voice, and reported more burnout and role ambiguity. t Effect size (d) 1. 99* 0. 11 t Effect size (d) 3. 92 (. 68) 4. 21 (. 61) 2. 67** -0. 45 1. 43 (. 60) 1. 26 (. 35) -1. 30** -1. 22 We found significant differences in responders and non-responders for important organizational correlates. Results varied based on comments related to strengths or weaknesses of the program. We will examine whether engagement moderates the relationship between tone of comments and correlates. This study extends previous research to samples of volunteers. It is important to understand volunteers' experiences and motivations to better support those who donate their time to hospitals, animal shelters, and other organizations. *p <. 05 **p <. 01 ***p <. 001 Borg, I. , & Zuell, C. (2012). Write-in comments in employee surveys. International Journal of Manpower, 33, 206 -220. References Poncheri, R. M, Lindberg, J. T, Thompson, L. F. , & Surface, E. A. (2008). A comment on employee negativity bias in open-ended responses. Organizational Research Methods, 11, 614 -630.