March 11 2021 XSEDE Staff Climate Study Lizanne

  • Slides: 24
Download presentation
March 11, 2021 XSEDE Staff Climate Study Lizanne De. Stefano and Lorna Rivera |

March 11, 2021 XSEDE Staff Climate Study Lizanne De. Stefano and Lorna Rivera | Georgia Institute of Technology Julie Wernert | Indiana University Bloomington

Purpose Support XSEDE’s organizational health by providing data over time to: • Better understand

Purpose Support XSEDE’s organizational health by providing data over time to: • Better understand current working conditions • Recognize successes and areas of concern • Develop responses to improve working conditions • Improve workplace efficiency and satisfaction 2

Method • Annual on-line Survey to all XSEDE staff and leadership – Core items

Method • Annual on-line Survey to all XSEDE staff and leadership – Core items – XSEDE specific items – June administration • Disaggregation by Level 2, 3, site, FTE, length of employment – Special requests by L 2 and L 3 managers • EXTENSIVE dissemination and interaction around results • Documentation of XSEDE response to results – Quarterly Meetings 3

Study Response Rates 100% 62% 50% 49% 56% 52% 56% 57% 2017 (N=181) 2018

Study Response Rates 100% 62% 50% 49% 56% 52% 56% 57% 2017 (N=181) 2018 (N=206) 0% 2013 (N=230) 4 2014 (N=264) 2015 (N=298) 2016 (N=249)

Study Response Rates Explained • Total response to 2018 survey is 117/206 (57%) •

Study Response Rates Explained • Total response to 2018 survey is 117/206 (57%) • In 2018, staff totals were generated by staff in the XSEDE budget office based on PY 7 billing history. 5

2018 Respondent Demographics (N=117) Staff Level (Self Report) Length of Employment (Self Report) 100%

2018 Respondent Demographics (N=117) Staff Level (Self Report) Length of Employment (Self Report) 100% Gender (Self Report) 71% 70% 56% 8% 21% 6% 15% 0% 0% Level 1 or Level 3 2 manager 6 29% 24% Staff 0% <1 year 1 - 3 years 3 - 5 5+ years Female Male

2018 General Climate Study Findings • Most dimensions continue to increase from 2013 baseline

2018 General Climate Study Findings • Most dimensions continue to increase from 2013 baseline scores and meet or exceed comparable project ratings. The highest gains were made in the Wiki & Website, Communication Tools, and Decision Making dimensions. • While Equity items increased, differences between 2018 and 2017 were not significant. All groups rated the index above a 3. 6, though some L 2, Gender, and Racial/Ethnic differences exist. • Leadership/management training was a common request among training topics. • Inclusion ratings declined slightly in 2018 though not significant. Some L 2 differences exist. 7

[VALUE]*** [VALUE]*** 3. 50 [VALUE]*** 2018 (N=117) 3. 79 3. 95 4. 12 [VALUE]**

[VALUE]*** [VALUE]*** 3. 50 [VALUE]*** 2018 (N=117) 3. 79 3. 95 4. 12 [VALUE]** 4. 00 [VALUE] 4. 26 4. 14 5. 00 [VALUE]* 2013 (N=127) [VALUE]*** 2013 (baseline) vs 2018 Index Scores (Scale 1 – 5) 3. 00 2. 00 1. 00 Communication Decision Making Tools * p <. 05, ** p <. 01, *** p <. 001 8 Equity Inclusion Leadership & Management Support & Belonging Value & Satisfaction Wiki & Website

[VALUE]** [VALUE] 3. 94 [VALUE] 3. 70 [VALUE] 4. 09 2018 (N=117) [VALUE] 4.

[VALUE]** [VALUE] 3. 94 [VALUE] 3. 70 [VALUE] 4. 09 2018 (N=117) [VALUE] 4. 17 [VALUE] 3. 94 [VALUE] 4. 00 [VALUE]** 2017 (N=98) [VALUE]* 5. 00 [VALUE] 2017 vs 2018 Index Scores (Scale 1 – 5) 3. 00 2. 00 1. 00 Communication Decision Making Tools * p <. 05, ** p <. 01, *** p <. 001 9 Equity Inclusion Leadership & Management Support & Belonging Value & Satisfaction Wiki & Website

Wiki & Website: Ratings of Wiki and Website items increased significantly in 2018. While

Wiki & Website: Ratings of Wiki and Website items increased significantly in 2018. While all items were rated higher than previous years, staff found the wiki significantly more helpful for finding information across the project this year. Wiki & Website Index Response Over Time, (Scale 1, Strongly Disagree – 5, Strongly Agree) 2018 (N=117) 3. 83 2017 (N=98) 3. 91 3. 96 3. 74 3. 88 2016 (N=125) 3. 82 3. 73 3. 95 2015 (N=142) 3. 04 2. 99 3. 18 3. 25 3. 12 3 2014 (N=159) 3. 76 [VALUE] 3. 81 3. 26 3. 58 3. 55 3. 51 [VALUE]*** 4 3. 04 3. 14 3. 68 3. 71 3. 56 [VALUE]** 5 2013 (N=127) 2 1 Q 1 a. The XSEDE staff Q 1 b. The XSEDE Q 1 c. The XSEDE staff Q 1 d. The XSEDE Q 1 e. The XSEDE staff Q 1 q. The XSEDE staff wiki helps me find website (xsede. org) wiki contains website (xsede. org) wiki helps me to wiki is useful for information across helps me find information that is contains information communicate disseminating and the project. information across useful to me and my that is useful to me effectively with other sharing information the project. work. and my work. XSEDE staff. [RETIRED with other staff ITEM] 0 Wiki & Website * p <. 05, ** p <. 01, *** p <. 001 10

Wiki & Website Quotes • “I don't look at the website as an XSEDE-staff

Wiki & Website Quotes • “I don't look at the website as an XSEDE-staff resources--I rely on the wiki for that. ” - 2018 Respondent • “Website is great, but sometimes when searching for information, old or deprecate documents turn up in the search. ” – 2018 Respondent • “It is really difficult to find anything in the staff wiki, best is to ask a manager where to find information and get a direct link or a person to contact. It would be useful to have a constantly updated page of contact points for each XSEDE area and each provider. ” – 2018 Respondent 11

Communication Tools: In 2018, ratings of the Communication Tools Index increased significantly compared to

Communication Tools: In 2018, ratings of the Communication Tools Index increased significantly compared to last year. Significantly higher ratings for useful collaboration resources were reported along with increased positive responses regarding conference calls and the adequacy of available collaboration resources. Communication Tools Index Response Over Time, (Scale 1, Strongly Disagree – 5, Strongly Agree) [VALUE]** 3. 79 3. 71 3. 87 4. 13 2018 (N=117) 3. 96 2017 (N=98) 3. 90 4. 46 4. 44 2016 (N=125) 4. 50 4. 53 4. 45 2015 (N=142) 4. 39 [VALUE] 4. 16 4. 11 2014 (N=159) 3. 96 4. 00 3. 91 [VALUE]* [VALUE] 4. 13 4. 27 5. 00 4. 14 2013 (N=127) 3. 00 2. 00 1. 00 Communication Tools * p <. 05, ** p <. 01, *** p <. 001 12 Q 1 f. Conference calls help me Q 1 g. Email helps me Q 1 h. XSEDE provides me with Q 1 i. XSEDE provides me with communicate effectively with adequate resources for useful resources for other XSEDE staff. collaboration with other staff.

Communication Tools Quotes • “…the movement from Skype for Business to Zoom has been

Communication Tools Quotes • “…the movement from Skype for Business to Zoom has been a huge improvement in terms of meetings/accessibility/ collaboration. ” – 2018 Respondent • “Tools have made a significant improvement over the past two years, that is a positive highlight. ” – 2018 Respondent • “Continued refinement of tools to help manage a large virtual organization [would help me complete my XSEDE work more effectively]. The move to Zoom, JIRA, and Confluence have been big positive steps. ” – 2018 Respondent 13

Decision Making: Staff reported significantly higher decision making ratings this year including increases in

Decision Making: Staff reported significantly higher decision making ratings this year including increases in decision making efficiency, understanding, and learning/adapting based on past experience. (Role-specific items are listed on future slide. ) Decision Making Index Response Over Time, (Scale 1, Strongly Disagree – 5, Strongly Agree) 3. 45 3. 49 3. 64 3. 65 [VALUE]* 2018 (N=117) 3. 55 3. 53 3. 69 2017 (N=98) 3. 77 3. 74 3. 77 3. 95 2016 (N=125) 3. 04 3. 22 3. 16 [VALUE]* 2015 (N=142) 2. 93 2. 96 3. 33 3. 24 [VALUE]** 3 2014 (N=159) 3. 30 3. 57 3. 65 3. 66 3. 61 3. 74 4 3. 16 3. 30 3. 29 3. 15 3. 26 5 3. 30 3. 41 3. 49 3. 39 [VALUE]** 2013 (N=127) 2 1 Decision Making Index Q 2 e. I receive Q 2 g. I receive Q 3 a. XSEDE's decision- Q 3 b. I understand adequate making process is how decisions are communication about efficient. made within the how I will be what is happening organization. evaluated as an XSEDE across XSEDE. staff member. * p <. 05, ** p <. 01, *** p <. 001 14 Q 3 c. I have input in Q 3 d. When decisions Q 3 e. As an decision making that are made, they are organization, XSEDE relates to my work. effectively learns and adapts communicated back based on past to me. experience.

Decision Making Quotes • “I am very happy with how XSEDE is run as

Decision Making Quotes • “I am very happy with how XSEDE is run as an organization and my role in it. ” – 2018 Respondent • “XSEDE has not been allowed to be as innovative as I would like it to be. Given that, I believe that we are underutilizing our staff on XSEDE and their potential creative contributions to the project. ” – 2018 Respondent • “XSEDE is very good at supporting established, familiar user needs, but it isn't structured to put resources toward addressing genuinely novel needs. New system services only seem to appear during the five-year program transitions. Everything in between is just "tweaks" to existing stuff. ” – 2018 Respondent 15

In 2018 item Q 5 “Which of the following responsibilities are part of your

In 2018 item Q 5 “Which of the following responsibilities are part of your current role in XSEDE 2. 0? (Select all that apply)” was added prior to select decision making items. Over one third (35%, 41/117) of respondents and nearly all (97%, 31/32) L 1/2 – 3 managers who responded report utilizing KPIs and metrics to guide program area activities. 2018 Count of Self Reported XSEDE 2. 0 Responsibilities 120 100 80 60 41 40 33 32 35 20 0 Manage budgets 16 Utilize KPIs and metrics to guide program area activities Manage staff Complete quarterly/IPR/NSF reporting requirements

Decision Making: At the project’s request, items Q 3 l-3 p & Q 3

Decision Making: At the project’s request, items Q 3 l-3 p & Q 3 r were added to the decision making index. These items were only visible to respondents with related responsibilities (previous slide). Ratings of repeated questions (Q 3 h & Q 3 J) increased though not significant. Item Q 3 i was separated into Q 3 l & Q 3 n which were rated considerably higher in 2018 than 2017. Role-Specific Decision Making Index Response Over Time, (Scale 1, Strongly Disagree – 5, Strongly Agree) 2017 (N=98) 2018 (N=117) 5 4. 17 4 3. 47 3. 60 3. 32 3. 31 3. 55 3. 34 3. 61 3. 73 3. 67 3. 98 3. 95 3 2 1 Q 3 h. The time Q 3 i. I have the Q 3 j. The KPIs and Q 3 k. I am Q 3 l. I have the Q 3 m. I have the Q 3 n. I have the and effort information I metrics are empowered to information I authority I need information I required to need to adquately effectively used make decisions need to to make decisions need to complete XSEDE manage staff and to guide activities regarding budgets adequately reporting budgets [retired in my area and staffing manage staff manage budgets requirements is in 2017] [retired in 2017] reasonable * p <. 05, ** p <. 01, *** p <. 001 17 Q 3 o. I have the Q 3 p. I have what I Q 3 r. XSEDE is an authority I need to guide innovative to make decisions activities in my organization regarding staffing area with the KPIs and metrics

Decision Making Quotes: Staffing / Workload Balance • “I feel there is still a

Decision Making Quotes: Staffing / Workload Balance • “I feel there is still a disconnect between XSEDE and the local manager sites. It is frustrating managing a virtual team when the local sites abuse power and use funded XSEDE employees to do non-XSEDE work on a regular basis. ” – 2018 Respondent • “I feel that XSEDE work is considered secondary to needs at my local site regardless of how highly funded I am to work on XSEDE and this leads to more work than can be completed in a normal working week. ” – 2018 Respondent • “It is challenging to balance demands of a virtual organization with local site responsibilities. The local site responsibilities will often pull more strongly since the ties to the site organization (and the effects on one's career) are stronger. Thus, having a large amount of effort in a virtual organization tends to lead to over commitment. ” – 2018 Respondent 18

Equity: Unlike previous years where equity items trended down, in 2018 many equity item

Equity: Unlike previous years where equity items trended down, in 2018 many equity item ratings increased though the index was not rated significantly higher than 2017 (p=. 054). While all groups rated the index positively, differences were found between women (M=3. 88) and men (M=4. 25, p=. 012), URMs (M=3. 65) and majority groups (M=4. 17, p=. 016), and PO (M=3. 62) staff compared to other areas (M=4. 20, p=. 002). 4. 02 4. 08 3. 94 [VALUE] 2018 (N=117) [VALUE] 4. 06 3. 74 [VALUE] 2017 (N=98) 4. 23 4. 22 4. 11 4. 25 2016 (N=125) 4. 20 4. 21 4. 04 [VALUE] 2015 (N=142) 4. 18 4. 15 4. 06 4. 22 2014 (N=159) [VALUE] 4. 15 3. 92 [VALUE]* 4. 00 [VALUE] 4. 25 3. 95 [VALUE]* 5. 00 [VALUE] 4. 13 3. 94 [VALUE] Response to Equity Index Over Time, (Scale 1, Strongly Disagree – 5, Strongly Agree) 3. 00 2. 00 1. 00 * p <. 05, ** p <. 01, *** p <. 001 19 0. 00 [VALUE]* 0. 00 Q 2 b. I feel that XSEDE Q 10 a. I feel that XSEDE Q 10 b. I feel that XSEDE Q 10 c. I feel that XSEDE Q 10 d. I feel that XSEDE Q 10 e. I feel that XSEDE Q 10 f. I feel that XSEDE staff treat each other staff are treated staff are treated equally, regardless of equally by each other equally by leadership gender. regardless of regardless of field of race/ethnicity. study/work. [VALUE]* Equity Index

Equity Quotes • “XSEDE leadership (and most staff) have generally been very inclusive and

Equity Quotes • “XSEDE leadership (and most staff) have generally been very inclusive and supportive. ” – 2018 Respondent • “I think the situation has improved with respect to discrimination, although I still detect some times where some staff are not treated well for reasons that do not make sense (i. e. have no basis in reality). ” – 2018 Respondent • “I am only in [one XSEDE program area] at [my center] which I may be overlooking some staff but rarely see any other [people in my racial/ethnic minority group] working with XSEDE, again maybe I am overlooking this and they have more involved outside of [my center] and the [XSEDE program area that I’m in]” – 2018 Respondent 20

In 2018 Leadership/management training was a common response to Q 13. “Thinking about staff

In 2018 Leadership/management training was a common response to Q 13. “Thinking about staff training that could help you perform your XSEDE work more effectively, what training topics would help you most? ” • “Leadership, Diversity and Inclusion, Running Effective Meetings. ” – 2018 Respondent • “Leadership, organizational development (high level), project management (high level), emotional intelligence/difficult conversations, improving communication skills (with users/customers as well as coworkers, effective listening and communication), program evaluation (high level)” – 2018 Respondent • “program management, management in general” – 2018 Respondent • “I'd suggest some leadership and management training for those interested”– 2018 Respondent 21

Inclusion: While the Inclusion index declined slightly in 2018 compared to last year, no

Inclusion: While the Inclusion index declined slightly in 2018 compared to last year, no significant differences were found. RAS respondents (M=4. 33) rated the index higher than other L 2 areas (M=4. 12, p=. 035). Inclusion Index Response Over Time, (Scale 1, Strongly Disagree – 5, Strongly Agree) 2017 (N=98) 2018 (N=117) 3. 81 3. 93 3. 88 3. 65 3. 95 3. 83 2016 (N=125) 4. 83 4. 80 4. 68 4. 71 [VALUE] 2015 (N=142) 4. 31 4. 45 4. 37 4. 35 4. 41 4. 47 2014 (N=159) 3. 78 3. 93 [VALUE] 4 3. 59 3. 79 4. 02 3. 96 4. 23 3. 99 5 3. 95 4. 38 4. 40 4. 00 4. 17 [VALUE] 2013 (N=127) 3 2 1 * p <. 05, ** p <. 01, *** p <. 001 22 0. 00 0. 00 Q 1 k. When I need to, it is Q 1 l. When I need to, it is Q 9. How often do you Q 10 g. I feel supported by Q 10 h. I feel supported by Q 10 i. I am included in non easy for me to contact experience discrimination XSEDE leadership other XSEDE staff to professional activities with XSEDE staff outside of my XSEDE staff within my by other XSEDE staff? to pursue my professional other XSEDE staff (i. e. program area. goals social events, informal meetings, etc. ) 0. 00 Inclusion Index

Recommendations • Communication Tools: Consider transitioning all meetings to Zoom based on positive feedback

Recommendations • Communication Tools: Consider transitioning all meetings to Zoom based on positive feedback gathered this year. • Decision Making/Human Resources: Continue developing an optional staff evaluation to be included in local site evaluations by request. Consider coupling the implementation of new processes with management training. To address workload balance issues, consider allowing the SP forum to review plans and processes. • Equity: Explore how equity can be improved within the organization. Couple these activities with additional staff and management training to improve cultural competence beyond race/ethnicity and gender to include other disciplines/domains. 23

Discussion • How can the annual Climate Study be better leveraged in XSEDE 2.

Discussion • How can the annual Climate Study be better leveraged in XSEDE 2. 0? – Should we be conducting the analysis any differently in 2. 0 (i. e. different group comparisons)? – How can the study be leveraged to aid in transition planning for XSEDE? Or NSF? 24