Diversity and Inclusion XSEDE RAS XRAC Users XSEDE

Diversity and Inclusion: XSEDE, RAS, XRAC, Users XSEDE RAS Retreat, September 2017 Lorna Rivera, Research Faculty Georgia Institute of Technology

Outline • XSEDE and RAS Climate (10 min) • XRAS Post Allocation Submission Survey Dimensions (5 min) • Diversity and Inclusion Activities (45 min) • Implications for RAS (5 min)

Climate Study Findings • 90% (9/10) RAS Response Rate to Climate Study • Invitations were sent to staff who charged FTE in April 2017 • Further disaggregation not conducted to protect confidentiality due to relatively small size of group • RAS rated all items similar to non-RAS staff

Climate Study Index Ratings Comparisons: RAS and XSEDE 5 4, 17 4, 09 4, 02 3, 95 4 3, 22 4, 24 4, 17 4, 00 4, 04 3, 59 3, 67 3, 39 3, 84 3, 87 3, 56 3, 22 3 2 1 Communication Tools Decision Making Equity Inclusion RAS (N=9) Leadership & Management XSEDE (N=98) Support & Belonging Value & Satisfaction Wiki & Website

Climate Study Index Ratings by L 2 Area 3, 62 3, 43 3, 65 3, 72 ]VALUE*[ 3, 22 3, 84 4, 00 3, 80 3, 75 3, 77 3, 80 3, 59 3, 82 3, 73 3, 57 3, 52 3, 73 4, 00 4, 07 3, 96 4, 15 3, 94 4, 02 4, 24 4, 30 4, 13 4, 04 4, 13 3, 99 4, 06 3, 65 3, 59 4, 02 3, 92 4, 04 3, 22 3, 39 3, 40 3, 33 3, 24 3, 45 4 4, 17 4, 14 3, 96 4, 02 4, 00 4, 11 5 3 2 1 Communication Tools Decision Making RAS (N=9) * p <. 05, ** p <. 01, *** p <. 001 Equity CEE (N=20) Inclusion ECSS (N=23) Leadership & Management XCI (N=13) Support & Belonging Operations (N=20) Value & Satisfaction Program Office (N=9) Wiki & Website

RAS Decision Making Index Ratings 3, 22 2017 Index: Decision Making (N=9) Q 2 e. I receive adequate communication about how I will be evaluated as an XSEDE staff member. (N=9) 3, 00 3, 56 Q 2 g. I receive adequate communication about what is happening across XSEDE. (N=9) Q 3 a. XSEDE's decision-making process is efficient. (N=9) 3, 00 Q 3 b. I understand how decisions are made within the organization. (N=9) 3, 00 3, 78 Q 3 c. I have input in decision making that relates to my work. (N=9) 3, 67 Q 3 d. When decisions are made, they are effectively communicated back to me. (N=9) 3, 33 Q 3 e. As an organization, XSEDE learns and adapts based on past experience. (N=9) 3, 29 Q 3 h. The time and effort required to complete XSEDE reporting requirements is reasonable (N=7) 3, 14 Q 3 i. I have the information I need to adquately manage staff and budgets (N=7) Q 3 j. The KPIs and metrics are effectively used to guide activities in my area (N=7) 3, 00 Q 3 k. I am empowered to make decisions regarding budgets and staffing (N=8) 3, 00 1 1, 5 2 2, 5 3 3, 5 4 4, 5 5

RAS Decision Making Index Response, N=9 44% Q 2 e. I receive adequate communication about how I will be evaluated as an XSEDE staff member. Q 2 g. I receive adequate communication about what is happening across XSEDE. 11% 22% Q 3 a. XSEDE's decision-making process is efficient. 11% 22% Q 3 b. I understand how decisions are made within the organization. 11% 22% Q 3 d. When decisions are made, they are effectively communicated back to me. 11% Q 3 e. As an organization, XSEDE learns and adapts based on past experience. 11% 44% 33% 22% 44% 14% 43% Q 3 i. I have the information I need to adquately manage staff and budgets 14% 43% Q 3 j. The KPIs and metrics are effectively used to guide activities in my area 14% 11% 56% 11% 14% 29% 50% Agree 56% 43% 57% Neutral 11% 29% 25% Disagree 22% 33% 0% Strongly Disagree 11% 22% Q 3 h. The time and effort required to complete XSEDE reporting requirements is reasonable Q 3 k. I am empowered to make decisions regarding budgets and staffing 11% 56% 33% Q 3 c. I have input in decision making that relates to my work. 22% Strongly Agree 25% 100%

RAS Decision-Making Comments • “The XSEDE budgeting process is such that I feel few, if any, staff are "empowered" to make substantial budget and staffing decisions, and that few opportunities exist to make substantial budget/staffing decisions. ” • “I feel that the more of my funding that is from XSEDE, the smaller any raises from my local site get because the work I'm doing for XSEDE is not directly useful or noticeable to my local site and people getting to make contributions to my local site are going to be better rewarded. I don't think there is any performance evaluation in XSEDE. Sometimes I think XSEDE management isn't paying attention to what is happening in the world outside of XSEDE and tries to solve problems that don't need solved. ” • “There is rarely any reasoning given for why high level XSEDE decisions are made or who they benefit. XSEDE tends to do the same things over and over without making any improvements necessary to make subsequent attempts go better. Most metrics throughout XSEDE are shockingly silly as any kind of real performance measure. ” • “Deal with the continuous reporting cycle. The first "year" of XSEDE has felt like an unbroken series of reporting efforts. Less reporting would mean more actual work to report on. ” • “The level 2 and 3 managers across program areas need to be better coordinated. ”

Questions on Climate Study?

XRAS Post Submission Survey: Dimensions • Submission Process (14 items) • XSEDE Staff Support (4 items) • Motivation For Choosing XSEDE (4 items)

Submission Process (14 items) Scale 1, Not at all satisfied – 5, Extremely Satisfied Mean=3. 85, SD=. 77, N=876 • • 7 H. Level of detail required relative to the type and/or size of submission request • • 7 G. Documentation available regarding resources available for request • • 7 D. The time I spent preparing my required supporting documents • • 7 C. The documentation available to me for preparing my allocation request • • 7 B. XSEDE allocation procedures • • 7 A. XSEDE allocation policies/guidelines • • 13 H. Ease of finding information about your prior allocation request. 13 G. Email notifications from the submission system 13 F. Availability of online help to complete the submission form 13 E. Time spent completing the online submission form 13 D. Interactive responsiveness of the online submission system 13 C. Visual appeal of the online submission system 13 B. Ease of use of the online submission system 13 A. Amount of information required to complete the online submission form

XSEDE Staff Support (4 items) Scale 1, Not at all satisfied – 5, Extremely Satisfied Mean=2. 70, SD=1. 57, N=873 • 7 J. Usefulness of the reviewer comments related to my allocation request • 7 I. The outcome of the allocation request I submitted • 7 F. Guidance provided by XSEDE staff related to my request • 7 E. Availability of XSEDE staff to help me prepare/complete my request

XSEDE Staff Support Items Response Scale 1, Not at all satisfied – 5, Extremely Satisfied Item Mean Std. Deviation N 7 E. Availability of XSEDE staff to help me prepare/complete my request 2. 72 2. 14 872 7 F. Guidance provided by XSEDE staff related to my request 2. 72 2. 14 870 7 I. The outcome of the allocation request I submitted 3. 05 2. 03 863 7 J. Usefulness of the reviewer comments related to my allocation request 2. 29 2. 03 863

Motivation for Choosing XSEDE (4 items) Scale 1 not at all influential – 5, extremely influential Mean=3. 03, SD=. 75, N=871 • 4 D. I wanted specialized usage or provisioning support methods that are unavailable to me elsewhere (i. e. cloud-based access, virtual clusters, data services, gateway support). • 4 C. I wanted access to specialized hardware components that are unavailable to me elsewhere (i. e. accelerators, large memory, etc. ) • 4 B. I wanted to run jobs larger than other resources available to me. • 4 A. My jobs are relatively small but I can't run enough of them on my campus or from other sources outside of XSEDE.

Submitters of Research allocations are significantly less satisfied with staff support than Startup and Education allocation submitters ( p<. 001). This group is also significantly less satisfied with the submission process than Startup submitters (p<. 01). 3, 62 3, 88 ]VALUE**[ 3, 78 [VALUE] ]VALUE***[ 3, 33 [VALUE] 3 3, 00 3, 01 4 3, 30 5 ]VALUE***[ Index Scores by Allocation Type, N=871 Scale (1, Not at all satisfied/influential – Extremely satisfied/influential) 2 1 Motivation for Using XSEDE Research (N=483) * p <. 05, ** p <. 01, *** p <. 001 Staff Support Startup (N=315) Education (N=49) Submission Process Campus Champion (N=24)

No significant differences were found between submitter roles on the project. Index Scores by Project Role, N=857 Scale (1, Not at all satisfied/influential – Extremely satisfied/influential) 5, 00 3, 87 3, 74 3, 85 4, 00 3, 02 2, 94 3, 11 2, 98 2, 95 3, 00 2, 72 2, 77 3, 98 2, 79 2, 88 2, 53 3, 55 I provide high level or administrative oversight to others. (N=92) I provide general suggestions or research troubleshooting to others. (N=57) I am actively engaged with others in conducting this research. (N=354) I perform the majority of the work for this project with some help from others. (N=260) I am the sole researcher on this project. (N=90) 2, 38 2, 00 I provide administrative assistance. (N=4) 1, 00 Motivation for using XSEDE * p <. 05, ** p <. 01, *** p <. 001 Staff Support Submission Process

Submitters identifying as Administration or Executive Leadership had significantly different motivations for using XSEDE than graduate students (p<. 05). No other differences between job title/academic status were found. Index Scores by Academic Status/Job Title, N=838 Scale (1, Not at all satisfied/influential – Extremely satisfied/influential) 2 1 Motivation for using XSEDE Staff Support Submission Process Administration or executive leadership (N=22) Faculty (N=458) Post-doctoral fellow (N=171) Graduate Student (N=108) Staff (N=77) * p <. 05, ** p <. 01, *** p <. 001 Undergraduate Student (N=2) 3, 75 4, 07 3, 88 3, 9 3, 85 3, 81 2, 88 3, 00 2, 77 2, 59 2, 88 2, 93 3, 13 [VALUE] 3, 02 3 3, 08 4 ]VALUE*[ 5

Submitters from small institutions were significantly less satisfied with staff support than those from medium institutions (p<. 01) or non-academic institutions (N/A, p<. 05). Submitters from large institutions were also significantly less satisfied with staff support than those from medium institutions (p<. 05). Index Scores by Academic Status/Job Title, N=838 Scale (1, Not at all satisfied/influential – Extremely satisfied/influential) 5 3, 81 4 3 3, 05 2, 99 3, 05 3, 01 3, 82 ]VALUE*[ ]VALUE**[ 2, 34 3, 97 ]VALUE*[ 2 1 Motivation for using XSEDE Staff Support Submission Process Small (< 3, 000 degree-seeking students), N=88 Medium (3, 000 -10, 000 degree-seeking students), N=164 Large (> 10, 000 degree-seeking students), N=586 Not applicable, N=34 3, 88

Questions about XRAS Post Submission Dimensions?

Diversity and Inclusion Activities • Activity #2: Reflect on Unconscious Assumptions (10 min) • Activity #3: Implications of Diversity Research (15 min) • Activity #4: Case Studies (20 min)

Now What? • How does the information we discussed today affect RAS? XSEDE?