Data Decisions Telling YOUR Story Gina M Vanacore

Data, Decisions, & Telling YOUR Story • Gina M. Vanacore, • Executive Director, Housing & Residence Life, University of North Texas • HERE IN THOUGHT : Lindsey Fields, M. S. , M. Ed. , LPC • Assistant Director for Residence Life

Why Do We Assess? Accountability Institutional and Programmatic Improvement The relentless pursuit of perfection through innovation drives continuous improvement.

About You: Raise your hand if… Time in our field is less than three years ? Originally from Texas ? First generation college student ? You have a master’s degree ? Prior assessment experience ?

Take A Moment Please draw a Residence Life staff member conducting assessment activities: - What is this person doing? - If there were posters on the walls describing the work being done what would they say? - What items are on this person’s desk?

Why do we see this in that way? How does that vision differ from: • Your images of describing what we do to interested parties • How we visualize staff telling stories about the student experience • How you communicate your achievements

Iterative Model Does your Program, Department, and Division have a Strategic Assessment Plan? If not, what expectations are put on your Program, Department, or Division about how, when, and how often to communicate assessment results? Outcomes Assessment Intervention Assessment Outcomes

Access & Analyze Data Commit to Routines for Evaluation Understand Student Pathways Create Channel for Policy & Practice Change Connect with Campus Change Agents Communicate Data Findings

Dreams and data are interconnected • Every student has a dream and through our efforts we either help them achieve it or put a roadblock in their path • Data are just numbers unless residence life staff can use it to close loops, make new meaning and improve our programming and services • SO…. I believe effort in assessment is a vital component to helping more of our students succeed and vital to that is telling our story in a • Compelling • Interesting • Accurate • Persuasive ADD MORE WORDS HERE ……. . manner

Stories are how we build empathy

Tell Our Story, Build Empathy, Shape Our Culture Storytelling Empathy Culture

Where is the story telling in the assessment cycle ? Develop (Division/Dept. ) Goals Inform Decisions & Practces Identify Outcomes Review & Interpret Assessment Results Connect with programs, services, and learning opportunities Gather Data

DESIGN THINKING dschool. standford. edu From Stanford d. School: dschool. standford. edu

DESIGNED ASSESSMENT THINKING Lesley D’Souza, 2017 Empathize Storytelling Define Goals & Outcomes Implement Assess Interpret Ideate Pilot

Assessment 10 Years Ago and Now Satisfaction-Oriented Indirect Evidence-Based Technologically Limited Justification-Focused Methodological Simplicity Bottom-Up Culture Some Master’s level expertise needed • One person shops (usually part of shared duties) • Isolated professional opportunities • • • • Learning Outcome Oriented Indirect and Direct Evidence-Based Technologically Expanding Complex Foci Depending on Goals/Needs Increasing Study Complexity and Analysis Bottom-Up and Top-Down Culture Master’s level expertise the norm, increasingly Doctoral level expertise needed Increasing staff sizes NILOA, SAAL, ASSESS, NASPA and ACPA growth

7 Key Areas of Assessment • Cultural - Is your department ready to assess? • National Standards – How do you compare to your peers? What does CAS state? • Mission – Is your mission relevant? Current? • Users – Who actually uses your service? How do they match the larger population? • Needs – What do your residents need? What are you offering? Is it still relevant? • Cost Effectiveness – Do residents perceive value? Are you using resources wisely? • Learning Outcomes – What are residents learning? How do you know? Do you integrate learning outcomes into programming?

What are some common assessment methods you have heard of or used?

Our Assessment Toolbox • Online surveys • Paper-based surveys • Focus groups • National studies • Historical document review • Secondary data analysis • Reflective journals • One minute essays • Photo analysis • Process cartoons • Expense categorization Pre and post tests Marketing/brand encounters Institutional data mining Social media analysis Environmental scans Student art opportunities Campus poster review Direct evidence (ID card scanning and data capture) • Staff initiated rubrics • Student self-scored rubrics • •

Resources for Understanding Data • National Center for Higher Education Management Systems • http: //www. higheredinfo. org/resources. php • National Surveys • Institutional Resources • Institutional Research Office • Graduate programs • Qualtrics or Survey Dashboard

Assessment from Problem to Report How Might You Frame Out an Approach? Step 1: Identify the problem Step 8: Determine who should collect the data Step 9: Determine how the data will be analyzed Step 2: Determine the purpose of the study Step 7: Determine what instruments will be used Step 10: Determine the implications of the study for policies and practice Step 3: Determine where to get the information needed Step 6: Determine how data will be collected Step 11: Report the results effectively Step 4: Determine the best assessment methods Step 5: Determine whom to study Get some rest for a day, then implement changes tomorrow! Source: Schuh, J. H. , Upcraft, M. L. & Associates. (2001)

Step 10: Determine the implications of the study for policy and practice We suggest you rely on your findings and present how your findings impact the campus or program area. • What are the implications of this study? • What approaches to solving the problem should be considered in light of this study? • What policies and procedures need to be created or overhauled? • What is the call for action for the UNIT?

Step 11: Report the results effectively • Understand we must package the findings carefully so that it motivates change • How a study is distributed and formatted may be more important than the results found • Segment your audience and tailor specific reports to specific audiences • Make sure the results get into the right hands of people who can make a difference

Communicate Data Findings They know I’m paying attention to it (data), that I care about it, that I talk to them about it. Monitoring data and responding to it can be a powerful culture change on campus. TEACH

Sharing YOUR Story • Inside your department • Outside your department, within your institution • Outside your institution Examples on the following slides may or may not be representative of actual data/results from the University of North Texas TEACH

Other Institutional Practices – Ohio State

Other Institutional Practices – Southern Illinois

Longitudinal Data Highlighting your own growth

What stood out to you about these schools and how they translate data into action?

Elements of Storytelling From www. forbes. com: Data Storytelling: The Essential Data Science Skill Everyone Needs, 2016

COMBINED Elements of Storytelling From www. forbes. com: Data Storytelling: The Essential Data Science Skill Everyone Needs, 2016

Executive Summaries (cont. ) • Standard length – 2 pages • Should be written in jargon-free language • Content can be used for: • annual reports • performance evaluations • Resumes • Conference proposals • Grants from NASPA, ACPA, etc. • reaccreditation • website content • print marketing • speeches

Assess YOUR Level of Sharing YOUR Story • Assessment data and analysis is regularly communicated to campus partners and other relevant educational partners. • Assessment data and analysis is regularly communicated to the professional staff members and educators who design and execute the program/initiative being assessed. • Assessment data and analysis is regularly communicated to student staff members who execute the program/initiative being assessed. • Assessment data is regularly analyzed and used to enhance the effectiveness of the program/initiative as a part of a commitment to continuous improvement.

Assess YOUR Level of Sharing YOUR Story • A review process is developed whereby the entire program/initiative is reviewed by educational experts who provide guidance on its improvement. • A review team incorporates a diverse set of internal and external voices – including students, staff, faculty, campus partners, and educational experts. • Suggestions for improvement made through a review process are acted upon and incorporated into future planning.

Concluding Thoughts

Why Assessment Takes Partners • This work builds community from a data-driven perspective • This work helps inform what students need from you the most • This work provides your Department with a platform to talk about itself • This work can help you become more marketable in the highered workspace • This work can make all the difference in the world to our students – remember we are in the dreams business

One final drawing: Draw another Housing staff member conducting assessment? - What is this person now doing? - What items are on this person’s desk? - If there were posters on the walls describing the work being done what would they now say?

I love to talk about assessment, research and my job…. You can reach me at gina. vanacore@unt. edu 940 565 2282