Impact Outcome Evaluation Quarterly Call January 19 2021

Impact Outcome Evaluation Quarterly Call January 19, 2021 Hosted by: Amanda Linares

• Announcements and reminders • FFY 20 analysis • Communications updates • EATS administration updates • Q&A

Announcements • Next IOE Quarterly Call: April 20, 2021, 10 -11 am • Cross-Project Office Hours on hold • If I haven’t heard from you this year, be sure to check in with me ASAP! • Keep an updated Planning Worksheet on hand

Reminder! Attrition Log • As you conduct pre and post-testing, don’t forget to complete your Attrition Log • https: //ucanr. edu/sites/ioe//FFY_20 -22_Resources/ • Purpose: • Capture completes(Qualtrics)/total students present (Attrition Log) and number of students who actively decline participation (Attrition Log) • For both USDA reporting and eventual publications

FFY 20 Recap: The Good! • Collected over 6, 000 pre-tests • Breakdown: • 47 intervention site pre-tests, 40 intervention site SLAQs • 22 comparison site pre-test, 22 comparison site SLAQs • Remember, that pre-test data will be used in the 3 -year cross-sectional analysis!

And the not so good. . • Due to COVID-related shutdowns: • Very little post-testing took place…we were a few weeks shy of beginning • 156 matching pre-post from 2 sites who post-tested early due to classroom teacher need/request • Sample size was far too low to expect any significant results • No change in dietary or physical activity behaviors

Communications Updates

Communications Updates • Thanks to all of you for filling out the NPI Evaluation Communications Survey last fall • We learned that: • There are too many websites housing evaluation materials; it can be confusing and you aren’t always confident about where to go to get what you need • 50: 50 split in preference for our means of communication

CFHL LHD Evaluation Landing Page • NEW! CFHL LHD Evaluation landing page: https: //ucanr. edu/sites/LHDEvaluation/ Bookmark! • Provides links to all NPI-led evaluation project websites in one place • Includes contact person/email for each project

NEW! IOE Website • New IOE website (no UC ANR account required): https: //ucanr. edu/sites/ioe/ Bookmark this one too! • Provides all the same project resources as the IOE Collaborative Site • I will maintain both! • Do still request Collaborative Site access (new staff) • All project communication comes from this listserv

Share. Point Project Folders • Evaluation project folders on our NPI Share. Point • Folders will be organized by project and year • Allows us to share (both ways!) documents privately and individually • Project director access, you can invite others as directed by PD • If you have questions or haven’t received an invite contact: Evaluate. SNAPEd@ucanr. edu • CDPH project officers will also be granted read access

Eating and Activity Tool for Students

Progress • As of 1/15… • I have data from 13 (of 21) LHDs • 38 schools (for reference, we had 75 sites in FFY 20) • 30 intervention sites and 8 comparison sites • 2, 494 total pre-tests • However, this includes many non-valid surveys and repeated IDs • Reach out if you’d like an update on your progress!

Administration Best Practices 1. LHD staff/subs administration (if distance learning) during synchronous learning 2. Teacher administration during synchronous learning 3. Self-guided during asynchronous learning X

Administration Best Practices ü Work with teachers ahead of time üAcquire class lists for ID assignment ü Inquire about best time of day to capture the most students üRun through survey together and see if there are questions with answers that would be consistent for the entire class üi. e. If all students attended yesterday/last week in the same manner (Q 5/Q 25); recess (Q 26); PE (Q 28)

Issues so far… • Repeated IDs • Students entering erroneous ID, identical IDs assigned to multiple students at one site, or same student entering into survey link multiple times (this may be unavoidable) • Skip sequences • Be aware of them if completing the survey question-by-question as a group

And finally! • Let’s work together to figure out when to “close” pre-testing and “open” post-testing • I will likely reach out in the next month or so to confirm all planned pre-tests have occurred • Close the pre-test sometime in February/March? • As soon as you have a date in the books for post-testing (6 weeks or less from last day of school), alert me and I will open post-test • Even though you likely already have post links, do alert me so I open the survey in Qualtrics