Evaluating Mobile Science Education Programs for Youth Best

Evaluating Mobile Science Education Programs for Youth: Best Practices & Lessons Learned Christina Olenik, MSW, Ph. D & Kathleen Dowell, Ph. D Partners in Evaluation & Planning

Mobile Science Education Programs l Md. Bio Lab l l l Mobile science lab on tractor trailer 13 high schools; 9 jurisdictions; 841 students Visits school 1 week per year – paid instructors Expected impact: science literacy & student career interest Discover Genomics l l Mobile science lab on modified motor coach bus 14 middle schools; 5 jurisdictions; 1, 288 students Visits school 3 -5 times per year – paid instructors Expected impact: science knowledge & interest in science

Mobile Science Education Programs l Georgia State Bio-Bus l l Mobile science hands-on presentations using modified coach bus & a van 4 high school visits; 3 middle schools; 1, 178 students Visits classes as requested – graduate school fellows are the instructors Expected impact: exposure to science concepts & excitement/enthusiasm for science

Evaluation Methods l Md. Bio l l l Student surveys – response rate average 40% Student focus groups – 3 schools, 27 students Teacher surveys – 68 teachers Teacher interviews – 7 invited, 1 agreed Discover Genomics l l Student surveys – response rate average 59% Teacher surveys – 13 surveys

Evaluation Methods l Georgia State Bio-Bus l l l Student surveys – unsure of response rate Teacher surveys – 7 teacher surveys Fellows focus group – 7 participants

Findings/Best Practices l Md. Bio l l l Significant increases in knowledge for 3 of 5 labs Slight (but significant) increase in science interest Teachers felt curriculum was excellent, liked the instructors & equipment Students very much like having the hands-on lab experience They like getting out of the classroom They would like more detailed career information

Findings/Best Practices l Discover Genomics l l Significant increases in knowledge for all labs No increase in science interest Teachers felt the curriculum was very good or excellent, liked the labs, equipment, & staff Georgia State Bio-Bus l l l Slight (but significant) increase in science interest Slight (but significant) increase in confidence of science abilities Teachers were satisfied with presentation & felt the program worthwhile

Lessons Learned for Evaluation l Student Surveys l l Evaluation integrated into curriculum works best Pre and post test done with each lab visit worked better All materials should be provided to teachers for returning survey packages (envelopes, postage) Teacher Data l l For Discover Genomics, meeting with participating teachers secures buy-in Teachers very busy – surveys are better than interviews or better done informally during focus group visits

Lessons Learned for Evaluation l Focus Groups l l Student focus groups beneficial, but not necessary every year Round robin focus groups work best with middle schooler’s – sometimes with high schooler’s too Works to have teacher in the room, but not participating Overall Challenge l Programs disappointed over little impact on science interest