Current status of Blue Implementation of Blue is

Current status of Blue Ü Implementation of Blue is effective since October Ü Presentations to Colleges/Schools and Departments are currently ongoing Ü The Electronic Course Assessment Implementation (ECAI) committee (subcommittee of the FDAI committee) is supervising the implementation, in conjunction with the Provost’s Office and OIT Ü ECAI is the point of contact for the Faculty Senate with regard to all issues about electronic evaluation Ü You can find information about online course evaluation at: https: //www. uaf. edu/provost/blue/

Adoption of Blue Ü What are the implications Ü What could be done to improve confidence on the survey for each class

Implications of adopting Blue Inter-system comparison Ü Consistency of evaluation over time Ü Quality Intra-system comparison Ü Representativeness Ü Accuracy Ü Quality

Inter-system comparison Ü Q: How my evaluation in Blue measures against IAS? Ü Qualitatively, the two questionnaire are different (no “effectiveness of teaching” is assessed). Ü Quantitatively, the scores are expected to be different because the structure and the content of the surveys are different. Ü Each Blue survey should be compared to Campus-wide Blue aggregate data. Ü Few rounds of evaluation will be needed to be able to make comparison with IAS, if needed. Ü Revise unit criteria for tenure and promotion.

Inter-system comparison: quality Ü Q: Students who are strongly negative about the course or the instructor have been the most likely group to complete the online evaluation. Ü Results from many studies on this topic have proven this to be a misconception, with results from online evaluations shown to be as trustworthy as those from paper-based evaluations (Liu, 2005; Thorpe, 2002; Johnson, 2002). Ü A large scale study of the results of the Individual Development and Educational Assessment (IDEA) student rating system between 2002 and 2008 (Benton et al. , October, 2010) examined a total of 651, 587 classes that used paper-based evaluations and 53, 000 classes that used web-based evaluations. This comparison showed no meaningful differences between survey methods.

Intra-system comparison: representativeness Ü Q: Those who have responded to the survey have very different views than those who have not, hence the results from the survey would not reflect the opinion of the population as a whole. Ü The link between response rate and non-response bias has not been established (Marketing Research and Intelligence Association, October 2003 and 2011; Curtin et al. , 2000, Langer, 2003; Holbrook et al. , 2005).

Intra-system comparison: accuracy Ü Q: Small sample size leads to greater margin of error of the results. Ü Depends on the class size and the opinions’ skewness

What can be done to increase confidence toward this system Ü Response analysis will reveal biases and potential correlation with the polled cohort. Ü Results of the analysis will be shared with students and instructors. Ü If the wording and/or the content of one or more questions appear to skew quality of responses, those questions will be re-evaluated and re-worded or eliminated/substituted.

What can be done to increase confidence toward this system Ü Response rates: Ü Showing evaluation matters Ü Communication Ü Making it easy for students Ü Providing incentives Ü UAF evaluation portal: www. uaf. edu/inspireus Ü Intensify the use of Blue: Ü Mid-term evaluation Ü Department-specific questions