Development and Testing of a Survey Measure Training

Development and Testing of a Survey Measure Training Effects on The Culture of Safety to

Utah State University Mark Mc. Lellan

What is it? Where Does it Come From? CULTUR E of SAFETY How Do We Measure It? How To Change It?

Research Pedigree • Dr. Goldberger • Dr. Evans • Dr. Dauben • Dr. Poulter • Me

Pellagra 1912 Dr. Joseph Goldberger

Experiment 1 Orphanage

Experiment 2 Mental Asylum

Experiment 3 Filth Parties

Measurement Attitude Culture of Safety Knowledge Behavior

Knowledge Acquisition

Attitude Toward Safety

Predicting Behavior

Vignette • You are working alone in you lab when you find a 4 -L amber glass bottle sitting on another persons lab bench labeled as “Experiment 113 bdj”. The container is hissing (venting gas) from around the lid.

Expert Panel Name Michael Blaney Jeff Christensen Kent Clawson Sean Collins Joy A. Diaz Mary Duda Michele Edenfield Gordon Evans Jerry Gordon Heather Jackson Hans Nielsen Fred Miller Laurie St. Clair Affiliation Northwestern University of Arizona King Abdula University Santa Clara University Whitworth University Creighton University Emory University Texas A&M Cornell University of Illinois/Chicago Hawaii University Whitman College Oklahoma State University e-mail address Michael. blayney@northwestern jgchrist@email. arizona. edu kent. clawson@kaust. edu. sa spcollins@scu. edu jdiaz 02@whitworth. edu mjduda@creighton. edu Michele. edenfield@emory. edu gevans@tamus. edu jpg 29@cornell. edu heather 9@uic. edu hansn@hawaii. edu millerfl@whitman. edu laurie. stclair@okstate. edu

Pilot Survey • • • Section Number of Questions Demographics 5 Learning Style 4 Course Evaluation 12 Attitude Toward Safety 12 Behavior Vignettes 24 – 4 Scenarios each with 6 solutions

Likert Scale Strongly Disagree Somewhat Neutral Somewhat Agree Strongly Disagree Agree A Discrete 7 -point Likert Scale can be considered an Interval scale rather than Ordinal for statistical calculations

Pilot Survey School USU (as part of RLST) USU Chemistry Department (9 months) Uof. U (6 months) Uof. H (12 months) Total Out Returned Rate 116 72 62. 1% 50 22 44% 75 25 266 30 13 131 40% 52% 49. 2%

Survey • • Section Number of Questions Demographics 4 Course Evaluation 10 Attitude Toward Safety 10 Behavior Vignettes 9 – 3 Scenarios each with 3 solutions

Survey School Auburn BYU LSU UIC UNR USU Total Out 75 40 200 100 341 856 Returned 35 15 81 44 47 171 393 Rate 46. 7% 37. 5% 40. 5% 44. 0% 47. 0% 50. 1% 45. 9%

Number of Science Courses 20% MS/Ph. D 30% 4 yr Degree 14% Undergrads 30% Upper Level Undergrads

Major 60% Physical Sciences 17% Engineering

Laboratory Experience 55% Research Laboratory Experience

Previous Lab Safety Training 42% Previous RLST 13% None 31% Teaching Laboratory

Accidents 2% Near Miss 1% Sever 34% Minor 62% Never

Course Evaluation Cronbach’s α – 0. 853

Attitude Cronbach’s α – 0. 732

Behavior Predictions Cronbach’s α – 0. 704

Overall Survey Cronbach’s α – 0. 824

Factor Analysis SRMR – 0. 0669 CFI – 0. 904

Criterion Validity Rank 10 9 8 7 6 5 4 3 2 1 Total Number sent Labs People 0 (1 BE/1 Bio) (2 Bio) (3 Bio/1 ME) (4? Bio) (3 Bio/1 ME) (2 Bio/2 ME) (2 ME) 28 0 6 20 11 13 12 16 19 9 7 113 Returned % 0 5 7 7 6 13 4 6 62 83. 3 35. 0 63. 6 53. 8 58. 3 37. 5 68. 4 44. 4 85. 7 54. 9 BE – Biological Engineering, Bio – Biology, ME – Mechanical Engineering

Criterion Validity r - 0. 615 2 r – 0. 379

http: //rgs. usu. edu/ehs-tools/

Summary • Reliability is High, α – 0. 824 • Content Validity – Approved by the Expert Panel • Criterion Validity – Pearson Correlation – r = 0. 615, p=0. 0001 • Construct Validity – CFA – 3 factors, 37. 9 % of Variance Explained • Final Test – Need a Large Scale Broad Spectrum Experiment – 500 Volunteers at 10 Different Institutions