Likert Scales are the meaning of life Dane

Likert Scales …are the meaning of life: Dane Bertram

Overview » » » Basics Analysis Methods Worked Example Strengths Weaknesses dane. bertram CPSC 681 – Likert Scales February 22, 2021

Basics dane. bertram { Basics } CPSC 681 – Likert Scales February 22, 2021

Likert Scale lick-urt, n. { Basics } » Who? • Dr. Rensis Likert » What? • Psychometric response scale » When? • 1932 » Where? • Questionnaires » Why? • Obtain preference or degree of agreement dane. bertram CPSC 681 – Likert Scales February 22, 2021

Sample Likert Scale Question { Basics } Please indicate how much you agree or disagree with each of the following statements: Strongly disagree Somewhat disagree Neither agree nor disagree Somewhat agree Strongly agree 1. The “U of C • This is now” website is easy to use 1 2 3 4 5 2. The “My U of C” website is easy to use. 1 2 3 4 5 3. The “Peoplesoft Student Center” website is easy to use. 1 2 3 4 5 dane. bertram CPSC 681 – Likert Scales February 22, 2021

Analysis Methods dane. bertram CPSC 681 – Likert Scales { Analysis Methods } February 22, 2021

Analysis Methods { Analysis Methods } » Individual questions (ordinal data) • Descriptive statistics – – – Median Mode Range Inter-quartile range Not mean or standard deviation • Non-parametric tests – Mann-Whitney U test – Wilcoxon signed-rank test – Kruskal-Wallis test dane. bertram CPSC 681 – Likert Scales February 22, 2021

Analysis Methods Cont’d { Analysis Methods } » Summed responses (interval data) • Restrictions – All questions use same Likert scale – Defendable approximation to an interval scale • Parametric tests – Analysis of variance (ANOVA) » Reduced to nominal levels of agree vs. disagree: • Chi-square test • Cochran Q test • Mc. Nemar test dane. bertram CPSC 681 – Likert Scales February 22, 2021

Worked Example dane. bertram CPSC 681 – Likert Scales { Example } February 22, 2021

Worked Example { Example } » Raw data » Descriptive statistics » Mann-Whitney U test Please indicate how much you agree or disagree with each of the following statements: Strongly disagree Somewhat disagree Neither agree nor disagree Somewhat agree Strongly agree 1. The “U of C • This is now” website is easy to use 1 2 3 4 5 2. The “My U of C” website is easy to use. 1 2 3 4 5 3. The “Peoplesoft Student Center” website is easy to use. 1 2 3 4 5 dane. bertram CPSC 681 – Likert Scales February 22, 2021

Raw Data Participant ID { Example } Q 1. U of C Q 2. My U of C Q 3. Peoplesoft 1 4 4 3 2 3 4 3 2 2 3 4 5 3 3 6 4 2 2 7 3 3 3 8 4 4 4 9 3 4 3 10 2 5 2 11 2 4 1 3 2 14 2 2 3 15 4 3 3 16 1 1 2 4 5 12 13 Category MSc Ph. D Table 1. Raw Data dane. bertram CPSC 681 – Likert Scales February 22, 2021

Descriptive Statistics { Example } Median Mode Range Inter-quartile Range Q 1. U of C 3 4 4 2 Q 2. My U of C 3 3 4 1. 25 Q 3. Peoplesoft 3 3 2 1 Range Inter-quartile Range Table 2. Descriptive Statistics 1 A Median Mode MSc Ph. D Q 1. U of C 4 2 3 3 1 1. 5 Q 2. My U of C 3 3 3 4 2 4 1 2. 25 Q 3. Peoplesoft 3 2. 5 3 3 2 1 0. 5 1 Table 3. Descriptive Statistics 1 B dane. bertram CPSC 681 – Likert Scales February 22, 2021

Descriptive Statistics Cont’d Q 1. U of C Q 2. My U of C Q 3. Peoplesoft { Example } Strongly disagree Somewhat disagree Neither agree nor disagree Somewhat agree Strongly agree # 2 4 3 6 1 % 13% 25% 19% 38% 6% # 2 2 6 5 1 % 13% 38% 31% 6% # 0 6 8 2 0 % 0% 38% 50% 13% 0% Somewhat agree Strongly agree Table 4. Descriptive Statistics 2 A Strongly disagree Q 1. U of C Q 2. My U of C Q 3. Peoplesoft Somewhat disagree Neither agree nor disagree MSc Ph. D # 0 2 1 3 2 1 4 2 1 0 % 0% 25% 13% 38% 25% 13% 50% 25% 13% 0% # 0 2 1 1 4 2 3 2 0 1 % 0% 25% 13% 50% 25% 38% 25% 0% 13% # 0 0 1 4 4 4 2 0 0 0 % 0% 0% 25% 50% 50% 25% 0% 0% 0% Table 5. Descriptive Statistics 2 B dane. bertram CPSC 681 – Likert Scales February 22, 2021

Mann-Whitney U test { Example } » Requirements: • Non-parametric test • Two samples must be statistically independent • Observations must be on an ordinal scale » Null hypothesis: • Equal probability that an observation from one sample will exceed an observation from the other sample dane. bertram CPSC 681 – Likert Scales February 22, 2021

Step 1: Calculate U statistic { Example } » Combine observations from both samples » Write down observations in rank-order » Label which sample each observation came from • Alternate between samples when repeated values appear in both Tie Tie Rankordered: 1 1 2 2 3 3 3 4 4 4 5 Origin sample: P P M P M M M dane. bertram CPSC 681 – Likert Scales February 22, 2021

Step 1: Calculate U statistic { Example } » For each observation from sample 1 (MSc students): • Move left-to-right • Count 1 for each observation from sample 2 (Ph. D students) occurring after it in the list • When there is a tie, count 0. 5 » Add all these counts together to form UMSc Count for first MSc response: Tie 1 1 2 2 3 3 3 4 4 4 5 Origin sample: P P M P M M M ++ 11 } Rankordered: Count: = dane. bertram 0. 5 ++ 11 CPSC 681 – Likert Scales ++ 11 = =5. 5 February 22, 2021

Step 1: Calculate U statistic Count for second MSc response: { Example } Tie 1 1 2 2 3 3 3 4 4 4 5 Origin sample: P P M P M M M +1 +1 } Rankordered: 0. 5 Count: = 0. 5 + 1 ++ 11 = =2. 5 5. 5 Count for third MSc response: Rankordered: 1 1 2 2 3 3 3 4 4 4 5 Origin sample: P P M P M M M +1 +1 Count: = dane. bertram 0. 5 +1 CPSC 681 – Likert Scales +11 ++ 11 = =2 5. 5 February 22, 2021

Step 1: Calculate U statistic { Example } » Sum these counts together: UMSc = 5. 5 + 2 + 1. 5 + 0 + 0 = 12 » Similarly, calculate UPh. D: UPhd = 8 + 7. 5 + 7 + 6. 5 + 4. 5 + 3. 5 = 52 » Sanity check: U 1 + U 2 = (# of observations in S 1) x (# of observations in S 2) dane. bertram CPSC 681 – Likert Scales February 22, 2021

Step 2: Critical values table { Example } » Use the lower of the two U stats (UMSc = 12) with: n 1 = # of observations in S 1 n 2 = # of observations in S 2 α = level of significance » If below table value: • Reject null hypothesis • Samples are significantly different at the α level Table 6. Mann-Whitney U Distribution Critical Values dane. bertram CPSC 681 – Likert Scales February 22, 2021

Why does it not suck? { Strengths } » Simple to construct » Generally a highly reliable scale » Easy to read and complete for participants » Fun for the whole family! dane. bertram CPSC 681 – Likert Scales February 22, 2021

Ok then, what does suck? { Weaknesses } » Central tendency bias • “Fence riding” » Acquiescense bias • Try to “please” you » Social desirability bias • Portray themselves favourably » Lack of reproducibility » Difficult to demonstrate validity • Did you measure what you set out to measure? dane. bertram CPSC 681 – Likert Scales February 22, 2021

Questions? Thank you! Dane’s presentation totally rocked my socks: