Scaling Survey Research Questionnaires and Interviews Both experimental

Scaling

Survey Research • Questionnaires and Interviews • Both experimental and nonexperimental research • Read pages 212 through 223 in Martin • Scaling = construction of instruments for measuring abstract psychological constructs.

Thurstone Scales: Method of Equal-Appearing Intervals • Define the Concept • Generate Potential Scale Items – about 100 statements – differ with respect to the extent to which agreement indicates presence of the attribute to be measured

Thurstone • Rating the Potential Scale Items – Judges rate the items on 11 -point scale – 1 = agreement indicates very low amount of the attribute – 11 = agreeing indicates very high amount of the attribute – encourage judges to use entire range of scale, assigning some statements to each of the 11 values – sort them into 11 piles.

Thurstone • Computing the Scale Score Values for Each Item – Find median and SD or inter-quartile range – Arrange in table – Sort by median – Within items with same median, sort by SD or inter-quartile range

Thurstone • Select the Final Scale Items – 1 (or 2 or 3) item(s) for each possible scale score value – Prefer items with low variability among judges – End up with 10 -30 items – See this example – Items and scale score values are shown on the next slide.

• People with AIDS deserve what they got. (1) • AIDS is good because it helps control the population. (2) • AIDS will never happen to me. (3) • I can't get AIDS if I'm in a monogamous relationship. (4) • It's easy to get AIDS (5) • Because AIDS is preventable, we should focus our resources on prevention instead of curing (5)

• People with AIDS are like my parents (6) • If you have AIDS, you can still lead a normal life (8) • AIDS doesn't have a preference, anyone can get it (9) • AIDS is a disease that anyone can get if they are not careful (9) • Aids affects us all (10) • People with AIDS should be treated just like everybody else. (11)

Thurstone • Administer the Final Scale – Randomize the order of the items – For each item, respondent chooses Agree or Disagree – For each item the scale score is the median from the judges’ ratings – Total Score = mean scale score for items on which the respondent agreed.

Thurstone • Thurstone scales are rarely used these days • They are just too much trouble to create. • Likert scales were developed in response to this difficulty

Guttman Scaling • Define the Concept • Generate Potential Scale Items • Evaluating the Potential Items – For each item, judges are asked if someone high in the attribute would agree with the statement – Yes or No.

Guttman • Conduct a Scalogram Analysis of Judges’ Responses – use special software to do this – if successful, it will create an ordered list of items such that • agreeing with the first item indicates you have at least a little of the measured attribute • agreeing with the second indicates you have at least a little more of the attribute • etc.

Guttman – The scalogram analysis also computes a scale score value for each statement. – See the example in Trochim’s Internet document Guttman Scaling (reproduced on the next slide). – It is assumed that anybody who would agree with the nth item would also agree with all preceding items. – The order of the items may be scrambled prior to administering the scale.

• I believe that this country should allow in more immigrants. • I would be comfortable with new immigrants moving into my community. • It would be fine with me if new immigrants moved onto my block. • I would be comfortable if a new immigrant moved next door to me. • I would be comfortable if my child dated a new immigrant. • I would permit a child of mine to marry an immigrant.

Guttman • Administer the Final Scale – Respondents are asked to check items with which they agree – Respondent’s score = sum of the scale score values for checked responses. • Like Thurstone scales, Guttman scales are not often used these days.

Likert Scales • Define the Concept • Generate Potential Items – About 100 statements. – On some, agreement indicates being high on the measured attribute – On others, agreement indicates being low on the measured attribute

Likert – Instead of a dichotomous response scale (agree or disagree), use a multi-point response scale like this:

Likert • Evaluating the Potential Items – Get judges to evaluate each item on a 5 -point scale • • • 1 -- Agreement = very low on attribute 2 – Agreement = low on attribute 3 – Agreement tells you nothing 4 – Agreement = high on attribute 5 – Agreement = very high on attribute – Select items with very high or very low means and little variability among the judges.

Likert • Alternate Method of Item Evaluation – Ask some judges to respond to the items in the way they think someone high in the attribute would respond. – Ask other judges to respond as would one low in the attribute. – Prefer items that best discriminate between these two groups – Also ask judges to identify items that are unclear or confusing.

Likert • Pilot Test the Items – Administer to a sample of persons from the population of interest – Conduct an item analysis – Prefer items which have high item-total correlations – Consider conducting a factor analysis

Likert • Administer the Final Scale – on each item, response which indicates least amount of the attribute scored as 1 – next least amount response scored as 2 – and so on – respondent’s total score = sum of item scores or mean of item scores – dealing with nonresponses on some items – reflecting items (reverse scoring)

Psychometric Analysis • Whenever you use such an instrument, you should conduct basic psychometrics • See “Cronbach’s Alpha and Maximized Lambda 4” • Factor analysis can help determine if the instrument is unidimensional or not. • Factor analysis of the Cultural Values Survey • Factor analysis of Patel’s SBS.

PCA of Cultural Values Survey • • 45 items Reduced to seven orthogonal components Weighted linear combinations of variables Used as outcome variables in Culture x Sex x Age Group factorial ANOVA • Loadings used to decide what the components are

7 Components • Family Solidarity (respect for the family) • Executive Male (men make decisions, women are homemakers) • Conscience (important for family to conform to social and moral standards) • Equality of the Sexes (minimizing sexual stereotyping)

• Temporal Farsightedness (interest in the future and the past) • Independence (desire for material possessions and freedom) • Spousal Employment (each spouse should make decisions about his/her own job)

ANOVA Results • US Students (especially the women) – Sexually egalitarian – Desire independence – Family not important to younger students • Taiwanese Students – Temporally farsighted – Men more sexually egalitarian than women – Women more interested in independence

• Mexican Students – Like the Taiwanese, family is important – Like the US students , not temporally farsighted – Men thought independence more important than did women

Patel’s SBS • 21 items, measures aggression toward homosexual persons • Cronbach’s alpha =. 91 • FA revealed three factors • Avoidance Behaviors (13 items) – Moving away from gay – Staring to communicate disapproval of proximity

• Aggression from a Distance (6 items) – Writing anti-gay graffiti – Damaging gays’ property – Making harassing phone calls • Up-Close Aggression – Physical fighting