Operationalization of Variables and Use of Rating Scales

Operationalization of Variables and Use of Rating Scales 1

Measurement--Operationalization Earlier we said: – Only when we begin to assign numbers to an object do we begin to learn about that object. “I often say that when you can measure what you are speaking about and express it in numbers you know something about it. But, when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be. ” Lord Kelvin (19 th century physicist) 2

Measurement--Operationalization First step in measurement is operationalization of theoretical concepts (i. e. , variables/constructs). Operationalization? – Specification of empirical indicators to represent theoretical concepts – While a theoretical/conceptual definition describes a concept in terms of other concepts, an operational definition describes the operations that need to be performed to measure a concept. (Kerlinger, 1986) – Represents a bridge between theory and data – Operationalization helps convert theoretical propositions to empirical (testable) hypotheses--Example 3

Measurement Theoretical Hypothesis: “Juvenile delinquency is a function of adult supervision” • Theoretical Concepts? u Juvenile u Delinquency u Adult u Supervision 4

Measurement • THEORIES discuss relations between attributes of objects • MEASUREMENT: Involves assigning numbers to attributes of objects according to a set of rules • OBJECTS in social/behavioral sciences are individuals and groups • Some ATTRIBUTES/TRAITS of individual and group are simple, straight forward, and, directly measurable: ü e. g. , height, weight, gender, income, education, etc. ü These represent “observed variables” (or “variables” for short) SO: “VARIABLES” (or observed variables) represent directly observable entities/attributes capable of assuming two or more values. 5

Measurement Other attributes/traits represent ABSTRACT and COMPLEX concepts that CANNOT be measured directly: u u Examples: – Personality Traits (e. g. , aggressiveness, competitiveness, locus of control, etc. ) – Social/psychological characteristics, and mental/cognitive/emotional dispositions (e. g. , attitudes, values, perceptions) These entities represent “Latent Constructs” (or “constructs” for short). “CONSTRUCTS” are attributes which are not directly measurable, but may be measured on the basis of their multiple indicators , where : ü Each indicator is represented by a variable. ü Multiple variables/indicators collectively serve as a proxy for the construct. ü That is, for each subject, scores on multiple indicators/ variables will be combined (often averaged/summed) into a single score. The resulting score is called a “summated scale. ” 6

Construct C Stress D 1 D 2 Physical Tension Emotional or Psychological Tension Dimension D D Nervousness Anxiety Could be measured by D 3 Mental Tension D Fear Could be measured by V 1 Blood pressure V 1 Extent of sleeplessness V 1 Headaches V 2 Pulse Rate V 2 Sweating V 2 Fatigue V 3 Stomach upsets V 3 Confusion

Construct Dimension 1 DRIVEN BY WORK Variable 1 Constantly working Achievement motivation D 2 UNABLE TO RELAX V 3 Persevering despite setbacks V 2 Very reluctant to take time off for anything IMPATIENCE WITH INEFFECTIVENESS Does not like to work with slow or inefficient people Does not have any hobbies SEEKS FEEDBACK V 2 Opts to do a challenging rather than a routine job Opts to take moderate, rather than overwhelming challenges V 4 Swears under one’s breath when even small mistakes occur V 3 D 5 SEEKS MODERATE CHALLENGE V 1 V 2 Thinks of work even at home D 4 D 3 V 1 Asks for feedback on how the job has been done V 2 Is impatient for immediate feedback 8

Measurement—Rating Scales Many constructs (e. g. , personality traits, attitudes, perceptions) are often operationalized through rating scales, whereby: • Subjects select standard responses to one or more verbal statements along some continuum. Ø Rating scales represent: Choice of a category or relative position along some spectrum. (Very Pessimistic=1 2 3 4 5=Very Optimistic) Ø Advantages of rating scales (vs. open comments)? Capture responses to many questions: – quickly, – in very little space, – from a relevant/appropriate perspectives – in a standard, comparable form Ø Rating scale construction is a critical skill. 9

Measurement Important considerations when designing rating scales: u Sometimes (ideally) the response scale is implicit in the item/question. (EXAMPLE: Do you visit your doctor at least once a year? ) u But, sometimes no such common understanding exists between the researcher and the respondent. Example: (e. g. , How long has it been since you last visited your doctor? ) In such a case, the researcher will need to explicitly specify the scale in the question. EXAMPLE? u How many months has it been since …? When that is NOT possible, scale must be incorporated into the response options verbally, numerically, or graphically – Knowledge of scale construction becomes critical. 10

Measurement A REVIEW OF SELECTED TYPES OF RATING SCALES Commonly Used Rating Scale Formats Click the following URL for a compendium of all sorts of useful organizational measures that are provided in the U-Drive: CLIKCK HERE 11

Rating Scales The Multiple choice, Single Response scale What kind of newspaper do you most often read for business news? (Check only one). (1)____Local, morning paper (2)____Local, evening paper (3)____Local, weekly paper (4)____Regional, weekly paper (5)____National, daily paper (6)____National, weekly paper (7)____Other (What kind? _______) Level of Measurement? --Nominal/categorical How many variables represented? --Represents a single variable 12

Rating Scales Multiple Choice—Multiple Response Scale: Please check any type of newspaper you regularly read for business news (Check all that apply). ____Local morning paper ____Local, evening paper ____Local, weekly paper ____National, daily paper ____National, weekly paper ____Other (What kind? ______) Level of Measurement? --Nominal/categorical How many variables represented? --Represents Multiple variables 13

Rating Scales The forced Ranking Scale: Please rank the soft drinks listed below in their order of preference to you. Jot the number 1 next to the one you prefer most, number 2 by your second choice, and so forth. (NOTE: Do NOT assign the same rank to more than one item) 1. ___Pepsi-Cola 2. ___Coca-Cola 3. ___Royal Crown Cola 4. ___Like Cola Level of Measurement? --Ordinal How many variables? --Multiple variables (i. e. , 4) 14

Rating Scales The Paired Comparison Scale: For each pair of soft drinks listed below, please put a check mark by the one you most prefer, if you had to choose between the two. (1) ____Pepsi-Cola (2) ____Coca-Cola (1) ____Royal Crown Cola (2) ____Pepsi-Cola (1)____Royal Crown Cola (2) ____Like Cola (1)____Royal Crown Cola (2) ____Coca-Cola (1) ____Coca-Cola (2) ____Like Cola (1) ____Like Cola (2) ____Pepsi-Cola Number of Comparisons = K (K – 1) / 2 15

Rating Scales The Fixed (Constant) Sum Scale: Of the last 10 times that you ate lunch or dinner at a casual or fast food restaurant, how many times did you have each of the things listed below? (Please be sure to make the total equal 10) ___Hamburgers ___Hot dogs or sausage ___Chicken ___Pizza - Level of Measurement? ___Chinese Food Ordered Metric (Metric) ___Fish or seafood ___Deli sandwiches - Not a summated scale--each ___Hot sandwiches option is a separate variable ___Mexican Food ___Other (What? ________) TOTAL = 10 16

Rating Scales Another Example for Fixed-sum Scales: LIFE VALUES: Divide a total of 100 points among the following aspects of your life to indicate the relative importance of each in your life at present time. Please make sure the numbers add up to 100. __Your leisure (such as hobbies, recreation and contacts with friends) __Your community (such as voluntary organizations, neighborhood activities) __Your work __Your political activities __Your religion/faith __Your wealth (such as the things you own) 17 __Your family

Rating Scales The Ordered Response Scale: Ordinarily, when do you (or someone in your family) first turn on a television set in your home on a weekday? (please check only one. ) (1)__The first thing in the morning (2)__A little while after awakening (3)__Mid-morning (4)__Just before lunch (5)__Right after lunch (6)__Mid-afternoon (7)__Early evening, before dinner (8)__Right after dinner (9)__Late evening (0)__Usually don’t turn it on Level of Measurement? Generates a single metric var. (response options follow an ordered sequence; larger number represent a later time) 18

Rating Scales The comparative Scale--an evaluative scale: Compared to the previous management group, the new management team is…. (circle the number that best reflects your opinion). Very Inferior 1 2 About the same 3 4 Very Superior 5 NOTES—Two issues: a. Regarding whether to assign verbal anchors (label) the scale’s all or only some of the response options (e. g. , only the two ends of the scale): 1. 2. Both approaches are commonly used Assigning explicit labels (verbal anchors) to more options a. increases the likelihood that respondents will interpret each option in the same manner, and b. Increases test-retest reliability of the scale 3. Labeling only a few of the options is especially useful when more variance is desirable, however, it is difficult to come up with meaningful yet distinctly different labels for all scale options. 4. Some suggest that when working with truly bipolar scales (like the above) using -2, -1, 0, +1, +2 are preferable because they reinforce the bipolar nature of the requested responses. 19

Rating Scales The Likert Scale: Please pick a number from the scale to show much you agree or disagree with each statement and jot it in the space to the right of the item 1=Strongly Disagree 2=Disagree 3=Neutral 4=Agree 5=Strongly Agree A man should never cry in public (T—if agreed) _______ Swearing & obscenity is more repulsive in the speech of a woman than a man (T) _______ Women should receive equal pay for equal work (P/M) _______ A man should NOT resent a woman supervising his work (P/M) _______ There is nothing wrong with a man becoming a stay at home dad (P/M _______ Women should pay their share when dating (P/M) _______ The husband should handle family legal decisions (T) _______ Women should never put career before family (T) _______ The initiative in dating should come from men (T) ______ Economic and social freedom is worth far more to a woman the acceptance of the ideal of femininity which has been set by men (P/M)_______ What does it Measure? -- Note the direction of response options. -- Number of scale points (see next slide) Sex Role Attitudes--Traditional vs. Progressive/Modern A summated metric Scale (a single construct/variable)– Min scale value = 1 x 10=10, Max scale value = 5 x 10=50 20

Rating Scales Appropriate number of points in the scale (5, 7, etc. ) depends on u : Sophistication of the respondents – Ability to discriminate scale anchors meaningfully u Consider respondents’ age, education, intelligence, mental capacity, etc. u Complexity of the issue/object – For views, decisions, intentions, etc. about complex, multi-dimensional issues that require significant thought and soul searching, more points/anchors needed u (e. g. , attitudes toward welfare benefits, criteria for voting for a presidential candidate, views on capital punishment, attitude toward unions, views on deregulation of a particular industry). – For views, decisions, intentions, etc. about issues that are simple and straight forward, fewer points/anchors may work OK u (e. g. , belief in god, attitudes toward polygamy, views on whether one should attend college) – For perceptions, intentions, etc. about a simple and inexpensive products that require very little effort/thought to purchase, or are habitually purchased, fewer scale points/anchors may work OK u (e. g. , snacks, soft drinks, soaps, toothpaste or soap) – For products that are complex, multifaceted, and expensive, more scale points/anchors are required u (e. g. , a house, car, TV set, or computer) (continued) 21

Rating Scales Number of points/anchors in the scale (3, 5, 7, etc. ) : u Salience of the issue/object – For views, decisions, etc. regarding serious issues, or issues and objects, that have a high impact on, or play an important role in, one’s life, welfare, career, peace of mind, health, etc. , more scale points/anchors are needed u u (e. g. , intention to leave one’s employer, what type of college to attend, what type of person to marry, or support for going to war. The number of items in the scale – If few items are used in a multi-item summated scale (fewer than 4 or 5 items), then more scale points/anchors can make up for that deficiency; otherwise, fewer anchors work OK. The Bottom Line: To obtain accurate results, meaningful variance is needed! So, all else considered, more scale points are preferable (unless other considerations instruct us us against it). – More scale points generate more variance, which helps prevent range restriction and, thus, produce more accurate results and give greater flexibility in choice of statistical techniques. 22

Rating Scales The Verbal Frequency Scale--useful for capturing one’s behavioral tendency/general policy toward an issue. Please pick a number from the scale to shows how often you do each of the things listed below and jot in the space to the right of the item. 1=Always 2=Often 3=Sometimes 4=Seldom 5=Never Actually vote during a strictly local election ___ Actually vote during a state and national election ___ Vote along strict party lines ** ___ Contribute money to a local political campaign ___ Contribute money to a national political campaign ___ Volunteer to work on a local political campaign ___ Volunteer to work on a state political campaign ___ Volunteer to work on a national political campaign ___ Not all summated scales have to have reverse items. What does it Measure? Political Activism (A summated Scale --a single variable) Note the direction/values of response options 23

Rating Scales Horizontal, Numeric Scale of Importance (An Evaluative Scale): When choosing a place to buy sports equipment, how important is: (mean values shown Not from highest to lowest) Important 1 Quality of merchandise Selection of brands Variety of products Service after purchase Store guarantees Price of the goods Courtesy of service Help and advice offered Speed of service Store hours Location of store Very Important 2 3 4 5 - Non-Summated Version (multiple variables) - Very useful for measuring decision making criteria 24

Rating Scales Horizontal, Numerical Importance Scale: Please indicate below how important to you is each of the public issues listed below? Very Unimportant = 1 The The 2 3 4 5 = Very Important protection of endangered species of animals improvement of the quality of the air discovery of additional petroleum reserves (R) development of ‘renewable” sources of energy reduction or elimination of water pollution development of additional nuclear power (R) rapid industrial growth of the nation (R) protection of overall ecological balance -Environmentalism -Summated Version (single variable) Note: Not all scales can be bipolar and balanced with a neutral mid-point, e. g. , funniness)! What Does it Measure? 25

Rating Scales The Semantic Differential Scale--very useful for measuring mental images and perceptual profiles: Please circle a number on each line below to show your opinion of the pizza served here. Hot Bland Expensive Moist Soggy Good Unattractive Fresh Small Natural 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5 6 6 6 6 6 7 7 7 7 7 How many variables? Cold Spicy Inexpensive Dry Crisp Bad Attractive Stale Large Artificial 26

Rating Scales Semantic Differential Image Profile: 1 Hot Bland Expensive Moist Soggy Good Unattractive Fresh Small Natural 2 * 3 4 5 6 * * * * 7 Cold * Spicy Inexpensive Dry Crisp Bad Attractive Stale Large Artificial Mean Ratings are shown with “*” 27

Rating Scales Positioning Profile Vis-a-Vis the Ideal Pizza: 1 Hot Bland Expensive Moist Soggy Good Unattractive Fresh Small Natural * Our Pizza 2 * * 3 4 5 6 7 Cold * Spicy Inexpensive * ** Dry * * ** * * Crisp Bad Attractive Stale Large Artificial * The Ideal Pizza 28

Rating Scales An alternative application of the Semantic Differential Scale — Descriptive-Normative Gap Analysis Please circle a number on each line below to show your opinion of the pizza served here, first in terms of the way it currently is, an then in terms of the way it should be. The way it is: Hot = 1 2 3 4 5 6 7 = Cold The it should be: Hot = 1 2 3 4 5 6 7 = Cold The way it is: Bland = 1 2 3 The way it should be: Bland = 1 4 5 6 7 = Spicy 2 3 4 5 6 7 = Spicy The way it is: Expensive = 1 2 3 The way it should be: Expensive = 1 4 5 6 7 = Inexpensive 2 3 4 5 6 7 = Inexpensive The way it is: Moist = 1 2 The way it should be: Moist = 1 4 2 3 5 3 6 7 = Dry 4 5 6 7 = Dry 29

Rating Scales The Adjective Check List: Please put a check mark in the space in front of any word or phrase that describes your job. __Easy __Technical __Boring __Interesting __Low-paying __Strenuous __Routine __Dead-end __Changing __Important __Demanding __Temporary __Safe __Exhausting __Difficult __Rewarding __Secure __Slow-paced __Enjoyable __Rigid __Pleasant Multiple __Satisfying nominal __Degrading variables __Risky 30

Rating Scales The Stapel Scale: Please pick a number from the scale to show well each word or phrase below describes your job and jot it in the space in front of each item. SCALE: Not at all = 1 __Easy __Technical __Boring __Interesting __Low-paying __Strenuous __Routine __Dead-end __Changing __Important __Demanding __Temporary 2 3 4 5 6 7 = Perfectly __Safe __Exhausting __Difficult __Rewarding __Secure __Slow-paced __Enjoyable __Rigid __Pleasant __Satisfying __Degrading __Risky Multiple metric variables 31

Rating Scales Guttman Scale (a series of items tapping progressively higher levels of the attribute being measured): Please check the option that best reflects how you feel about your job: ___I like may job a great deal ___I am somewhat satisfied with my job ___I don’t strongly like or dislike my job ___I am somewhat dissatisfied with my job ___I don’t like my job at all A single metric variable 32

Graphic Rating Scale, Stressing Visual Communications 3 VERY GOOD 2 1 VERY POOR. . 33

u Measurement Among the important characteristics that we use rating scales to measure are people’s social-psychological properties (e. g. , attitudes, perceptions, values) – Often illusive and multidimensional--embrace more than a simple reaction to a simple statement (constructs) – Often measured through responses to a series of verbal statements – Among these, measurement of attitudes is very common, but particularly challenging. For a sample of organizational measures (rating scales) see: - U-Drive: Measurement Instruments - http: //www. dmidi. net/All+Scales – http: //division. aomonline. org/rm/ – http: //www. psywww. com/resource/bytopic/testing. html – http: //www. isworld. org/surveyinstruments. htm – http: //www. personalityresearch. org/tests. html 34

Measurement ATTITUDES: Fairly enduring psychological predispositions focused on some target (e. g. , person, event, idea, issue) Note that attitudes have 3 components/ Dimensions: – Affective/Emotional (How do you feel about it? liking/disliking) – Cognitive/Beliefs/knowledge (What do you think/know about it? ) – Behavioral/action (What are you likely to do about it? ) 35

Measurement Example: Job Attitudes u u u Affective/Emotional Components? – I love my job. Cognitive/Beliefs/Knowledge Component? – My job allows me to use a wide range of my abilities. Behavioral/Action Components? – If I had the choice, I would quit my job today. 36

MEASUREMENT OF ATTITUDES: Guidelines for constructing scale items to measure ATTITUDES: – Use multiple items (i. e. , statements or questions) to measure the attitude--use a summated scale – Each item in the scale should deal with only one point/issue. – Likert scales (i. e. , Agreement. Disagreement scales) are very helpful in construction of these scale items. 37

MEASUREMENT OF ATTITUDES Guidelines for constructing scale items to measure ATTITUDES: – Attitudes have a target object; keep focus of each and all items on the target object u That is, keep the focus on the entity whose favorableness/unfavorableness you are assessing) Example: Which of the items is problematic? u I love my job u I will quit my job the first chance I get. u I am very competent in my job u In this economy, good Jobs like mine are hard to come by. 38

MEASUREMENT OF ATTITUDES: Guidelines for constructing scale items to measure ATTITUDES: – Prepare a list of relevant dimensions (+ & -) of the target object, and use it as a basis for constructing the scale items (e. g. , attitudes toward welfare system--what are some of the issues that make people favor/oppose it? The + or -_Features? ): • Cost: Costly to taxpayers, drain on federal budget • Protection Against Poverty/Safety Net: –Provides safety net for the needy –Helps reduce poverty –Takes care of those who can’t take care of themselves • Social-Economic Values: –Undermines the competitive and free economic system –Source of social ills (promotes laziness) –Undermines sense of individual responsibility –Reduces motivation to search for a job –Creates a cycle of dependency –Reflects society’s compassion toward the needy • Fairness: –Helps distribute wealth more justly –Creates a fairer society –Unfair to those who work hard and pay their taxes 39

MEASUREMENT OF ATTITUDES: Guidelines for constructing scale items to measure ATTITUDES: – Use only items that are clearly indicative of a favorable or unfavorable viewpoint/disposition toward the target object. – When applicable, make some statements favorable and others unfavorable (Reverse Items). – Consider asking about reactions to hypothetical situations when necessary. – Consider including items capturing the feeling, belief, and action components of attitudes. – Select an appropriate range for response options (e. g. , a 3 -point vs. 5 -point vs. 7 -point scale). 40

MEASUREMENT OF ATTITUDES: Guidelines for constructing scale items to measure ATTITUDES: – Consider whether or not to use a “Neutral” midpoint in the scale. Choose your word carefully. Note that “Neutral” is not equivalent to “Don’t Know. ” u Neutral, Unsure, or Undecided could mean Indifferent, Ambivalent, Equally Favorable and Unfavorable, Honest Lack of Opinion, or Not Have Made up One’s Mind. – Use “Don’t Know” sparingly as a choice. u “Don’t Know” implies lack of sufficient information to express an opinion. u If used, “Don’t Know, ” like “Not Applicable ”, has to be treated as missing data. 41

EVALUATION OF PRODUCTS, SERVICES, PROGRAMS, OBJECTS, ETC. Guidelines for Developing EVALUATIVE Instruments: – Develop a comprehensive list of possible characteristics/attributes to be evaluated. – Select the salient attributes to be evaluated. – For services and programs, consider including both summative (service outcomes) and formative (process of service delivery) evaluation items. 42

EVALUATION OF PRODUCTS, SERVICES, PROGRAMS, OBJECTS, ETC. Guidelines for Developing EVALUATIVE Instruments: – Consider also asking respondents to rank the three/five most important attributes/criteria to them if they were to select from among alternative product brands/service providers. – Consider also asking them to rank the three most appealing/satisfactory attribute of the product/service/program being evaluated. – Include an item soliciting an overall evaluation. 43

EVALUATION OF PRODUCTS, SERVICES, PROGRAMS, OBJECTS, ETC. Guidelines for Developing EVALUATIVE Instruments: – Consider asking if they would recommend the product, service, or program to others. – Consider asking how they would characterize the ideal product/service in terms of salient attributes. – Consider including “Don’t Know” as a response choice. – Consider asking some open ended questions. 44

QUESTIONS OR COMMENTS ? 45