DEFINING MEASURING VARIABLES Research Basics Overview 2 Defining

DEFINING & MEASURING VARIABLES Research Basics

Overview 2 Defining Variables Types of Measures Scales (i. e. , levels) of Measurement Reliability Validity

3 Defining Variables

Variables & Definitions 4 Variable Something that varies and can be measured Hypothetical constructs Attributes/mechanisms that help explain/predict behavior May not be directly observed But may be associated with something that can be measured

Variables & Definitions 5 Conceptual Definitions: What you’d read in a textbook Aggression? ? “A response intended to inflict pain or discomfort” (Averill, 1982, p. 30) “Any form of behavior directed toward the goal of harming or injuring another living being who is motivated to avoid such treatment” (R. A. Baron, 1977, p. 7) “The use of threat or force, in territory not clearly one’s own, without clear evidence that a majority of the emotionally involved people in that territory want such intervention” (R. K. White, 1984, p. 14)

Defining Variables 6 Operational definition “Operationalization” Measurement or measurement procedure How would you measure aggression?

Defining Variables 7 “Exercise leads to weight loss” Exercise Thumb twiddling 1 minute per week 10 pushups each day ½ hour aerobic activity, 5 days a week where heart rate is greater than 140 bpm Weight loss Pounds Waist Size Body fat content Clothes size

Multiple Outcomes 8 Converging evidence is more convincing Exercise leads to lower scale-weight, smaller waist size, and reduced body fat More measured outcomes increases the chances we’ve tapped into all relevant parts of our construct

Let’s practice 9 Creativity Forgiveness Love Extraversion

“Southern Culture of Honor” 10 “Affronts are met w/ violent retribution” Hypotheses Perceive insults as damaging Emotional/physiologically upset Prepared to respond aggressively after insult (cognitive, physiologically, & behaviorally) (Cohen, Nisbett, Bowdle, & Schwarz, 1996)

“Southern Culture of Honor” 11 Hostility assessed 2 observers measured emotional reactions Face rating task Guess Word emotion being expressed in pictures completion task __ight, gu__ (fight or gun) OR (light or gum) Scenario completion task Cohen, Nisbett, Bowdle, & Schwarz, 1996)

Scenario Completion Task 12 It had only been about twenty minutes since they had arrived at the party when Jill pulled Steve aside, obviously bothered about something. “What’s wrong? ” asked Steve. “It’s Larry, I mean, he knows that you and I are engaged, but he’s already made two passes at me tonight. ” Jill walked back into the crowd, and Steve decided to keep his eye on Larry. Sure enough, within five minutes Larry was reaching over and trying to kiss Jill. Cohen, Nisbett, Bowdle, & Schwarz, 1996)

Example: “Southern Culture of Honor” 13 Hostility assessed Before & after cortisol levels Hormone Before & after testosterone levels Hormone Desire associated w/ stress, anxiety, & arousal associated w/ aggression & dominance to demonstrate toughness Public opportunity to take shock in “electric shock stress test” The “chicken game” Firmness of handshake Cohen, Nisbett, Bowdle, & Schwarz, 1996)

What’s the point? 14 Many ways to operationalize vars Didn’t use them all in one study though Reliable measures? Valid measures?

An operational definition should be… 15 Specific Reliable Valid

Approaches Quantitative Expressed numerically Statistical tests available Qualitative In-depth Grounded theory – use information to develop interpretation or theory Identify ideas, then write theoretical narrative (broad themes) Difficulties w/ analyses

How to measure? 17 Which measure provides the best indication of the underlying construct? Self-reports Behavioral, Cognitive, Affective Tests Personality, Behavioral Concern Physical Achievement, Aptitude measures about reactivity measures Each has advantages & disadvantages

Selecting Measures Scales of Measurement Nominal Scales Classification Qualitative differences Ordinal Scales Classify & rank

Selecting Measures Scales of Measurement Interval Scales Classify, rank, & categories the same size Arbitrary zero Ratio Scales Classify, rank, categories the same size, & anchored by zero point

20 Reliability

Choosing Your Measures Reliability Stability or consistency of measure Observed score = True Score + Measurement Error

Types of Error 22 1. Observer Error 2. Participant Changes Stable attributes (E. g. , suspicion, intelligence) Transient states Something about person that changes regularly 3. Environmental Changes 4. Test characteristics

Assessing Reliability Test-retest reliability Participants have changed Respond in similar way Practice Effects Parallel-forms (or alternative-forms) reliability Tests at time 1 and time 2: are they equivalent?

Assessing Reliability Internal consistency Can be measured with correlation coefficient Split-half reliability - Split items on tests into equivalent halves

Assessing Reliability Judgments/Ratings by Multiple Observers Interrater (interjudge) reliability Extent of agreement between raters Can be measured w/ correlation or percent agreement

26 Validity

Choosing Your Measures Validity Is 1. the measuring what it is meant to? Face Validity Does the measure appear to measure what it is intended to measure? 2 + 2 = ____ 6 + 10 = ____ 7 + 3 = ____

What do you think this test measures? 1 = Strongly disagree 2 = Disagree 3 = Not sure 4 = Agree 5 = Strongly agree I enjoy taking risks. I prefer things that involve change and variety even if they make me feel uneasy at first. I would enjoy fast driving. I would do almost anything on a dare.

Types of Validity 1. Face Validity When might we NOT want a face valid measure? 2. If we think it might influence participants’ answers Content Validity How well do the items represent the entire universe of items?

Types of Validity Criterion validity 3. How well is the measure related to other variables that reflect the same construct? Concurrent validity Do scores correlate with other relevant current outcome/behavior? IQ test should correlate with current school performance Occupational aptitude test should correlate with current work performance Predictive validity Does it correlate with future outcomes? According to theory

Types of Validity 4. Discriminant (Divergent) Validity Looking for evidence that the measure is distinct from other behaviors/traits (i. e. , not correlated)

Types of Validity Face Content Criterion Concurrent Predictive Discriminant (Divergent)

Relationship Between Reliability & Validity Can be reliable, without being valid. But can’t be valid, without being reliable.

Re-cap 34 Defining Variables Types of Measures Scales (i. e. , levels) of Measurement Reliability Validity

35 1. 2. 3. Mini-Review What is the difference between reliability and validity? Provide an example on a measure that would have high content validity. Imagine I wanted to measure helping behavior. How might I operationalize this variable?