Chapter 4 Measurement Issues in Research Sum 2014

Chapter 4 Measurement Issues in Research Sum 2014 Dawn M. Mc. Bride The Process of Research in Psychology Second Edition

Steps in the Research Process 2 Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Variables: Operational Definition l Way to measure or manipulate a concept (construct) in a research study § § Example: intelligence - numerical measure Ø Q test score; GPA --Ø problem-solving accuracy Example: treatments - Groups - Nominal Ø Ø 3 Manual for Cognitive Behavior and for Interpersonal treatment Violent videogames and non violent-videogames Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Scales of Measurement 4 Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Scales of Measurement l Nominal: non-ordered category responses § Example: Groups formed based on : #1 How do you feel today? (Circle one) happy sad anxious excited #2 Condition: Type of Condition Violent or non-violent videogame 5 Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Scales of Measurement l Ordinal: ordered category responses; no equal space between categories; not numerical l Example: How do you feel today? (Circle one) 6

Scales of Measurement 7 l Interval: equally spaced, numerical responses, values are not ratio relations l Example: Rate your pain (Likert Scale) A rating of 10 twice the pain of a 5

Scales of Measurement l Ratio: equally spaced, numerical responses, values are ratio relations (difference between 4 and 8 = difference between 6 and 12) l Examples: Time Distance § Length Height § Accuracy Weight § 8 Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Validity of Scales l Construct § validity: scale measures the construct it was designed to measure (e. g. behavior, attitude, belief, value, feeling, state) Ø Face validity ®Do the scale (items, drawings) reflect the construct of interest ? 9 Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Construct Validity l Concurrent Validity § l Discriminant Validity § 10 A new measure of depression correlates highly with the Beck Depression Inventory A measure of parenting does not correlate highly with a measure of Social Desirability Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Construct Validity l Predictive Validity § l In sum § 11 SAT score (taken now) predicts College GPA (some time later) Validity of numerical scales is determined by examining the correlation of the scores with scores in other numerical scales Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Name of the Scale: Type of Scale: Nominal, Ordinal, Interval, Ratio ? 12 Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Name of the Scale: Distress Disclosure Index Type of Scale: Nominal, Ordinal, Interval, Ratio Likert 13

Reliability: Consistency of Measurement 14

Reliability: Measures 15 l Test-retest reliability l Internal reliability l Inter-rater reliability - Observations

Reliability: Measures l Test-Retest § l Internal § 16 Responses do not change across time Ø With no intervention, participants respond to scale similarly across two points in time Responses are consistent across items in the survey Ø Similar score with different forms Ø Similar scores in two half's of the test

Reliability: Measures l Inter-rater reliability - Observations § 17 Several observers rate behaviors in a similar way

Independent Variables: Experimental Research 18

Manipulation Independent Variables: Type and Amount l Type § 19 Compares different versions of the “something” that is manipulated Ø drugs or therapies Ø versions of an ad or product Ø different types of instructions § All groups include the factor being compared § Each group receives a different type or version of that factor (between groups designs) Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

IV: Type of Video Game DV: 20

Manipulation: Independent Variables Type and Amount l Amount § 21 manipulates amount (quantity, intensity, length of time) of a factor in each group Ø Quantity of drug that is optimal for relieving symptoms Ø Off time between the study and test of material memory research Ø Intensity of noise Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

IV: Amount of Sleep DV: Reaction Time 22

Designs: Independent Variables l Bivalent design: § l Multivalent design: § 23 an independent variable with two levels or groups: an independent variable that includes three or more levels or groups Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

IV: Type: consequence observed in the model DV: Extent child imitated model’s behavior 24

Quasi- Independent Variables: 25

Quasi-Independent Subject Variables in Experimental Designs: 26 l Characteristic of the participant s that is used to group participants in the analysis of the data– e. g. Gender – Age l Effect of IV on DV is observed separagtelty for the various groups l Caution in reaching conclusions, since randomization is not possible.

IV(s): Consequence observed & Gender (ANOVA) DV: Extent child imitated model’s behavior 27

Quasi-Experiment Design l Participants are assigned to different treatments based on already existing groups § 28 5 th grade Homerooms A and B and C- Bandura study l Researcher is unable to manipulate participants assignment to treatments, therefore there is no Randomization at all l Groups are compared in the same way as in experimental designs Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

HRA- Reward Model; HRB Punish Model ; HRC- No consequence Model (Bias) 29

Bias: Experimental Designs l Bias can lower internal validity of experiment l A study with good internal validity provides a good test of l 30 § a causal hypothesis § controls for confounding factors that could affect results, but are not of interest in the study. In other words, a study with good internal validity will remove alternative explanations for the findings (besides the independent variable). Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Sources of Bias in Research l Treatment Group differences (randomization does not always work) l Experimenter bias (e. g. allegiance effect) § l Testing effects (order, practice, fatigue) § 31 Single and double blind studies (deception) Survey Research: vary order of measures across participants Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Sources of Bias in Experiments l Regression towards the mean § § Extreme scores are not likely to recur Difficult Bias to Remove Ø Ø 32 Random assignment Use several repetitions of the test that measures DV for each condition Taking outliers out of the data Large number of participants (dilutes extreme scores) Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

External Validity: Experiment l External validity is the degree to which a study: § § l measures realistic behaviors and results can be generalized to every day life If participants behave in a research study the way they would in their everyday lives, then the study has good external validity. 33 Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

External Validity l External 34 Validity can be affected by: § Hawthorne effect § Demand Characteristics Ø Participants comply with or contradict what they perceive is expected from them Ø Deception – does not always work Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

ZERO-SUM Game Internal Validity 35 External Validity

Between-Within Subjects Designs l Between Subject § l 36 Different participants receive different treatments Within Subjects § All participants receive the same treatments some times in different order Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Smith & Moynan (2008) l Within Subject Manipulation § IV #1 = type of word list – Categories: neutral, emotional/memorable, and emotional/not memorable Between Subject § 37 IV #2 = filler activity (control and “forget” tasks). . Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.

Smith & Monyan (2008) l Within § l #1 type of list (neutral, emotional and memorable, and emotional but not memorable); Between Subjects § 38 Subjects #2 filler activity (control and “forget” tasks). Dawn M. Mc. Bride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.