Data and the Nature of Measurement Graziano and

Research Variables n Variable: Any characteristic that can take on more than one value

Measuring Variables n Measurement: Assigning numbers to indicate the level of a variable –

Scales of Measurement n n Based on how closely the measurement scale matches the

Nominal Scales n Naming scale – Each number reflects a category – Examples: diagnostic

Ordinal Scales n Scale indicating rank order – Reflects the order, but not the

Interval Scales n Scale with equal intervals – The scale indicates amount, but with

Ratio Scales n Scale that fits the number system well – Includes equal intervals

Psychological Tests n Most psychological test fall somewhere between an ordinal and a ratio

Measurement Error Decreases the accuracy of measurement n Possible sources of measurement error n

Operational Definitions Specific procedures for measuring and/or manipulating a variable n Every variable should

Reliability n n The consistency of measurement Consistency can be conceptualized in different ways

Types of Reliability n Interrater reliability: degree of agreement between two independent raters n

Perfect Reliability n n Reliability is a measure of consistency. Perfect reliability means that

Good Reliability n n Reliability deteriorates when the consistency is lost. Here the rank

Fair Reliability n As reliability deteriorates further, the rank orderings from the two testings

Poor Reliability n When you reach the point where the rank orderings have no

Effective Range n n The range in which a measure gives an accurate indication

Scale Attenuation Effects n Results from a restriction of the range of the measure

Illustrating These Effects n Situation n – Assume that we are measuring the weight

Weights for 9 Men Copyright © Allyn & Bacon (2007)

Floor Effect n n Range on the bottom of the scale is insufficient to

Floor Effect Copyright © Allyn & Bacon (2007)

Ceiling Effect n n The range on the top of the scale is insufficient

Ceiling Effect Copyright © Allyn & Bacon (2007)

Source of These Effects n Result of psychological scale with an insufficient range to

Validity n n A scale is valid if it measures what it is supposed

Measuring Validity n Validity is usually measured with a correlation – The correlation is

Perfect Validity n n Validity is the degree to which one measure predicts another.

Good Validity n If the rank orderings from the predictor and criterion measures are

Fair Validity n But the less the rank orderings from the predictor and criterion

Poor Validity n When you reach the point that the rank orderings of predictor

Objective Measurement n The hallmark of science – Objective measures produce the same result

Summary n n n Measuring variables is central to research Several scales of measurement

Slides: 34

Download presentation

Data and the Nature of Measurement Graziano and Raulin Research Methods: Chapter 4 This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program. Copyright © Allyn & Bacon (2007)

Research Variables n Variable: Any characteristic that can take on more than one value – Examples: speed, level of hostility, accuracy of feedback, reaction time n Research is the study of the relationship among variables – Therefore, there must be at least two variables in a research study (or there is no relationship to study) Copyright © Allyn & Bacon (2007)

Measuring Variables n Measurement: Assigning numbers to indicate the level of a variable – Sometimes the number assignment is easy to understand (e. g. , time measured in seconds) – Sometimes it is more arbitrary (e. g. , 1 for male and 2 for female) Copyright © Allyn & Bacon (2007)

Scales of Measurement n n Based on how closely the measurement scale matches the real number system Scales of Measurement (Stevens, 1946, 1957) – – Nominal Ordinal Interval Ratio Copyright © Allyn & Bacon (2007)

Nominal Scales n Naming scale – Each number reflects a category – Examples: diagnostic categories, political affiliations Produces nominal or categorical data n Mathematical properties n – Identity Copyright © Allyn & Bacon (2007)

Ordinal Scales n Scale indicating rank order – Reflects the order, but not the amount Example: order of finish in a race, class rankings Produces ordered data n Mathematical properties n – Identity – Magnitude Copyright © Allyn & Bacon (2007)

Interval Scales n Scale with equal intervals – The scale indicates amount, but with no zero point – Examples: temperature on the Celsius scale, most psychological tests n n Produces score data Mathematical properties – Identity – Magnitude – Equal intervals Copyright © Allyn & Bacon (2007)

Ratio Scales n Scale that fits the number system well – Includes equal intervals and a true zero – Examples: time, distance, frequency n n Produces score data Mathematical properties – – Identity Magnitude Equal intervals True zero Copyright © Allyn & Bacon (2007)

Psychological Tests n Most psychological test fall somewhere between an ordinal and a ratio scale – Ordinal in that the distance between scores may not be equal – Ratio in that one can view the test score as the number of correct items n Norm is to assume such tests represent an interval scale Copyright © Allyn & Bacon (2007)

Measurement Error Decreases the accuracy of measurement n Possible sources of measurement error n – Response set biases – Inconsistent measurement procedures – Sloppy procedures – Unreliable measures Copyright © Allyn & Bacon (2007)

Operational Definitions Specific procedures for measuring and/or manipulating a variable n Every variable should be operationally defined n The more careful and complete the operational definition, the more precise the measurement of the variable n Copyright © Allyn & Bacon (2007)

Reliability n n The consistency of measurement Consistency can be conceptualized in different ways – Therefore, there are different types of reliability n Usually measured with a correlation – Covered in Chapter 5 – Sensitive to the consistency of rank orderings of participants Copyright © Allyn & Bacon (2007)

Types of Reliability n Interrater reliability: degree of agreement between two independent raters n Test-retest reliability: degree of consistency over time n Internal consistency reliability: degree to which the items of a measure all measure the same thing Copyright © Allyn & Bacon (2007)

Effective Range n n The range in which a measure gives an accurate indication of the level of the variable Critical to select measures with an effective range appropriate for – – – Your sample Your study Example: Using a calculus test to measure math skills in second graders would NOT work Copyright © Allyn & Bacon (2007)

Scale Attenuation Effects n Results from a restriction of the range of the measure – Therefore, people above or below the effective range are not measured accurately n Two types of scale attenuation effects – Floor effects – Ceiling effects Copyright © Allyn & Bacon (2007)

Illustrating These Effects n Situation n – Assume that we are measuring the weight of 9 men using a standard bathroom scale Floor Effect – The needle is stuck so that it never reads below 175 n Ceiling Effect – The needle is stuck so that it never reads above 200 Copyright © Allyn & Bacon (2007)

Floor Effect n n Range on the bottom of the scale is insufficient to measure people who score near the bottom If the scale needle was stuck so that it never read below 175 pounds – There would be a floor at 175 n Error due to this floor effect is shown in dark blue in the next slide – Weights recorded for these people are too high Copyright © Allyn & Bacon (2007)

Ceiling Effect n n The range on the top of the scale is insufficient to measure people who score near the top If the scale could not read above 200 pounds – There would be a ceiling at 200 (everyone above 200 would be reported as weighing 200) n Error due to this ceiling effect is shown in dark blue in the next slide Copyright © Allyn & Bacon (2007)

Source of These Effects n Result of psychological scale with an insufficient range to measure the range of performance in the sample – Example: a measure of memory that is either too easy or too hard Copyright © Allyn & Bacon (2007)

Validity n n A scale is valid if it measures what it is supposed to measure Validity also refers to how well a scale predicts other variables – Example: An IQ test is likely to be a valid predictor of grades in school. – When used this way, the scale is called the predictor measure and the measure predicted is called the criterion Copyright © Allyn & Bacon (2007)

Measuring Validity n Validity is usually measured with a correlation – The correlation is between n n The measure and A SPECIFIED criterion measure – Always list the criterion when reporting the level of validity n For example, the IQ test is a valid predictor of school grades, but not a valid predictor of athletic ability. Copyright © Allyn & Bacon (2007)

Perfect Validity n n Validity is the degree to which one measure predicts another. With perfect validity, the rank orderings on the predictor and criterion measures are identical. Copyright © Allyn & Bacon (2007)

Objective Measurement n The hallmark of science – Objective measures produce the same result no matter who does the measuring – Therefore, scientific principles will apply no matter who tests these principles n Objective measures reduce biases that could distort results (see Chapters 8 and 9) Copyright © Allyn & Bacon (2007)

Summary n n n Measuring variables is central to research Several scales of measurement exist Reduce measurement error with carefully developed operational definitions Enhance reliability and validity of your measures The goal is to produce objective, accurate measures of your variables Copyright © Allyn & Bacon (2007)