Chapter 4 Conceptualization and Measurement Learning Objectives 1

Chapter 4 Conceptualization and Measurement

Learning Objectives 1. Define and distinguish conceptualization and operationalization. 2. List four different means of operationalizing concepts. 3. Give two examples of constant and two of variable phenomena. 4. Identify the different forms of single questions and response choices. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 2

Learning Objectives 5. Give examples of the four levels of measurement. 6. Compare the advantages and disadvantages of the three approaches to testing the validity of measures. 7. Define the five methods of evaluating measurement reliability. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 3

What Do We Have in Mind? Conceptualization • Concepts can be measured in many ways. • Conceptualization – Involves creating concepts – Measurements must be consistent with definitions. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 4

What Do We Have in Mind? Variables and Constants • Constant • Four steps to choosing variables: – Examine relevant theories – Review the research literature – Consider constraints of setting – Look ahead to data analysis Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 5

How Will We Know When We’ve Found It? • Operationalization – Should be consistent with research question. – Consider time and resource limitations. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 6

How Will We Know When We’ve Found It? Using Available Data • To determine usefulness, consider. . . – Appropriateness for concepts of interest – Other potential measures – Whether they can measure our concepts at all. • Cannot assume available data are accurate. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 7

How Will We Know When We’ve Found It? Content Analysis • Words or features of units are coded to measure the variables in the research question. • Reliability assessed by comparing different coders’ results for the same variables. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 8

How Will We Know When We’ve Found It? Constructing Questions • Single Questions – Closed-ended (fixed-choice) question • Response choices should be mutually exclusive and exhaustive. – Open-ended question • Mainly used when studying small groups • Preferable when responses cannot be predicted Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 9

How Will We Know When We’ve Found It? Constructing Questions • Indexes and Scales – Must demonstrate consistency of responses – Many valid indexes already exist – Indexes often presented in matrix format – Indexes usually calculated by averaging responses with each question counting equally. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 10

How Will We Know When We’ve Found It? Making Observations • Observations can be. . . – Primary form of measurement – Supplementary measures to questions • Direct observations are often method of choice for measuring behavior in natural settings – Can allow measurement in context, but observations can also be distorted Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 11

How Will We Know When We’ve Found It? Combining Measurement Operations • Triangulation – Can strengthen measurement – Divergence could indicate measures operationalize different concepts. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 12

How Much Information Do We Really Have? Nominal Level of Measurement • Examples: State, religious affiliation, and so on. • Categories must be mutually exclusive and exhaustive. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 13

How Much Information Do We Really Have? Ordinal Level of Measurement • Focused on greater than and less than • As with nominal variables, different values must be mutually exclusive and exhaustive. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 14

How Much Information Do We Really Have? Interval Level of Measurement • No absolute zero point • Interval-level measures can be created by combining responses to ordinal measurements into an index. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 15

How Much Information Do We Really Have? Ratio Level of Measurement • Values are fixed. • There is an absolute zero point. • Ratio numbers can be added, subtracted, multiplied, and divided. • Can be used in more complex data analyses. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 16

How Much Information Do We Really Have? Comparison of Levels of Measurement • Researchers choose levels of measurement when operationalizing variables. • Usually a good idea to measure at highest possible level. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 17

Did We Measure What We Wanted to Measure? Measurement Validity • Face Validity – A measure has face validity if it obviously pertains to the meaning of the concept – Is not scientifically convincing. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 18

Did We Measure What We Wanted to Measure? Measurement Validity • Criterion Validity – Relies on an already validated measure of a phenomenon. – For many concepts, it is hard to find a wellestablished criterion. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 19

Did We Measure What We Wanted to Measure? Measurement Validity • Construct Validity – Must be sure that two measures are truly independently produced. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 20

Did We Measure What We Wanted to Measure? Reliability • A prerequisite for measurement validity. • Different kinds of reliability: – Test–retest – Interitem – Alternate-forms – Split-halves – Interobserver Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 21

Did We Measure What We Wanted to Measure? Can We Achieve Both Reliability and Validity? • Reliability and validity of measures must be tested after the fact. – If they are not, little can be done to save the study. • Usually, it is best to use established measures. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 22

Conclusion • Measurement validity is a necessary foundation for social research. • Achieving validity varies with the concept and circumstances of the study. • Plan ahead. • Carefully evaluate others’ research and remain skeptical. • Use statistical tests when appropriate. Chambliss, Making Sense of the Social World, 6 e. © SAGE Publishing, 2020 23