STAT 472 Survey Design Measurement and Research Design

STAT 472 Survey Design

Measurement and Research Design l There are five suggestion for coming up with a measure

Suggestion. 1 l Remember the conceptual Definition: The underlying principle for measure is to match it to the specific conceptual definition of the construct that will be used in the study

Suggestion. 2 l Keep an Open Mind Do not get locked into a single measure or type of measure. Be creative and constantly look for better measures, Avoid using one measurement instrument for all problems.

Suggestion. 3 l Borrow from others Do not be afraid to borrow from other researchers, as long as credit is given. Good ideas for measure can be found in other studies or modified from other measures.

Suggestion. 4 l Anticipate Difficulties Logical and practical problems often arise when trying to measure variables of interest. Sometimes a problem can be anticipated and avoided with careful forethought and planning

Suggestion. 5 l Do not forget your units of analysis Your measure should fit the units of analysis of the study and permit you to generalize to the universe of interest

Reliability and Validity Reliability and validity are central issues in all scientific measurement. l Perfect reliability and validity are virtually impossible to achieve l They are ideals researchers strive for. l Researchers want to maximize the reliability and validity indicator’s. l

l Reliability tells us about an indicator’s dependability and consistency l Validity tells us whether an indicator actually captures the meaning of the construct in which we are interested.

Reliability l It means the information provided by indicators ( e. g. questionnaire) does not vary as a result of characteristics of the indicator, instrument, or measurement device itself.

l For Example: Car Speedometer If I am driving at a constant slow speed on a level surface, but the speedometer needle jumps from one end to the other, then my speedometer is not a reliable indicator of how fast I am traveling

Types of reliability l Stability Reliability: is reliability across time It addresses the question: Does the measure or indicator deliver the same answer when applied in different time periods? Use Test-Retest method

Representative reliability l Is reliability across Subpopulations or groups of people It addresses the question: Does the indicator deliver the same answer when applied to different people? Use Independent knowledge about subpopulation or subgroups.

Equivalence Reliability l Is reliability across Indicators l It addresses the question: Does measure yield results across different indicators? l Is used when researchers use multiple indicators (several items in a questionnaire all measure the same construct) Use Split-Half Method or Cronbach’s Alpha