UNIT 2 Research Design 1 Research Design Although

UNIT 2 Research Design 1

Research Design • Although every problem and research objective may seem unique, there are usually enough similarities among problems and objectives to allow decisions to be made in advance about the best plan to resolve the problem. • There are some basic research designs that can be successfully matched to given problems and research objectives. • The research design is the master plan specifying the methods and procedures for collecting and analyzing the needed information. 2

Types of Research Design • Three traditional categories of research design: • Exploratory • Descriptive • Causal • The choice of the most appropriate design depends largely on the objectives of the research and how much is known about the problem and these objectives. • The overall research design for a project may include one or more of these three designs as part(s) of it. • Further, if more than one design is to be used, typically we progress from Exploratory toward Causal. 3

Basic Research Objectives and Research Design Research Objective Appropriate Design To gain background information, to define terms, to clarify Exploratory problems and develop hypotheses, to establish research priorities, to develop questions to be answered To describe and measure marketing phenomena at a point in time To determine causality, test hypotheses, to make “if-then” statements, to answer questions Descriptive Causal 4

Research Design: Exploratory Research • Exploratory research is most commonly unstructured, “informal” research that is undertaken to gain background information about the general nature of the research problem. • Exploratory research is usually conducted when the researcher does not know much about the problem and needs additional information or desires new or more recent information. • Exploratory research is used in a number of situations: • To gain background information • To define terms • To clarify problems and hypotheses • To establish research priorities 5

Research Design: Exploratory Research • A variety of methods are available to conduct exploratory research: • Secondary Data Analysis • Experience Surveys • Case Analysis • Focus Groups • Projective Techniques 6

Research Design: Causal Research • Causality may be thought of as understanding a phenomenon in terms of conditional statements of the form “If x, then y. ” • Causal relationships are typically determined by the use of experiments, but other methods are also used. Causal Research is • undertaken with the aim of identifying cause and effect relationships amongst variables • are normally preceeded by exploratory and descriptive research studies • Often difficult to determine because of the influence of other variables (concommitant Variation and the presence of other hidden variables) Example: Higher ice-cream consumption causes more people to drown (indicative of a causal relationship (? )) 7

Research Design: Descriptive Research • Descriptive research is undertaken to provide answers to questions of who, what, where, when, and how – but not why. • Two basic classifications: • Cross-sectional studies • Longitudinal studies 8

Research Design: Descriptive Research Cross-sectional Studies • Cross-sectional studies measure units from a sample of the population at only one point in time. • Sample surveys are cross-sectional studies whose samples are drawn in such a way as to be representative of a specific population. • On-line survey research is being used to collect data for crosssectional surveys at a faster rate of speed. 9

Research Design: Descriptive Research Longitudinal Studies • Longitudinal studies repeatedly draw sample units of a population over time. • One method is to draw different units from the sampling frame. • A second method is to use a “panel” where the same people are asked to respond periodically. • On-line survey research firms recruit panel members to respond to online queries. 10

Research Design: Descriptive Research Longitudinal Studies • Two types of panels: • Continuous panels ask panel members the same questions on each panel measurement. • Discontinuous (Omnibus) panels vary questions from one time to the next. • Longitudinal data used for: • Market tracking • Brand-switching • Attitude and image checks 11

Experiments • An experiment is defined as manipulating (changing values/situations) one or more independent variables to see how the dependent variable(s) is/are affected, while also controlling the affects of additional extraneous variables. • Independent variables: those over which the researcher has control and wishes to manipulate i. e. package size, ad copy, price. • Dependent variables: those over which the researcher has little to no direct control, but has a strong interest in testing i. e. sales, profit, market share. • Extraneous variables: those that may effect a dependent variable but are not independent variables. 12

Experimental Design • An experimental design is a procedure for devising an experimental setting such that a change in the dependent variable may be solely attributed to a change in an independent variable. • Symbols of an experimental design: • O = measurement of a dependent variable • X = manipulation, or change, of an independent variable • R = random assignment of subjects to experimental and control groups • E = experimental effect • After-Only Design: X O 1 • One-Group, Before-After Design: O 1 X O 2 • Before-After with Control Group: • Experimental group: O 1 X O 2 • Control group: O 3 O 4 • Where E = (O 2 – O 1) – (O 4 – O 3) Treatment, control, comparison to do something to subjects who are randomly selected and randomly assigned to groups for the purpose of determining the 13 cause of an effect (difference between groups)

The language of experimentation • • • independent variable controlled variables experimental group control group random selection and random assignment to yield groups that are equivalent prior to treatment • measurement after (or before and after) treatment 14

Types of Experimental Designs • Basic Experimental Designs – A single independant variable is used to determine its impact on a single dependant variable. Basic experiments have the advantage of simplicity and easy measurability, but they also have the disadvantage of not being realistic • Factorial Experiment Designs – These allow for the investigation of the interaction of multiple (two or more) independant variables. Factorial experiments are more realistic but are also more complex and difficult to undertake than basic experiments • Laboratory experiments: those in which the independent variable is manipulated and measures of the dependent variable are taken in a contrived, artificial setting for the purpose of controlling the many possible extraneous variables that may affect the dependent variable • Field experiments: those in which the independent variables are manipulated and measurements of the dependent variable are made on test units in their natural setting • In field research (as in the lab) it is important to maximize treatment variance, minimize error variance, control extraneous variables 15

Experimental Research (1) Ø An experiment is a research method in which the conditions are controlled so that one or more variables can be manipulated in order to test a hypothesis Ø Typically, the purpose of undertaking experiments is to determine causal relationships between variables (chosen dependent and independant variables), while eliminating or controlling all other variables that may have an impact on these variables under investigation Ø The simplest form of experimental research involves only two variables: the independant variable, whose value is altered, and the dependant variable, whose value reflects the alteration in the independant variable‘s value 16

Experimental Research (2) Ø An ‚experimental group‘ is the group of subjects who are exposed to the experimental environment Ø A control group consists of individuals who are exposed to the ‚control condition‘ in a experiment, meaning, that they are not subject to the experiment in question, but are used as a reference to assess the impact on the experimental group Ø Some experiments can be quite complex, encompassing several independant variables. Special techniques have been developed to deal with such experiments 17

Issues in Experimental Design Manipulation of the Independant Variable Four Basic Elements of an Experiment in the Business Field Selection and Measurement of the Dependant Variable Selection and Assignment of Test Units Control over Extraneous Variables 18

Issues in Experimental Design: Manipulation of the Independant Variable Ø The independant variable‘s value can be altered without bring about any change in other variables – except the dependant variable Ø In business research, the independant variable can be qualitative or nonquantitative (for e. g. , the training programs, financial reporting formats), or quantitative (for e. g. , the amount of Rupees spent on training the employees in Organization X) 19

Issues in Experimental Design: Selection and Measurement of the Dependant Variable Ø The dependant variable‘s value depends or is determined by changes in the value of the independant variable, which in turn, is manipulated by the researcher as part of the experiment Ø The choice of dependant variable by the researcher can sometimes be a difficult, not-so-obvious undertaking, and requires considerable skill and insight on the part of the researcher in order to avoid making mistakes which reduce the value of the research (Example: New Products Introduction and Sales Potential) Ø The time factor should be taken into consideration when choosing a dependant variable, as sometimes the outcomes are measurable after a long time 20

Issues in Experimental Design: Selection and Assignment of Test Units Ø The Test Units are the subjects of the experimental research and can include individuals, organizational units, sales territories Ø Examples of Test Units: Consumers, Supermarkets, Functional Departments in an organization Ø In selecting test units, certain possible types of error must be taken into consideration, e. g. random sampling error (test units in the experimental and control groups should ideally have the same key characteristics but this may not be the case with statistical random assignment of the test units) and sample selection error (an administrative procedural error caused by improper selection of the sample, resulting in the introduction of a bias) 21

Issues in Experimental Design: Control Over Extraneous Variables (1) Extraneous Variable A Extraneous Variable B INDEPENDANT VARIABLE Experiment Extraneous Variable C Extraneous Variable D Environment 22

Issues in Experimental Design: Control Over Extraneous Variables (2) Ø There are types of extraneously-conditioned errors which have to be considered in the experimental environment as hey effect the quality of the research: Ø Constant Experimental Error – This occurs when extraneous influences which are not controlled or eliminated have a similar impact on the experiment‘s dependant variable(s) every time the experiment is performed. Demand Characteristics – This occurs when the research subject(s) are unintentionally exposed to the experimenter‘s hypothesis, causing them to respond or act in a manner which they may not have adopted were they not exposed to this information Ø Experimenter Bias – This occurs when the experimenter‘s presence, actions, or comments influences the research subjects‘ behaviour, making them to try to appear more favourable to the experimenter Ø Guineau Pig Effect – This occurs when theme of the experiment causes the research subjects‘ to consciously modify their attitudes in order to please the experimenter Ø Hawthorne Effect – This is the unintended effect on the results of a research experiment which is caused by the subjects knowing that they are the participants 23

Issues in Experimental Design: Control Over Extraneous Variables (3) Ø In order to reduce the chances of errors from reducing the value of an experiment, several counter-measures can be adopted, for example: § Making it difficult for the research subjects to know what the experiment is all about § Using trained and experienced experimenters § Designing experimental situations with a view to minimizing the chances of error § Preventing social interaction among research subjects so that they don‘t influence each other (joint decisions as opposed to the desired individual responses) 24

Issues in Experimental Design: Control Over Extraneous Variables (4) Ø Many times, extraneous variables cannot be controlled or eliminated by the experimenter. However, researchers do have some options at their disposal to help mitigate the impact of the extraneous variables on their experiments Ø Consistency of Conditions – This means that the subjects in experimental groups are exposed to situations that are exactly alike, except for the differing conditions of the independant variable (e. g. all experimental sessions are conducted in the same room at the same time by the same experimenter) Ø Counterbalancing – This strives to eliminate the so-called ‚order of presentation bias‘ which arises when research subjects, who are participating in multiple experimental phases, acquire experience in the initial experimental phase which enables them to perform better in subsequent phases (e. g. job assembly) Ø Blinding – This is used in order to control the research subjects‘ knowledge of whether or not they have been exposed to an experimental treatment, e. g. research subjects in a Coca-Cola taste test may be told that they have or have not been given a new Cola product in order to test their reactions Ø Random Assignment – This is used to randomly assign the research subjects to experimental groups as a means of curtailing the impact of 25 extraneous variables

Validity in Research • Refers to whether the research actually measures what it says it’ll measure. Validity is the strength of our conclusions, inferences or propositions. – Internal Validity: the difference in the dependent variable is actually a result of the independent variable – External Validity: the results of the study are generalizable to other groups and environments outside the experimental setting – Conclusion Validity: we can identify a relationship between treatment and observed outcome – Construct Validity: we can generalize our conceptualized treatment and outcomes to broader constructs of the same concepts Reliability in Research The consistency of a measurement, or the degree to which an instrument measures the same way each time it is used under the same condition with the same subjects. In short, it is the repeatability of your measurement. A measure is considered reliable if a person's score on the same test given twice is similar. It is important to remember that reliability is not measured, it is estimated. Measured by test/retest and internal consistency. 26

Validity and Reliability The relationship between reliability and validity is a fairly simple one to understand: a measurement can be reliable, but not valid. However, a measurement must first be reliable before it can be valid. Thus reliability is a necessary, but not sufficient, condition of validity. In other words, a measurement may consistently assess a phenomena (or outcome), but unless that measurement tests what you want it to, it is not valid. How Valid Are Experiments? • An experiment is valid if: • the observed change in the dependent variable is, in fact, due to the independent variable (internal validity) • if the results of the experiment apply to the “real world” outside the experimental setting (external validity) 27

Internal Validity Ø Internal Validity – This refers to whether the experimental treatment was the sole cause of observed changes in the dependant variable. If the observed results were influenced by extraneous variables, then a valid conclusion about the relationship between the experimental treatment and the dependant variable cannot be made Ø There are 6 types of extraneous variables that may jeopardize internal validity – history effect, maturation effect, test effect, instrumentation effect, selection effect and mortality effect External Validity Ø External Validity – This refers to the quality of the researcher or experimenter to generalize beyond the data of an experiment to other subjects or other groups in the population under study, i. e. the external environment (e. g. are the results of a new product study in district A be applicable to the whole country? ) Ø External validity can be jeopardized if internal validity of an experiment is lacking Ø Some issues have to be considered in the context of external validity, such as, the choice of research subjects and trade-offs between internal and external validity (e. g. : laboratory experiments have more internal validity than field experiments, but they have comparatively less external vaidity) 28

The language of experimentation • • • independent variable controlled variables experimental group control group random selection and random assignment to yield groups that are equivalent prior to treatment • measurement after (or before and after) treatment 29

Normal variation in the dependent variable that is present if the independent variable has no effect. Remember this example Dependent Variable was exam scores. Experimental Group Taught using new method. Control Group Taught using usual method. Score higher on Exam. Score lower on Exam. Maybe this group is smarter. Maybe this group isn’t as smart. No way--groups were equivalent. 30

In the experimental group, the dependent variable ends up here Notice that it can end up there even if the treatment had no effect--it can end up there 5 times out of a 31 hundred (p<. 05)

Remember---there is normal variation in the dependent variable even if the independent variable has no effect. Nothing stays the same!!!!! 32

Measurement and Scaling Measurement means assigning numbers or other symbols to characteristics of objects according to certain prespecified rules. – One-to-one correspondence between the numbers and the characteristics being measured. – The rules for assigning numbers should be standardized and applied uniformly. – Rules must not change over objects or time. 33

Measurement and Scaling involves creating a continuum upon which measured objects are located. Consider an attitude scale from 1 to 100. Each respondent is assigned a number from 1 to 100, with 1 = Extremely Unfavorable, and 100 = Extremely Favorable. Measurement is the actual assignment of a number from 1 to 100 to each respondent. Scaling is the process of placing the respondents on a continuum with respect to their attitude toward department stores. 34

Nominal scale §Classifies data according to a category only. §E. g. , which color people select. §Colors differ qualitatively not quantitatively. §A number could be assigned to each color, but it would not have any value. §The number serves only to identify the color. §No assumptions are made that any color has more or less value than any other color. 35

Nominal scale § § § Assign subjects to groups or categories No order or distance relationship No arithmetic origin Only count numbers in categories Only present percentages of categories Chi-square most often used test of statistical significance 36

Primary Scales of Measurement Ordinal Scale • A ranking scale in which numbers are assigned to objects to indicate the relative extent to which the objects possess some characteristic. • Can determine whether an object has more or less of a characteristic than some other object, but not how much more or less. • Any series of numbers can be assigned that preserves the ordered relationships between the objects. • In addition to the counting operation allowable for nominal scale data, ordinal scales permit the use of statistics based on centiles, e. g. , percentile, quartile, median. 37

Ordinal scale §Classifies data according to some order or rank; e. g. names ordered alphabetically §With ordinal data, it is fair to say that one response is greater or less than another. §E. g. if people were asked to rate the hotness of 3 chili peppers, a scale of "hot", "hotter" and "hottest" could be used. Values of "1" for "hot", "2" for "hotter" and "3" for "hottest" could be assigned. ØThe gap between the items is unspecified. 38

Ordinal scale ØCan include opinion and preference scales ØMedian but not mean ØNo unique, arithmetic origin ØItems cannot be added ØIn marketing research practice, ordinal scale variables are often treated as interval scale variables 39

Primary Scales of Measurement Interval Scale • Numerically equal distances on the scale represent equal values in the characteristic being measured. • It permits comparison of the differences between objects. • The location of the zero point is not fixed. Both the zero point and the units of measurement are arbitrary. • Any positive linear transformation of the form y = a + bx will preserve the properties of the scale. • It is meaningful to take ratios of scale values. • Statistical techniques that may be used include all of those that can be applied to nominal and ordinal data, and in addition the arithmetic mean, standard deviation, and other statistics commonly used in marketing research. 40

Interval scale §assumes that the measurements are made in equal units. §i. e. gaps between whole numbers on the scale are equal. §e. g. Fahrenheit and Celsius temperature scales §an interval scale does not have to have a true zero. e. g. A temperature of "zero" does not mean that there is no temperature. . . it is just an arbitrary zero point. §can’t perform full range of arithmetic equations. 40 degrees is not twice as hot as 20 degrees §permissible statistics: count/frequencies, mode, median, mean, standard deviation 41

Primary Scales of Measurement Ratio Scale • Possesses all the properties of the nominal, ordinal, and interval scales. • It has an absolute zero point. • It is meaningful to compute ratios of scale values. • Only proportionate transformations of the form y = bx, where b is a positive constant, are allowed. • All statistical techniques can be applied to ratio data. 42

Ratio scale Øsimilar to interval scales except that the ratio scale has a true zero value. Øe. g. the time something takes Øallows you to compare differences between numbers. Øpermits full arithmetic operation. Øif a train journey takes 2 hours and 30 min, then this is half as long as a journey which takes 5 hours. 43

Ratio scale • Indicates actual amount of variable – Shows magnitude of differences between points on scale – Shows proportions of differences • All statistical techniques useable • Most powerful with most meaningful answers • Allows comparisons of absolute magnitudes 44

Primary Scales of Measurement 45

Primary scales of measurement Nominal Ordinal Interval Ratio Numbers assigned to runners 4 81 9 Rank order of winners Third Place Second Place First Place Performance rating on a 0 to 10 Scale 8. 2 9. 1 9. 6 Time to finish in seconds 15. 2 14. 1 13. 4 46

Always use the most powerful scale possible Adding sophistication to scales • Concept: Desire to watch Star Wars movies – If a Star Wars movie is on television will you watch it? • Yes _____ No _____ – How likely are you to watch a Star Wars movie shown on television? • Very Likely ____ Indifferent ___ • Unlikely _____ Very Unlikely _____ 47

A classification of scaling techniques SCALING TECHNIQUES Comparative scales Paired comparison Rank order Constant sum Non-comparative scales Others Likert Continuous rating scales Semantic differential Itemized rating scales Stapel 48

Types of scaling Techniques Ø COMPARATIVE SCALES • Involve the respondent directly comparing stimulus objects. • e. g. How does Pepsi compare with Coke on sweetness Ø NON-COMPARATIVE SCALES • Respondent scales each stimulus object independently of other objects • e. g. How would you rate the sweetness of Pepsi on a scale of 1 to 10 49

A Comparison of Scaling Techniques • Comparative scales involve the direct comparison of stimulus objects. Comparative scale data must be interpreted in relative terms and have only ordinal or rank order properties. • In noncomparative scales, each object is scaled independently of the others in the stimulus set. The resulting data are generally assumed to be interval or ratio scaled. 50

Comparative scales: paired comparison items If we have brands A, B, C and D, we would have respondents compare • A and B • A and C • A and D • B and C • B and D • C and D –usually limited to N < 15 51

Comparative scales: paired comparison items Please indicate which of the following airlines you prefer by circling your more preferred airline in each pair: Air Canada Air Transat Horizon Air West. Jet Air Canada Horizon Air West. Jet Air Canada West. Jet Air Transat Horizon Air Transat 52

Comparative scales: constant sum scales Allocate a total of 100 points among the following softdrinks depending on how favorable you feel toward each; the more highly you think of each soft-drink, the more points you should allocate to it. (Please check that the allocated points add to 100. ) Coca-Cola _____points 7 -Up _____ points Mirinda _____ points Fanta _____ points Pepsi-Cola _____ points 100 points 53

Comparative scales: constant sum scales Please divide 100 points among the following characteristics so the division reflects the relative importance of each characteristic to you in the selection of a bank Hours of service ________ Friendliness ________ Distance from home ________ Investment vehicles ________ Parking facilities _________ 54

Comparative scales: rank order scales Rank the following soft-drinks from 1 (best) to 5 (worst) according to your taste preference: Coca-Cola _____ 7 -Up _____ Fanta _____ Pepsi-Cola _____ Mountain Dew _____ √ Top and bottom rank choices are ‘easy’ √ Middle ranks are usually most ‘difficult’ 55

Relative Advantages of Comparative Scales • Small differences between stimulus objects can be detected. • Same known reference points for all respondents. • Easily understood and can be applied. • Involve fewer theoretical assumptions. • Tend to reduce halo or carryover effects from one judgment to another. 56

Relative Disadvantages of Comparative Scales • Ordinal nature of the data • Inability to generalize beyond the stimulus objects scaled. 57

Comparative Scaling Techniques Paired Comparison Scaling • A respondent is presented with two objects and asked to select one according to some criterion. • The data obtained are ordinal in nature. • Paired comparison scaling is the most widely used comparative scaling technique. • With n brands, [n(n - 1) /2] paired comparisons are required • Under the assumption of transitivity, it is possible to convert paired comparison data to a rank order. 58

Paired Comparison Selling The most common method of taste testing is paired comparison. The consumer is asked to sample two different products and select the one with the most appealing taste. The test is done in private and a minimum of 1, 000 responses is considered an adequate sample. A blind taste test for a soft drink, where imagery, self-perception and brand reputation are very important factors in the consumer’s purchasing decision, may not be a good indicator of performance in the marketplace. The introduction of New Coke illustrates this point. New Coke was heavily favored in blind paired comparison taste tests, but its introduction was less than successful, because image plays a major role in the purchase of Coke. A paired comparison taste test 59

Comparative Scaling Techniques Rank Order Scaling • Respondents are presented with several objects simultaneously and asked to order or rank them according to some criterion. • It is possible that the respondent may dislike the brand ranked 1 in an absolute sense. • Furthermore, rank order scaling also results in ordinal data. • Only (n - 1) scaling decisions need be made in rank order scaling. 60

Comparative Scaling Techniques Constant Sum Scaling • Respondents allocate a constant sum of units, such as 100 points to attributes of a product to reflect their importance. • If an attribute is unimportant, the respondent assigns it zero points. • If an attribute is twice as important as some other attribute, it receives twice as many points. • The sum of all the points is 100. Hence, the name of the scale. 61

Itemized Rating Scales Semantic differential scale The Likert scale Stapel scale 62

Non-Comparative Scales 63

Semantic differential scale Here a number of statements that could be used to describe Tesco. For each statement tick ( ) the box that best describes your feelings about Tesco. Modern Store Old- fashioned store Low prices High prices Unfriendly staff Friendly staff Narrow product range Wide product range Sophisticated customers Unsophisticated customers 64

Likert scale Cost is the most important consideration when buying a new car Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree 1 2 3 4 5 65

AGREEMENT • Strongly agree • Agree • Undecided • Disagree • Strongly disagree • Agree very strongly • Agree • Disagree strongly • Disagree very strongly To what extent do you agree or disagree that X is a high quality brand? • Agree strongly • Agree moderately • Agree slightly • Disagree moderately • Disagree strongly • Disagree • Tend to disagree • Tend to agree • Agree strongly • Agree • Disagree • Agree • Undecided • Disagree • Completely agree • Mostly agree • Slightly disagree • Mostly disagree • Completely disagree 66

FREQUENCY Eg. How frequently do you go swimming? • Very frequently • Frequently • Occasionally • Rarely • Very rarely • Never • Always • Very frequently • Occasionally • Rarely • Very rarely • Never • Always • Usually • About half the time • Seldom • Never • A great deal • Much • Somewhat • Little • Never • Often • Sometimes • Seldom • Never • Always • Very often • Sometimes • Rarely • Never • Almost always • To a considerable degree • Occasionally • Seldom 67

IMPORTANCE How important is price to you when purchasing jeans? • Very important • Important • Moderately important • Of little importance • Unimportant QUALITY • Very important • Moderately important • Unimportant How would you rate the quality of Toshiba laptops? • Very good • Good • Barely acceptable • Poor • Very poor • Extremely poor • Below average • Above average • Excellent • Good • Fair • Poor 68

LIKELIHOOD How likely will you be to purchase a car in the next 6 months? • Likely • Unlikely • To a great extent • Somewhat • Very little • Not at all • True • False • Definitely • Very probably • Possibly • Probably not • Very probably not • Almost always true • Usually true • Often true • Occasionally true • Sometimes but infrequently true • Usually not true • Almost never true • True of myself • Mostly true of myself • About halfway true of myself • Slightly true of myself • Not at all true of myself 69

A Stapel scale for measuring a store’s image Select a plus number for words that you think describe the store accurately. The more accurately you think the work describes the store, the larger the plus number you should choose. Select a minus number for words you think do not describe the store accurately. The less accurately you think the word describes the store, the larger the minus number you should choose, therefore, you can select any number from +5 for words that you think are very accurate all the way to -5 for words that you think are very inaccurate. Tesco +5 +4 +3 +2 +1 HIGH POOR WIDE QUALITY SERVICE VARIETY -1 -2 -3 -4 -5 70

SOME BASIC CONSIDERATIONS WHEN SELECTING A SCALE Number of categories Forced versus non-forced choice Odd or even number of scale categories Balanced versus nonbalanced alternatives 71

Odd versus even if neutral responses likely, use odd number Odd Even Strongly agree _____ Strongly agree_____ Agree _____ Neutral _____ Disagree _____ Strongly disagree_____ 72

Balanced vs. Unbalanced Balanced Unbalanced Very good ______ Excellent ______ Good ______ Very good ______ Fair ______ Good ______ Poor ______ Fair ______ Very poor ______ Poor ______ 73

Forced vs. Unforced Forced Extremely reliable ___ Very reliable ___ Somewhat unreliable ___ Very unreliable ___ Extremely unreliable ___ Don’t know ___ 74

Labeled vs. End anchored Labeled End Anchored Excellent _____ Very good _____ Fair _____ Poor _____ Very Poor _____ Excellent Poor _____ 75

Intervals may not reflect the semantic meaning of the Adjectives Excellent _____ Labeled Intervals are not equal Excellent _____ Very good _____ Fair _____ Poor _____ Very poor _____ Intervals are not equal Very poor _____76

Number of scale points 5 Point Excellent 10 Point _____ Excellent _____________ _____________ _____ Poor _________ Poor _______ _______ 77

CHARACTERISTICS OF GOOD MEASUREMENT SCALES 1. Reliability • The degree to which a measure accurately captures a true outcome without error • synonymous with repetitive consistency 2. Validity • The degree to which a measure faithfully represents the underlying concept (it asks the right questions) 3. Sensitivity • The ability to discriminate meaningful differences between attitudes. The more categories the more sensitive (but less reliable) 78

VALIDITY AND RELIABILITY § Reliability can be more easily determined than validity § If it is reliable, it may or may not be valid § If a measure is valid, it may or may not be reliable § If it is not reliable, it cannot be valid § If it is not valid, it may or may not be reliable 79

Reliability and Validity Neither Reliable Nor Valid Reliable But Not Valid Reliable And Valid 80

Example of low validity, high reliability • Scale is perfectly accurate, but is capturing the wrong thing; for example, it measures consumers’ interest in creative writing rather than preference for kinds of stationery. 81

Example of modest validity, low reliability • Scale genuinely measures consumers’ interest in kinds of stationery, but poorly worded items, sloppy administration, data entry errors lead to random errors in data 82

Reliability The reliablity of a measure indicates the exte nt to which it is without bias and hence ensures consistent measurement across time and across the various items in the instrument. The ability of a measure to remain the same over time – despite uncontrollable testing conditions or the state of the respondents themselves is indicative of its stability and low vulnerablity to changes in situation. 2 tests of reliability are Test retest reliability & parallel form reliability 83

Test – Retest Reliability The reliability coefficient obtained with a repetition of the same measure on a second occasion is called Test Retest reliability. The higher it is, the better is the test Retest reliability and consequently, the stability of the measure across time. Parallel form Reliability: When responses on 2 comparable sets of measures tapping the same construct are highly correlated, it is parallel form reliability. i. e. when wordings and order of sequencing questions change, and the forms are highly correlated , we can ascertain that the measures are reasonably reliable with minimal error variance caused by wording, ordering or other factors. Interitem consistency Reliability: This is a test of the consistency of respondents answers to all the items in a measure. To the degree that items are independent measures of the same concept, hey will be correlated with one another. The higher the coefficients, the better is the measuring instrument. Split Half Reliability: It reflects the correlations between 2 halves of an instrument. The estimates would vary depending on how the items in the measure are split into 2 halves. 84

Validity ensures the ability of a scale to measure the intended concept. Validity tests may be grouped under the following three broad headings namely: 1. Content validity 2. Criterion- Related validity 3. Construct Validity 85

Content Validity: It ensures that the measure includes an adequate and representative set of items that tap the concept. The more the scale items represent the domain or universe of the concept being measured, the greater the content validity. Face Validity: It indicates that the items that are intended to measure a concept, do on the face of it look like they measure the concept. Criterion Related Validity: It is established when the measure differentiates individuals on a criterion it is expected to predict. . It is done by establishing concurrent validity or predictive validity Concurrent Validity: It is established when the scale discriminates individuals who are known to be different, i. e. they should score differently on the instrument. Predictive Validity: It indicates the ability of the measuring instrument to differentiate among individuals with reference to a future criterion. 86

Construct Validity: It testifies to how well the results obtained from the use of measure fit theories around which the test is designed. This is assessed through convergent and discriminant validity. Convergent Validity: It is established when the scores obtained with 2 different instruments measuring the same concept are highly correlated. Discriminant Validity: It is established when, based on theory, 2 variables are predicted to be uncorrelated and the scores obtained by measuring them are indeed empirically found to be so. 1. 2. 3. Some of the ways through which validity are established are: Correlational analysis Factor analysis Multi matrix analysis 87

Now this is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning. Sir Winston Churchill, 1942. 88