Chapter 5 Measuring Variables Sampling Variable and Measurement

+ Chapter 5 Measuring Variables & Sampling

+ Variable and Measurement n Variable – A condition/characteristic that can take on different values or categories n Measurement – the assignment of symbols or numbers to something according to a set of rules

+ Scales of Measurement n Nominal Scale n n use of symbols to classify or categorize Ordinal Scale n rank-order scale of measurement n n Allows to conclude who is 1 st, 2 nd, but not for how much! equal distances on scale not necessarily equal on dimension being measured n Difference between 1 st & 2 nd = 20 secs; difference between 2 nd & 3 rd = 60 secs. n Laredo #1 Most Diverse City in the US,

+ Entropy index (E): Index gauges how uniformly members of a population are spread. E = 0 = complete homogeneity or no diversity

+ Scales of Measurement (cont'd) n n Interval Scale n same properties of ordinal plus equal distances between adjacent numbers n e. g. , temperature on Fahrenheit scale Ratio Scale n highest scale of measurement n same properties of other scales plus absolute zero point n e. g. , weight, height

+ Psychometric Properties of Good Measurement n Reliability – Consistency or stability of the scores of your measurement instrument n Temperatures: 98. 9, 98. 6, 97. 6. 97. 8, 99. 0 RELIABLE? n Temperatures: 98. 0, 100. 3, 95. 3, 103. 0 RELIABLE? n Validity – Extent to which your measurement procedure is measuring what you think it is measuring and whether you have interpreted your scores correctly n A measure must be reliable in order to be valid but a reliable measure is not necessarily valid

+ Types of Reliability n n Test-Retest Reliability n consistency of individual scores over time n same test administered to individuals two times n correlate scores to determine reliability (high correlation? ) n how long to wait between tests? Equivalent-Forms Reliability n consistency of scores on two versions of test n each version of test given to different groups of individuals

+ Types of Reliability (cont'd) n Internal Consistency Reliability n consistency with which items on a test measure a single construct n involves comparing individual items within a single test n coefficient alpha or Cronbach’s Alpha (>. 70) is common index n e. HORMONY CLAIMS that with 5 Questions you can be MATCHED n n HIRES Social Psychology students. . Owner a social psychologist! Interrater Reliability n degree of agreement between two or more observers n interobserver agreement is the percentage of times different raters agree

+ Validity n Validity refers to the accuracy of the inferences, interpretations, or actions made on the basis of test scores. n The validity of test concerns what the test measures and how well it does so. It tells us what can be inferred from test scores (Anastasia & Urbina, 1997, p. 113) n Involves the measurement of constructs (e. g. , intelligence or happiness) n Do operational definitions accurately represent construct we are interested in? n LEADERSHIP. How do you measure leadership? Multiple indicators? n ULTIMATELY: PREVIOUS RESEARCH!

+ Methods Used to Collect Evidence of Validity n Content-Related Evidence (content validity): JUDGMENT of the degree to which the task(s) adequately represent the construct’s domain n n validity assessed by experts n do items appear to measure construct of interest? (FACE VALIDITY: Do questions address issues related to LOVE? n were any important content areas omitted? Romantic, Friendship n were any unnecessary items included? . . . Animal, classroom love. . Evidence Based on Internal Structure n how well do individual items relate to the overall test score or other items on the test: Different FACTOR LOADINGS? n factor analysis – statistical procedure used to determine the number of dimensions present in a set of items; Factors… Romantic…

+ Methods Used to Collect Evidence of Validity (cont'd) n Evidence Based on Relations to Other Variables n criterion-related validity: How good the scores of TEST predict or relate to an existing test… : n predictive validity – using scores obtained at one time to predict the scores on a criterion at a later time n concurrent validity – degree to which scores obtained at one time correctly relate to the scores on a known criterion obtained at the same time n E. G. , Administering a brand-new depression task that you created and at the same time ADMINISTERING THE BECK’S DEPRESSION INVENTORY… and the results of both SHOULD BE HIGHLY CORRELATED n EYETRACKING vs. Traditional behavioral measures (RT) n Run the experiment using eye-tracking and at the same time using RT Eye-tracking highly correlated to RT measures. .

+ Methods Used to Collect Evidence of Validity (cont'd) n Evidence Based on Relations to Other Variables n convergent validity – extent to which test scores relate to other measures of the same construct TRIANGULATION n Eye-tracking vs. Visual Moving window: Measure your eye movements as you read vs. pressing a word each time as you read the sentence. . n discriminant validity – extent to which your test scores do not relate to other test scores measuring different constructs: Test in Research methods…. Should not relate to therapy. n known groups validity evidence – extent to which groups that are known to different from one another actually differ on the construct being developed

+ Sampling Methods n Sample – a set of elements selected from a population n Population – the full set of elements from which the sample was selected

+ Sampling Methods (cont'd) n Sampling – process of drawing elements from population to form a sample n representative sample n equal probability method of selection method (EPSEM) n Statistic – a numerical characteristic of sample data n Parameter – a numerical characteristic of population data n Sampling error (Standard error) – difference between the value of the sample statistic and the value of the population parameter n You want a small sampling error? Use large N

+ Random Sampling Techniques n n Simple Random Sampling n choosing a sample in a manner in which everyone has an equal chance of being selected n sampling “without replacement” is preferred n Random numbers generators simplify the process or =rand()*n Stratified Random Sampling n random samples drawn from different groups or strata within the population n If gender stratification variable: Separate groups into M F n n Choose at random from the two groups! proportional stratified sampling involves insuring that each subgroup in sample is proportional to the subgroups in the population n If F population = 60%, choose 60% from F

+ Random number table http: //www. stattrek. com/statistics/random-number-generator. aspx

+ Random Sampling Techniques (cont'd) n Cluster Random Sampling n involves random selection of groups of individuals (clusters) n Cluster: Collective type of unit including multiple elements: neighborhoods, schools, families n e. g. , select at random 30 classrooms (from sampling distribution of 50: one-stage Cluster sampling). Include all subjects n Or select at random 10 subjects from the 30 classrooms (twostage)

+ Random Sampling Techniques (cont'd) n Systematic Sampling (similar to random sampling) n Involves three steps n (1) determine the sampling interval (symbolized by k) n (2) randomly select a number between 1 and k, and include that person in your sample: CHOOSE 5 n (3) also include each kth element in your sample 5 th person + 10 th (kth)= 15 So, subjects would follow the pattern: 5, 15, 25, 35, 45, and son on… (son increments of 10!) If in two (Choose 3): 3, 13, 23, 33, 43…

+ Nonrandom Sampling Techniques n Convenience Sampling – using research participants that are readily available – e. g. , college students n Quota Sampling – identifying quotas for individual groups and then using convenience sampling to select participants within each group

+ Nonrandom Sampling Techniques (cont'd) n Purposive Sampling – involves identifying a group of individuals with specific characteristics – e. g. , college freshmen who have been diagnosed with ADHD n Snowball Sampling – technique in which research participants identify other potential participants. n particularly useful in identifying participants from a difficult to find population

+ Random Selection and Random Assignment n Random selection involves selecting participants for research n n purpose is to obtain a representative sample Random assignment involves how participants are assigned to conditions within the research n purpose is to create equivalent groups to allow for investigation of causality n 20 Participants assigned at random to Condition A n 20 Participants assigned at random to Condition B

+ Determining Sample Size n If less than 100 use entire population n Larger sample sizes make it easier to detect an effect or relationship in the population n Compare to other research studies in area n Larger sample sizes are needed if population is n heterogeneous

+ Determining Sample Size (Power) n If N < 100 use entire population n Larger N make it easier to detect an effect/relationship in the population n Compare to other research studies in area n Larger sample sizes are needed if population is n Heterogeneous, you have multiple groups n if you want to increase precision n when you expect a small effect n for some statistical techniques n if you expect a low response rate n ***when you use less efficient methods of sampling

+ Sampling in Qualitative Research n Qualitative research focuses on in-depth study of one or a few cases. n Several different sampling methods are available. It is common to mix several different methods.