Section 3 How to Collect Quantitative Data Probability

Section 3: How to Collect Quantitative Data? Probability Sampling Survey and Questionnaire Index & Scale 1

Probability Sampling q The Need for Sampling Ø Time and Financial Limitations Ø Unknown Population Ø Destructiveness of the Evaluation Procedure q Concepts and Terminology of Sample Ø Population Ø Target population Ø Sampling unit Ø Sampling frame Ø Parameter/Statistics q Probability Sampling Designs Ø Simple random sampling Ø Systematic random sampling Ø Stratified sampling 2Ø Cluster sampling

Concept & Terminology Population Target Population Sampling Unit Sampling Frame Parameter Statistics 3

Probability Sampling Designs That part of the research plan indicates how cases are to be selected for observation. In sampling, we like to obtain a sample that will be representative of the target population. To be ‘representative’ means to provide a close approximation of certain characteristics of the target population. Example: If a population consists of all the full-time students of the Department of Social Studies in the XU, then a perfectly representative sample will be like XU students in all aspects: the sample has the same proportion of 1 st year students, 2 nd year students, the same proportion of Applied Sociology students, the same proportion of Social Policy students…. . as 4 are contained in the student body as a whole.

Probability Sampling (con’t) The quality of a sample must be judged in terms of the procedure that produced it in term of its sampling design. 5 n Simple Random Sampling n Systematic Random Sampling n Stratified Sampling n Cluster Sampling

Simple Random Sampling All cases in the population have a known probability of being included in the sample. Scientifically more acceptable. Not always feasible or economical. Sampling theory requires that mechanical or electronic aids be used to assure that chance alone dictates selection: n a. Mechanical n tossing a perfect coin, drawing a card from a well shuffled deck, drawing perfectly round, numbered balls from a thoroughly mixed basket.

Simple Random Sampling (con’t) b. Electronic aids n computer programmes that provide random selection or specially prepared tables of mechanically generated random numbers. n the random number table is quite commonly used. n 2 types: 1. Unrestricted sample—sampling with replacement. 2. Restricted sample—sampling without replacement. 7

Systematic Sampling More commonly applied than simple random sampling. Used when a complete list is available. Select every Kth (e. g. 10 th or 15 th) case from a complete list or file of the target population, starting with a randomly chosen case, known as a random start, from the first K cases on the list. 8

Systematic Sampling (con’t)

Stratified Sampling 10

Stratified Sampling Divide the population into 2 or more strata and 11 then take either a simple random sampling or systematic sample from each stratum. Population is first subdivided into 2 or more mutually exclusive segments-strata-based on categories of 1 or a combination of relevant variable. Subsamples are joined to form the complete, stratified sample. Stratified sampling is a method for obtaining a greater degree of representativeness— decreasing the probable sampling error.

Stratified Sampling (con’t) Sampling error is reduced by 2 factors in sampling design: a. a larger sample produces a smaller sampling error than a small sample. n b. a homogeneous population produces samples with smaller errors than does a heterogeneous population. n Example: To get a stratified sample of university students, one would first organize the population by class and then draw appropriate numbers of 1 st, 2 nd and final year student. The sampling error on the variable-class is reduced to zero. 12

Cluster Sampling The population is broken down into groups of cases-clusters and a sample of clusters is selected at random. 13

Cluster Sampling Cluster sampling may be applied when it is either impossible or impractical to compile an exhaustive list of elements composing the target population, for example, all church members in Hong Kong. Clusters generally consist of: a. natural groupings b. geographic units-census tracts; blocks n 14 Multistage cluster sampling involves sampling at 2 or more steps or stages.

Multistage Cluster Sampling 15

Survey & Questionnaire Features of Professional Survey Research Designs Advantages and Disadvantages of Survey Methods of Survey Research What is a Questionnaire? Construction of Questionnaire: Watch out for the Biases! Composite Measures: Index & Scale 16

Features of Professional Survey Research A large number of respondents are selected through probability sampling procedures to represent the population studied. Systematic questionnaire or interview procedures are applied to obtain information from respondents in a reliable and unbiased way. Sophisticated statistical techniques are used to analyze the data.

Survey Research Designs 1. Cross-sectional design: data on a cross section of respondents chosen to represent the larger population of interest are gathered at essentially one point in time. 2. Longitudinal design: data are obtained over an extended period of time. They are two types: trend studies and panel studies

The Major Advantages and Disadvantages of Survey Advantages: provides detailed descriptions of population. Disadvantages: cause-and-effect relationships cannot be established as easily in surveys as in experiments.

Methods of Survey Research 1. Face-to-face interviewing 2. Telephone interviewing Self-administered questionnaire

WHAT IS A QUESTIONNAIRE? Questionnaire is a document containing questions and other types of items designed to solicit information appropriate for analysis. Surveys include the use of a questionnaire—an instrument specifically designed to elicit information that will be useful for analysis. 21

Bias in Questionnaire Construction Wording Response set Leading questions Double barreled questions Memory problems 22

Composite Measures: Indexes and Scales It is difficult to measure a concept well with a single indicator or question. It is often necessary to rely on more than one indicator when measuring a concept. By combining several indicators into a composite measure, we generally obtain a more satisfactory overall representation of the concept, and the errors tend to cancel each other out, giving a more reliable measure. 23

Composite Measures: Indexes and Scales (con’t) Composite measures may be structured as: n a. index n n b. scale n 24 n (plural, "indices") - a composite measure which combines different items which are given equal weight in the determination of the score which represents the concept. a composite measure which combines different items which are given unequal weight in the determination of the score which represents the concept. They are efficient devices for data analysis.

Index An index is constructed through the simple accumulation of scores assigned to individual attributes. It usually refers to the arbitrary combination of indicators, such as when we simply add together the responses to separate items without regard to what each actually contributes to the measurement of the underlying concept. 25

Index with example For example: A question on abortion is included in the General Social Survey (GSS) (Davies and Smith 1985) § 26 Please tell me whether or not you think it should be possible for a pregnant woman to obtain a legal abortion if…. . (the interviewer then reads out each of the following statements. )

Index with example (con’t) § 1. If there is strong § § § 27 chance of serious defect in the baby? 2. If she is married and does not want any more children? 3. If the woman’s own health is seriously endangered by the pregnancy? 4. If the family has a low income and cannot afford any more children? 5. If she became pregnant as a result of rape? 6. If she is not married and does not want to marry the man?

How it is measured? Responses to these question may be not totally relevant to abortion. They include attitudes toward handicap (1), large families (2), women’s right (3), growing up in poverty (4), rape (5), and premarital sex (6). If only one of the questions was used, the answer surely does not reflect reality! How are separate measures combine? The simplest and most common procedure is just to add or to take an average of the scores of the separate items; this is what generally mean by an index. 28

How it is measured? (con’t) Good indexes provide an ordinal ranking of cases on a chosen variable. Indexes are based on this kind of assumption: A member of the Legco Council who voted for 7 conservative bills than one who only voted for 4 of them. We can make index for many concepts, such as public attitudes towards racial integration, happiness, religiosity, political involvement……. 29

Scales A scale is constructed through the assignment of scores to patterns of attributes. Thus, a scale takes advantage of any intensity structure that may exist among attributes. There are different types of scales: § a. Likert Scaling b. Semantic Differential Scaling § c. Bogardus Social Distance Scale § 30

Item format created by Likert is one of the most commonly used in contemporary questionnaire design n Ordinality of response categories “Strongly agree” n “Agree” n “Disagree” n “Strongly disagree” n 31

Likert Scaling (con’t) 32

Semantic Differential Scaling Asks respondents to choose between two opposite positions using qualifiers to bridge the distance between the two opposites 1 st determine dimensions subjects should judge Find two opposite terms (i. e. , polar extremes along each dimension) Prepare a rating sheet Avoid creating a biased response pattern by varying the placement of terms that are likely to be related to each other 33

Semantic Differential Scaling (con’t) 34

Bogardus Social Distance Scale 35

Bogardus Social Distance Scale (con’t) Directions: The steps below represent a social continuum – from close family relationships to complete physical and geographical separation – on which we may place people who are different from ourselves. Write a number beside each national and ethnic group listed below to indicate at what point on the continuum you would feel comfortable with members of those groups. You may keep your responses private, but you will be asked to discuss how you made your decisions. 36

Bogardus Social Distance Scale (con’t) Steps 1. To close kinship by marriage 2. To a social group as a personal friend 3. To my street as a neighbor 4. To employment in my place of work within my occupation 5. To citizenship in my country 6. As visitors only to my country 7. Would exclude from my country __America n __British n __Chinese n 37 __Japanese n __Korean n __Vietnamese n

Summary of Section 3 You should have learned: . . understand what is and why is sampling so important in quantitative research. . . the most commonly used sampling designs. . . what is social survey? . . how to construct a ‘good’ questionnaire . . how to construct index and scale. These are to advance your ability to construct a quality questionnaire.

Reading Schutt Russell K. (2015) Investigating the Social World (8 th Edition) Thousand Oaks, Calif. : Pine Forge Press, (Chapter 5: Sampling) May, T. (2011) Social Research (4 th Edition) OU Press, England. (Chapter 5 Social surveys: design to analysis)