Chapter Five Nonexperimental Methods II Ex Post Facto

  • Slides: 55
Download presentation
Chapter Five: Nonexperimental Methods II: Ex Post Facto Studies, Surveys and Questionnaires, Sampling and

Chapter Five: Nonexperimental Methods II: Ex Post Facto Studies, Surveys and Questionnaires, Sampling and Basic Research Strategies The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Ex Post Facto Studies • • Ex post facto is a Latin phrase meaning

Ex Post Facto Studies • • Ex post facto is a Latin phrase meaning “after the fact. ” When we conduct an ex post facto study, we are using an IV “after the fact” – it has already varied before we arrived on the scene. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • • Surveys typically request our opinion on some

Surveys, Questionnaires, Tests, and Inventories • • Surveys typically request our opinion on some topic or issue that is of interest to the researcher. Types of Surveys • Descriptive • Analytic The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Descriptive – seeks to determine what percentage of

Surveys, Questionnaires, Tests, and Inventories • Descriptive – seeks to determine what percentage of the population has a certain characteristic, holds a certain opinion, or engages in a particular behavior. • Examples: • The Gallup Poll • Nielsen television ratings The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Analytic – seeks to determine what the relevant

Surveys, Questionnaires, Tests, and Inventories • Analytic – seeks to determine what the relevant variables are and how they might be related. • Questions need to be chosen carefully. • Pilot testing is generally necessary to determine the type of questions that will be used in the final survey instrument. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Developing a Good Survey or Questionnaire (see table

Surveys, Questionnaires, Tests, and Inventories • Developing a Good Survey or Questionnaire (see table 5 -1, p. 100). • Step 1 – decide what type of instrument to use. How will the information be gathered? • Will you use a mail survey, conduct a phone interview, or conduct the survey in person? Will you use trained interviewers? The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • • • Yes-No Questions • The respondent answers

Surveys, Questionnaires, Tests, and Inventories • • • Yes-No Questions • The respondent answers yes or no to the items. Forced Alternative Questions • The respondent must select between two alternative responses. Multiple-Choice Questions • The respondent must select the most suitable response from among several alternatives. Likert-Type Scales • The individual answers a question by selecting a response alternative from a designated scale. A typical scale might be the following: (5) strongly agree, (4) agree, (3) undecided, (2) disagree, or (1) strongly disagree. Open-Ended Questions • A question is asked to which the respondent must construct his or her own answer. • Step 2 – identify the types of questions to use. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • • • Step 1 – decide what type

Surveys, Questionnaires, Tests, and Inventories • • • Step 1 – decide what type of instrument to use. How will the information be gathered? Step 2 – identify the types of questions to use. Step 3 – write the items. They should be clear, short, and specific. • Use familiar vocabulary and be at the reading level of the individuals you intend to test. • Step 4 – Pilot-test and seek opinions from knowledgeable others. • It is important to ask others, especially professionals who have expertise in your area of research interest, to review your items. They may be able to detect biases and unintended wordings that you had not considered. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Step 5 – determine the relevant demographic data

Surveys, Questionnaires, Tests, and Inventories • Step 5 – determine the relevant demographic data to be collected. • Includes items such as age, sex, annual income, size of community, academic major, and academic classification. • Step 6 – determine administration procedures and develop instructions. • Instructions must be clear, concise, easy to follow, and thoroughly rehearsed (for in-person or phone interviews). • How will informed consent be obtained? The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Survey Administration Procedures • Mail Surveys • Personal

Surveys, Questionnaires, Tests, and Inventories • Survey Administration Procedures • Mail Surveys • Personal Interviews • Telephone Interviews The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • • Mail Surveys Advantages • The researcher does

Surveys, Questionnaires, Tests, and Inventories • • Mail Surveys Advantages • The researcher does not have to be present while the survey is being completed. • Surveys can be sent to a much larger number of participants than a single researcher could ever hope to contact in person. • Disadvantages • The researcher cannot be sure who actually completes the survey. • There is no guarantee the respondent answered the questions in the same order in which they appeared in the survey. • Low return rate. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • • Personal Interviews Advantages • When a trained

Surveys, Questionnaires, Tests, and Inventories • • Personal Interviews Advantages • When a trained interviewer administers a survey in a respondent’s home, the response rate climbs dramatically. • The trained interviewer is able to clarify ambiguous questions, make sure that all questions are answered in the proper sequence, and generally assist with any problems that the respondents may experience. • Disadvantages • The potential for considerable expenditure of time and money exists. • People not always at home. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • • Telephone Interviews Advantages • It is easy

Surveys, Questionnaires, Tests, and Inventories • • Telephone Interviews Advantages • It is easy to create random samples with random-digit dialing. • 95% of households in the U. S. have telephones. • Computer technology allows responses to be entered directly as they are made. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Telephone Interviews - Disadvantages • Incoming calls may

Surveys, Questionnaires, Tests, and Inventories • Telephone Interviews - Disadvantages • Incoming calls may be screened and potential respondents lost. • It is easy to say no to a phone interviewer. • Cannot use visual aids to clarify questions. • Cannot evaluate nonverbal cues such as facial expressions, gestures, and posture. • It is more difficult to establish rapport. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Tests and Inventories • Tests and inventories are

Surveys, Questionnaires, Tests, and Inventories • Tests and Inventories • Tests and inventories are designed to assess a specific attribute, ability, or characteristic possessed by the individual being tested. • Characteristics of Good Tests and Inventories • Validity – a test or inventory has validity when it actually measures what it is supposed to measure. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • • • Content validity – indicates that the

Surveys, Questionnaires, Tests, and Inventories • • • Content validity – indicates that the test items actually represent the type of material they are supposed to test. A panel of expert judges is often used to assess the content validity of test items. Concurrent validity – established when a score on a test or inventory agrees with another measure (e. g. aggression score on a test agrees with a clinician’s assessment). Criterion validity – established when a test score compares favorably with a predicted future outcome (e. g. SAT score and college success). The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Reliability • Refers to the extent that the

Surveys, Questionnaires, Tests, and Inventories • Reliability • Refers to the extent that the test or inventory is consistent in its evaluation of the same individuals over repeated administrations. • The greater the similarity between scores produced by the same individuals on repeated administrations, the greater the reliability of the test or inventory. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Reliability (assessment) The Psychologist as Detective, 4 e

Surveys, Questionnaires, Tests, and Inventories • Reliability (assessment) The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Reliability (assessment) • Test-retest procedure The Psychologist as

Surveys, Questionnaires, Tests, and Inventories • Reliability (assessment) • Test-retest procedure The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Reliability (assessment) • Test-retest procedure • The test

Surveys, Questionnaires, Tests, and Inventories • Reliability (assessment) • Test-retest procedure • The test is simply given a second time and scores from the two tests are compared; the greater the similarity, the higher the reliability The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Reliability (assessment) • Test-retest procedure • Split-Half technique

Surveys, Questionnaires, Tests, and Inventories • Reliability (assessment) • Test-retest procedure • Split-Half technique The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Reliability (assessment) • Test-retest procedure • Split-Half technique

Surveys, Questionnaires, Tests, and Inventories • Reliability (assessment) • Test-retest procedure • Split-Half technique • Involves dividing a test or inventory into two halves or subtests and then administering them to the same individuals on different occasions or administering the entire test and then splitting it into two halves. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories The Psychologist as

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests • Are given when an evaluation of an individual’s level of mastery or competence is desired. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests • Are given when an evaluation of an individual’s level of mastery or competence is desired. • The score that distinguishes passing from failing determines the minimum level of achievement that must be attained. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests • Aptitude tests The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests • Aptitude tests • Are used to assess an individual’s ability or skill in a particular situation or job. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests • Aptitude tests • Personality tests or inventories The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests • Aptitude tests • Personality tests or inventories • Measure specific aspects of an individual’s motivational state, interpersonal capability, or personality. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Sampling The Psychologist as Detective, 4 e

Sampling Considerations and Basic Research Strategies • Sampling The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Sampling • Deals with the question of

Sampling Considerations and Basic Research Strategies • Sampling • Deals with the question of who will participate in our research project and whether they are a representative group. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Research Strategies The Psychologist as Detective, 4

Sampling Considerations and Basic Research Strategies • Research Strategies The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • The main strategies used by researchers are

Sampling Considerations and Basic Research Strategies • The main strategies used by researchers are the: The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • The main strategies used by researchers are

Sampling Considerations and Basic Research Strategies • The main strategies used by researchers are the: • Single strata approach The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • The main strategies used by researchers are

Sampling Considerations and Basic Research Strategies • The main strategies used by researchers are the: • Single strata approach • Cross sectional approach The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • The main strategies used by researchers are

Sampling Considerations and Basic Research Strategies • The main strategies used by researchers are the: • Single strata approach • Cross sectional approach • Longitudinal approach The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Population The Psychologist as Detective, 4 e

Sampling Considerations and Basic Research Strategies • Population The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Population • The complete set of individuals

Sampling Considerations and Basic Research Strategies • Population • The complete set of individuals or events that we want to represent. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Sample The Psychologist as Detective, 4 e

Sampling Considerations and Basic Research Strategies • Sample The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Sample • The group that we select

Sampling Considerations and Basic Research Strategies • Sample • The group that we select to represent the population is called the sample. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Sample • The group that we select

Sampling Considerations and Basic Research Strategies • Sample • The group that we select to represent the population is called the sample. • Random sample The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Sample • The group that we select

Sampling Considerations and Basic Research Strategies • Sample • The group that we select to represent the population is called the sample. • Random sample • Every member of the population has an equal likelihood of being included in the sample. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Random sample • Every member of the

Sampling Considerations and Basic Research Strategies • Random sample • Every member of the population has an equal likelihood of being included in the sample. • Random sampling without replacement The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Random sample • Every member of the

Sampling Considerations and Basic Research Strategies • Random sample • Every member of the population has an equal likelihood of being included in the sample. • Random sampling without replacement • The member of population chosen for the sample is not eligible to be selected again once returned to the population. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Random sample • Every member of the

Sampling Considerations and Basic Research Strategies • Random sample • Every member of the population has an equal likelihood of being included in the sample. • Random sampling without replacement • The member of population chosen for the sample is not eligible to be selected again once returned to the population. • Random sampling with replacement • Member of population chosen for sample is eligible to be selected again once returned to the population. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Random sample • Every member of the

Sampling Considerations and Basic Research Strategies • Random sample • Every member of the population has an equal likelihood of being included in the sample. • Random sampling without replacement • The member of population chosen for the sample is not eligible to be selected again once returned to the population. • Random sampling with replacement • Member of population chosen for sample is eligible to be selected again once returned to the population. • Stratified random sampling The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Random sample • Every member of the

Sampling Considerations and Basic Research Strategies • Random sample • Every member of the population has an equal likelihood of being included in the sample. • Random sampling without replacement • The member of population chosen for the sample is not eligible to be selected again once returned to the population. • Random sampling with replacement • Member of population chosen for sample is eligible to be selected again once returned to the population. • Stratified random sampling • Involves dividing the population into subpopulations or strata and then drawing a random sample from one or more of these strata. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Basic Research Strategies The Psychologist as Detective,

Sampling Considerations and Basic Research Strategies • Basic Research Strategies The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Single-strata approach The Psychologist as Detective, 4

Sampling Considerations and Basic Research Strategies • Single-strata approach The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Single-strata approach • Seeks to acquire data

Sampling Considerations and Basic Research Strategies • Single-strata approach • Seeks to acquire data from a single, specified segment of the population. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Single-strata approach • Seeks to acquire data

Sampling Considerations and Basic Research Strategies • Single-strata approach • Seeks to acquire data from a single, specified segment of the population. • Cross-sectional research The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Single-strata approach • Seeks to acquire data

Sampling Considerations and Basic Research Strategies • Single-strata approach • Seeks to acquire data from a single, specified segment of the population. • Cross-sectional research • Involves the comparison of two or more groups of participants during the same, rather limited, time span. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Single-strata approach • Seeks to acquire data

Sampling Considerations and Basic Research Strategies • Single-strata approach • Seeks to acquire data from a single, specified segment of the population. • Cross-sectional research • Involves the comparison of two or more groups of participants during the same, rather limited, time span. • Longitudinal research The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education

Sampling Considerations and Basic Research Strategies • Single-strata approach • Seeks to acquire data

Sampling Considerations and Basic Research Strategies • Single-strata approach • Seeks to acquire data from a single, specified segment of the population. • Cross-sectional research • Involves the comparison of two or more groups of participants during the same, rather limited, time span. • Longitudinal research • Involves acquiring data from a single group of participants over a long period of time. The Psychologist as Detective, 4 e by Smith/Davis © 2007 Pearson Education