Quasiexperiments Quasiexperimental designs Experimental designs for studies in

Quasi-experiments Quasi-experimental designs Experimental designs for “studies in nature”. Studying naturally occurring events q Measurement studies q Retrospective designs Evaluate existing groups or program q Single shot survey or measure q Non-equivalent groups q Time series designs Psychology 242, Dr. Mc. Kirnan Research Ethics

Psychology 242 Introduction to Research 2 True v. quasi-experimental designs True experiments: Quasi-experiments: Emphasize internal validity n Assess cause & effect (in Emphasize external validity n Describe “real” / naturally occurring events n Clear to exploratory hypotheses relatively artificial environment) n Test clear, a priori hypotheses Participants assigned to Existing or non-equivalent experimental v. control groups n Random or matching n Non-random assignment n Participants & experimenter n Participants not blind Blind to assignment n Control group not possible? Control study procedures Control often not possible § Create / manipulate independent n May not be able to manipulate variable the independent variable § Control procedures & measures n Partial control of procedures & measures Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 3 Quasi-experiments: naturally occurring events Studying naturally occurring events q Measurement studies q Retrospective designs Evaluate existing groups or program q Single shot survey or measure q Non-equivalent groups q Time series designs Psychology 242, Dr. Mc. Kirnan Research Ethics Back Home page Next

Psychology 242 Introduction to Research 4 Measurement studies 1. Naturally occurring events; examples Event Study question (“Predictor”) (“Outcome”) Natural disaster / stressor 3 -mile island S. F. earthquake Stress -> immune system Stress & coping Crime / trauma Iraq service, 9 / 11 / 01 PTSD & treatment Historical event 9/11 & air travel ban Economic collapse Contrails & climate change Voting patterns Publicity / cultural event Info. re: Hormone replacement Psychology 242, Dr. Mc. Kirnan Health behavior Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 5 Naturally occurring events, 2 § Independent variable: § § Predictor variable (e. g. , natural disaster) often assessed after the event (post-hoc). Researcher has little control over dose / type of predictor § Participant selection § § No control over who is exposure to event Some control over selection of sample (e. g. , via targeted sampling) § Many potential confounding variables § Outcome (dependent) variables: § § n No control with archival data Some control with surveys Use retrospective (measured) variables to clarify interpretation of outcomes or test hypothesis. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 6 Naturally occurring events: Retrospective designs Using retrospective (measured) variables to clarify interpretation of outcomes or test hypothesis. Retrospective variable(s) Event Outcome variable Social support (“Predictor variable”) earthquake [v. control city? ] Psych. history crime / trauma [archive? Self-report? ] [v. control people? ] Personal attitudes historical event voting patterns Demographics cultural event health behavior Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. stress & coping mental health Back Home page Next

Psychology 242 Introduction to Research 7 Quasi-experiments: Existing groups Studying naturally occurring events q Measurement studies q Retrospective designs Evaluate existing groups or program Single shot survey or measure q q Non-equivalent groups q Time series designs Psychology 242, Dr. Mc. Kirnan Research Ethics Back Home page Next

Psychology 242 Introduction to Research 8 Existing groups: Single self-selected group; no comparison possible ü users of psychotherapy (or any product) ü members of group or cult [contrast with demographically matched controls? ] Two or more groups, with self-selection and / or "non-blind" assignment ü Psychological interventions: therapy v. wait list, etc. Two or more groups, no random assignment ü Comparing schools / cities / existing groups… Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Group 9 “One shot” case studies Psychology 242 Introduction to Research Naturally occurring event or social change May control selection of study group, or must use Convenience sample. Other data may be available about group. Typically no control over event. Not a true Independent Variable Observe 1 Dependent Variable(s): May or may not have control over measures (e. g. , surveys v. archival measures). Ø Typical use: Surveys or measures after an event. Ø Heuristic value: generating hypotheses for later study or confirm controlled data in “real world” setting. Ø Internal / External validity: Ø No control over selection of people into the event. Ø Potentially no control over selection into measurement group. Ø No control group; uncontrollable event, or other groups may not “need” the intervention (e. g. , therapy) Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 10 Example of one-shot case study Example: Consumer Reports psychotherapy survey [Click for paper] Research questions: Ø Does psychotherapy “work” from consumer view? Ø Who gets therapy / what does it consist of? Ø Do consumer responses vary by type of therapy? Research approach: Ø One shot case study / survey Sampling frame: Ø Any therapy or psychological service user Ø No real information re: population of therapy users. Sampling procedure: Ø 4, 100 Consumer reports readers responding to “in magazine” mail-back survey form Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 11 One shot: Consumer reports survey, 2 Experimental Controls Ø Evaluate by gender, type of treatment, medications, to provide more differentiated analysis Negatives: Ø Selection bias Øno control over who got therapy (self-selection) Øof those who got therapy, no control over who returned a survey (secondary self-selection) Ø Cursory outcome measures: satisfaction rather than mental health Positives: Ø Huge, national sample Ø Wholly anonymous, 3 rd party data collection; less bias Ø “Real world” assessment of product quality Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 12 One shot: Consumer reports survey, 3 Key distinction in psychological interventions: “efficacy” v. “effectiveness” research n Efficacy; “true” experimental design / Lab-basis ü Rigorous controls; High internal validity ü Test basic theory or highly specific technique Do the specific ingredients (or theory…) of this treatment validly induce the key outcome? n Effectiveness; quasi-experimental; “natural” or applied setting ü Less or no control; naturally occurring treatment ü High external validity Does treatment “work” in real patients w/real therapists? Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research Comparisons of study types: Efficacy experiment v. • One specific diagnosis • Rigorous control group • “no treatment” condition • “attention control” • Random assignment • Manualized / uniform treatments • High Fidelity to treatment method • Fixed number of sessions. • Well operationalized outcomes, e. g. , • clinician-diagnosed disorder • Standard / validated self-report symptom scales • “Blind" raters or diagnosticians ("single-blind“: patient & therapist know what the treatment is. . ) • Patients followed for a fixed period 13 (Consumer reports 4) Effectiveness research • Multiple diagnoses & severities • No control group, 2 nd controls • Archival, via pt. characteristics • Self-selection; “shopping” • Multiple / mixed treatments • Highly tailored to patient • # sessions is patient based. • Diverse, self-referenced outcomes • Subjective sense of “wellness” • Lessening of “problem” behaviors or moods • Personal assessment of functioning • Self-rated: cannot be "blind" • Diverse times since treatment • Retrospective rather than prospective Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 14 Consumer reports survey, 5. Survey findings on therapy effectiveness: § People who got more treatment (> 6 months) did better. § For patients’ presenting problem(s) all specialists did about the same. Psychology 242, Dr. Mc. Kirnan § For general ψ health MH specialists did best, marriage counselors worst. Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 15 Consumer reports survey, 6. Other effectiveness / descriptive findings: § 3/4 went to mental-health specialist. § Patients who rated themselves worse at outset made the most progress. § AA very highly evaluated § Therapy did as well as medications § 40% got drugs; § § § MDs gave medications to 83% of patients MH pros; 20% drug treatments 50% who got drugs got no counseling 20% got no information about side effects 40% of anti-anxiety drugs given > 1 year Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 16 More complex case studies Group Naturally occurring event Basic selection / convenience biases, uncontrollable event. Observation + Archival Controls Dependent Variable(s): Combine survey or other measures with archival or ancillary data as Control variables. Example 1: San Francisco earthquake & coping Sampling frame: - Randomly selected survey participants Outcomes: - Standardized mental health scales - Self-reports of stress Quasi-controls: - population norms on outcomes - ancillary measures, e. g. , social support Findings: Psychology 242, Dr. Mc. Kirnan - High rates of stress Rx, - Social support ‘buffers’ stress Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 17 “One Shot” + archival control: examples Example 2: Stress and immune functioning Event: - Three-mile Island nuclear accident Sampling frame: - Randomly selected residents of geographic area around TMI Outcomes: - Blood draws for immune markers - Self-reports of stress Quasi-controls: - Demographically matched sample - Archival data on health & illness Findings: Psychology 242, Dr. Mc. Kirnan - Long-term suppression of key immune markers (natural killer cells, T cells) Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 18 One shot designs with archival controls, 3 Example 3: Psych. & Health effects of bereavement Event: - Loss of spouse Sampling frame: - Hospital records, self-selected spouses Outcomes: - Blood draws for immune markers - Standardized mental health scales - Occupational functioning Quasi-controls: - Population norms on MH scales - Archival data: occupation & illness Findings: - Long-term immune suppression - Social support ‘buffered’ stress - impact of bereavement > other stressors Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

19 Psychology 242 Introduction to Research One shot designs; Summary “One Shot” designs: no control over independent variable(s), only partial control over measurement: n n There cannot be a control group “Pre-” measures not possible or practical Virtue: n n An experiment is not possible Assess naturally occurring or uncontrollable socially or politically important events Provides “real world” look at processes that are typically studied in experiments: “Effectiveness” v. Efficacy data Archival data can help interpret the findings / “control” some alternate interpretations. Liability: n lack of control group creates multiple threats to internal validity n No pre-measure makes interpretation (e. g. , of change…) difficult. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 20 Quasi-experiments: Existing groups Studying naturally occurring events q Measurement studies q Retrospective designs Evaluate existing groups or program q Single shot survey or measure Non-equivalent groups q q Time series designs Psychology 242, Dr. Mc. Kirnan Research Ethics Back Home page Next

21 One group pre-test — post-test Psychology 242 Introduction to Research Group Observe 1 Selected or convenience sample. Intervention or event Baseline Assessment May or may not have control over measures (e. g. , surveys v. archival measures). Observe 2 Outcome Assessment Typically controllable, but may be archival. Event or intervention May or may not be controllable by researcher, e. g. , policy change. Uses: ü Educational & social environments ü Political or health policy change ü Not feasible to have a control group ü System-wide intervention / social change (school, public health campaign. . ) Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Key design feature: no control group. Psychology 242 Introduction to Research Group Observe 1 Intervention or event Observe 2 Observe 1 Confound Observe 2 22 Threats to internal validity (confounds): ü History Historical / cultural events occur between baseline & follow-up. ü Maturation Individual maturation or growth occurs between baseline & follow-up. ü Reactive measures People respond to being measured or being a measured a second time. ü Statistical regression Extreme scores at baseline “regress” to a more moderate level over time. ü Mortality / drop-out People leave the experiment nonrandomly (i. e. , for reasons that may affect the results…). Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 23 Examples: One group pre- post-, HIV testing Example 1: Effects of HIV testing on sexual risk. Event: - Receipt of HIV testing & counseling Sampling frame: - Participants in testing centers Study structure: - Baseline retrospective interview at testing session - Follow-up interview 3 months later Quasi-controls: - Population characteristics to predict between-group differences Outcomes: - Self-reports of sexual risk Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 24 Example: One group pre- post-, HIV, 2 Effects of HIV testing on sexual risk, cont. Findings: - Significant shifts toward safety - Few demographic predictors of risk or risk change Threats to - Self-selection into testing group internal - Mortality: non-random drop-out(? ) validity - History: general shift in norms & behavior during study time may account for observed change - Instrument change; people may answer more conservatively during a follow-up interview Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 25 Examples: One group pre- post- , Education Example 2: Educational reform & “No Child left behind” testing requirements. Intervention: - Standardized testing becomes integral to educational programs & school evaluation. Sampling frame & - Longitudinal data across multiple Study structure: years in target school grades. - No control group possible. Quasi-controls: - Population characteristics to predict between-group differences Outcomes: - Standardized test scores Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research Example: One group pre- post-, education, 2 26 Education reform & test scores. Findings: n n Modest, statistically significant increase in scores Usual demographic predictors of change; more affluent, better schools. . Internal validity? : n n n Reactive measures; teachers & students do better when measured; (they also cheat; see Houston Miracle article) Instrumentation: kids get better at taking standardized tests, teacher better at teaching them History: General cultural shift n Education more prominent in city n More affluent families sending kids to public schools Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

One group pre- post- designs; Summary 27 Psychology 242 Introduction to Research One group pre- post- test design useful where: n n There cannot be a control group Researchers have control over measurement and the independent variable Virtues: n n n An experiment is not possible provide data on naturally occurring socially or politically important events Pre-measure allows researcher to interpret change & examine status of groups at baseline. Liability: lack of control group creates multiple threats to internal validity: Psychology 242, Dr. Mc. Kirnan ü History ü maturation ü statistical regression ü reactive measures ü mortality / drop-out Week 12 -13, quasi-experimental designs. Back Home page Next

28 Non-equivalent two-group designs Psychology 242 Introduction to Research #1; Static Group Design Group 1 Group 2 (No baseline) Groups are not equivalent at baseline, due to. . ü Self-selection ü Non-random assignment ü Use of existing groups ü Participants not blind Psychology 242, Dr. Mc. Kirnan Intervention or event Observe 1 Contrast group Observe 1 Assessments may or may not be controlled ü Survey or interviews ü Archival / existing data, e. g. , clinic records, grades Intervention or event may or may not be controlled by researcher; ü Existing program ü Experimental intervention ü Naturally occurring event (. . 9/11. . ) Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 29 Non-equivalent designs; pre- post- #2 Two Group Pre- Post- Design Group Observe 1 Intervention or event Observe 2 Group Observe 1 Contrast group Observe 2 Non-equivalent groups ü Self-selection ü Non-random assignment ü Use of existing groups ü Participants not blind Intervention & Assessments often controlled by researcher in these designs. Observation 1 used to ü Assess equivalence of groups at baseline ü Test for threats to internal validity: üReactive measures üHistory, mortality effects üRegression effects Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Similar to true experimental design, except for nonequivalent groups Back Home page Next

Psychology 242 Introduction to Research 30 Examples: Non-equivalent groups, condoms Example Non-equivalent control group design: Effects of condom distribution on sexual safety Intervention: - Condom education & distribution in High School health classes Sampling frame: - Schools in New York & Chicago - Schools matched for SES, race, size Study structure: - NY = intervention schools, Chicago are contrast schools. - Baseline, sexual health programming, end of year Follow-up Outcomes: Psychology 242, Dr. Mc. Kirnan - Clinical measures: STDs - Self-reports: sexual activity & safety Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 31 Examples: Non-equivalent groups, condoms, 2 Condom distribution, cont. Findings: n n n NY (intervention) students; lower STD rate, safer sex NY and Chicago students; similar levels of sexual activity Thus; sexual health classes appeared to increase safety without increasing sexual activity. Internal validity? : n n Reactive measures; Study is not blind; NY students know they are the intervention group Non-equivalent groups: Possible differences between cities = unmeasured confounds Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research Non-equivalent designs 32 Soloman 4 -group design Group 1 Observe 1 Intervention Observe 2 Group 2 Observe 1 Contrast group Observe 2 Group 3 Intervention Observe 2 Group 4 Contrast group Observe 2 Groups 1 & 2: Observation 1 used to ü Assess equivalence of groups at baseline ü Test threats to internal validity Groups 3 & 4: Post-test only tests for reactive effects of assessment ü Compare 1+2 versus 3+4 ü Test interaction of treatment group x pre- post- versus post- only Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

33 Non-equivalent 2 group designs: Summary Psychology 242 Introduction to Research n n Most common quasi-experimental approach. Used where: n n Some form of control or contrast group is possible Groups cannot be equivalent: ü Participants cannot be blind re: group assignment ü Random assignment not possible ü Must use existing or self-selected groups. n Virtue: n Study natural / “real world” interventions Contrast group lessens major threats to internal validity Liability: non-equivalent groups = possible confound. n n Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 34 Quasi-experiments: Existing groups Studying naturally occurring events q Measurement studies q Retrospective designs Evaluate existing groups or program q Single shot survey or measure q Non-equivalent groups q Time series designs Psychology 242, Dr. Mc. Kirnan Research Ethics Back Home page Next

Psychology 242 Introduction to Research Group Interrupted time series design Measure 1 M 2 M 3 M 4 M 5 35 M 6 … Intervention or event n n Test effect of intervention or event on ongoing series of measurements. Intervention may be experimental or observed § Policy shift, e. g. , educational policy § Uncontrolled event; e. g. , 9/11/01, Media event § Assessments may be experimental or archival § Successive cross-sectional surveys § Traffic data, clinic or crime reports, test scores Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research Group Measure 1 M 2 n n M 3 M 4 M 5 M 6 … Intervention or event Multiple baseline n 36 Time series designs Demonstrate highly stable effect long-term crime rates disease prevalence economic performance… Show steady rate of change Hypothesis; tested by: n n Shift in stable rate after intervention Increase / decrease in rate of change after intervention Threats to internal validity: sensitive to very local history n Single group possibly prey to confound n n Advantage for internal validity Eliminates carryover effects of repeated measurement n tests maturation, history, reactive measurement, etc n Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research Example of interrupted time series: Shift 37 in Baboon culture. Core question: Do baboon troops develop and transmit a learned “culture”? Baseline: Long-term observational data on aggressiveness in a specific baboon troop. Intervention: ü Tuberculosis outbreak due to infected food. ü Dominant / aggressive males fed first ü are selectively infected ü are naturally culled from troop ü Naturally occurring event in >20 yr. ongoing field study. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 38 Baboon culture: findings Quasi-controls: Parallel data from other baboon troops. Outcome measures: Standardized indices of aggression & dominance behavior Core finding: ü With dominant males gone, remaining males showed more cooperative behavior ü Enhanced cooperation was transmitted across generation, showing learned “culture”. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 39 Example: Interrupted time series data The “Magic Johnson effect” on HIV testing Data: Archival records of HIV tests reported to CDC, collected monthly n n Data show stable baseline over multiple observations Timing of intervention precise relative to data collection Intervention: Magic reports infection on national TV. n n Uncontrollable, “naturally occurring” event Tests hypothesis re: modeling effects in health behavior Finding: Initial spike in testing rates, followed by leveling off at higher base rate. n n n Initial increase expected Hypothesis tested by longer-term shift in base rate, available due to archival time-series data Effect found for both genders. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 40 Example of time-series data: “Magic” / HIV effect. Time-series data showing shift in HIV testing after Magic’s announcement. Magic’s Announcement Initial spike New, higher base rate Low & variable baserate of testing Multiple (monthly) measures. Tesoriero, J. M. , Sorin, M. D. , Burrows, K. A. , La. Chance-Mc. Cullough, M. L. (1995). Harnessing the heightened public awareness of celebrity HIV disclosures: “Magic” and “Cookie” Johnson and HIV testing. AIDS Education and Prevention, 232 -250. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 41 Multiple time series study Multiple time series data Group 1 Measure 1 M 2 M 3 M 4 M 5 M 6 … Group 2 Measure 1 M 2 M 3 M 4 M 5 M 6 … Groups typically formed by blocking variable measured post-hoc; ü ü ü Health claims in NYC v. other cities post- 9/11/01 Younger v. older voting patterns post- Iraq invasion Intervention or event Hypothesis; tested by interaction of blocking variable by repeated measure: n Is shift in stable rate ( rate of change) greater in one group than another? Heterosexual v. gay HIV testing rates post- Magic Johnson media event. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 42 Blocking variables Testing blocking variables in the HIV testing time-series data. Core questions: n n Both heterosexuals and Ethnic minorities had low HIV testing rates n May feel HIV is not relevant to them – it is a “white gay” problem. n They may lack resources or venues for testing. Will having a prominent African-American Heterosexual disclose HIV+ status may change those perceptions? Hypotheses: n n Heterosexuals will respond more strongly to the Magic Johnson media event than will gay/bisexual men. African-American and Latino men and women will respond most strongly. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 43 Blocking variables: sexual orientation, 1. Testing blocking variables: Gay / IDU data. High base-line and high variability in testing rates among men with risky partners, and IDUs. Gay / bisexual men show less variable, but generally lower baserates. Risky men & IDUs slightly increase, with substantial variability. Gay & bisexual men show no change. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 44 Blocking variables: sexual orientation, 2. Testing blocking variables: Heterosexuals. In contrast to gay / bisexual men or IDUs, heterosexual show an initially low baserate. Followed by a large spike after the announcement And a much higher new baserate. The hypothesis that heterosexuals would be more affected by the “Magic” announcement was supported by the interaction of Time x the blocking variable of sexual orientation. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 45 Blocking variables: ethnic differences Testing blocking variables: Ethnic differences. African-Americans and Hispanics show low baserates and a high spike postannouncement Both groups go back toward their baselines shortly post-announcement. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 46 Blocking variables: ethnic differences, 2. Ethnic differences: White participants. ü HIV testing among Whites was similar to African-Americans & Hispanics at baseline, ü They showed stable, much higher testing rate after Magic’s HIV announcement. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 47 Summary: Blocking variables in time series data A series of measures before & after an event allows us to clearly identify patterns of behavior, and to test group differences (via blocking variables). The hypothesis that ethnic groups would differ was supported by interaction of Time x the blocking variable of ethnicity (but in a direction that was not predicted: Whites showed more change). Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

n n 48 Time series designs: Summary Psychology 242 Introduction to Research Time series is most common with archival data: existing, standard records collected for other purposes. Used where: n The hypothesis concerns changes in long-term trends n Typically an experiment cannot be run ü Simple practicality or cost, e. g. , health care issues ü Ethics; crime rates, rates of domestic violence, etc. ü The target events are not controllable. n Virtue: n n n Study natural / “real world” processes or interventions Blocking variables – comparing time trends across groups -- lessens major threats to internal validity Liability: lack of control = possible confound. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 49 Quick quiz Researchers often use _____ to help interpret “single shot” surveys A = paradigm change B = measurement studies C = experimental controls D = retrospective measures Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 50 Quick quiz, 2 Match: A = Mortality / drop-out B = Maturation Individual maturation or growth occurs between baseline & follow-up. C = History People respond to being measured or being a measured a second time. D = Statistical regression Extreme scores at baseline “regress” to a more moderate level over time. E = Reactive measures Psychology 242, Dr. Mc. Kirnan Historical / cultural events occur between baseline & follow-up. People leave the experiment nonrandomly (i. e. , for reasons that may affect the results…). Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 51 Quick quiz, 2 Match: A = Mortality / drop-out B = Maturation C = History People respond to being measured or being a measured a second time. D = Statistical regression E = Reactive measures Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 52 Quick quiz, 3 Match: A = Mortality / drop-out B = Maturation Growth or natural change between baseline & follow-up. C = History D = Statistical regression E = Reactive measures Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research 53 Quick quiz, 4 Match: A = Mortality / drop-out B = Maturation C = History D = Statistical regression People leave the experiment nonrandomly (i. e. , for reasons that may affect the results…). E = Reactive measures Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Psychology 242 Introduction to Research Group 54 Quick quiz 5 Measure 1 M 2 M 3 M 4 M 5 M 6 … Intervention or event This is called a: A = Threat to internal validity B = Manipulation check C = Multiple baseline D =. . lot of work. Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

Quasi-experiments; Summary 55 Psychology 242 Introduction to Research 1. Study naturally occurring events that could not be brought into a lab or a true experiment. n Measurement studies n Retrospective designs 2. Evaluate existing groups or program(s) n Single shot survey or measure of an intervention n n Non-equivalent / pre-existing groups n n With or without control variables Static group or 2 group pre- post- design Time series designs, often with archival data ü Trade off internal for external validity Psychology 242, Dr. Mc. Kirnan Week 12 -13, quasi-experimental designs. Back Home page Next

56 Psychology 242 Introduction to Research THANK YOU !! Back Home page Next