Foundations of Research 1 Research ethics Research Ethics

  • Slides: 76
Download presentation
Foundations of Research 1 Research ethics Research Ethics: q The Common Rule q The

Foundations of Research 1 Research ethics Research Ethics: q The Common Rule q The Belmont Report This is a Power. Point Show ü Click “slide show” to start it. ü Click through it by pressing any key. ü Focus & think about each point; do not just passively click. ü To print: ü Click “File” then “Print…”. ü Under “print what” click “handouts (6 slides per page)”. © Dr. David J. Mc. Kirnan, 2014 The University of Illinois Chicago Mc. Kirnan. UIC@gmail. com Do not use or reproduce without permission Recruitment flyer for Stanly Milgram’s study of obedience. From Boingboing. net

Foundations of Research The “Common Rule” criteria for Human Subjects Protection The Common Rule

Foundations of Research The “Common Rule” criteria for Human Subjects Protection The Common Rule Research institutions are mandated to have boards – Called Institutional Review Boards – that provide ethical review of all Federally funded research. That review follows seven elements, called the Common Rule. Most Universities apply the same standard to all research, whether federally funded or not. 2

Foundations of Research The Common Rule n Minimize risks n Risks must be reasonable

Foundations of Research The Common Rule n Minimize risks n Risks must be reasonable n Recruit participants equitably n Informed consent n Document consent n Monitor for safety n Protect vulnerable participants & maintain confidentiality 3

Foundations of Research Minimization of risks 4 n n Core issue: Risks should not

Foundations of Research Minimization of risks 4 n n Core issue: Risks should not exceed those of everyday behavior. n Potential harms from research: n n Simple inconvenience, loss of time, money… Withholding care or treatment § Use of deception in experimental manipulation n Physical harm n Psychological harm § Social or political harm n n n The Common Rule Minimize risks Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality

Foundations of Research 5 Potential harms from research n n Potential harms: n n

Foundations of Research 5 Potential harms from research n n Potential harms: n n Simple inconvenience: n n n Any research study, even an innocuous survey, has some “costs” to the participant. The Common Rule Minimize risks Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality Time, distraction, possible $ for transportation… All research must be justified by it’s potential contribution to science or society. Withholding care Direct: Indirect: Tuskegee-like Clinical trials, wait-list designs § Use of deception in experimental manipulation ü Can people provide informed consent if they are deceived? ü Possible embarrassment or negative shift in self-perception. ü Deception erodes trust & confidence in social science. ü Placebo arm in treatment trial

Foundations of Research 6 Potential harms from research n n Potential harms: n Simple

Foundations of Research 6 Potential harms from research n n Potential harms: n Simple inconvenience n n n Withholding care The Common Rule Minimize risks Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality ü Direct; • The Tuskegee trial was a glaring example of the intentional or direct harmful withholding of care. • When a known effective treatment was discovered it was withheld specifically to examine disease progression among untreated men. ü Indirect; • In clinical trials (of a new drug, behavioral treatment…) the control group does not receive the intervention. o o • Placeb control; a spurious or emty treatment. Wait-list control; intervention is phased in only after the experimental group has finished the treatment process. These approaches are considered ethical when: o o The informed consent makes the trial structure clear; It is unknown whether the experimental treatment “works”.

Foundations of Research 7 Potential harms from research n n n Simple inconvenience n

Foundations of Research 7 Potential harms from research n n n Simple inconvenience n Withholding care n Physical harm n n n The Common Rule Minimize risks Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality ü The history of behavioral and biomedical research contains many instances of intentional or unintentional physical harm. (Wikipedia has a good review here. ) • • Direct or intentional harm; o Participants are intentionally exposed to (potentially) toxic or harmful drugs or physical circumstances. o The goal of the research is to examine the human effects of noxious conditions. Indirect or unintentional harm; o Participants are exposed to conditions that may be harmful. o The harmful exposure is not a goal of the research, but nonetheless has a high probability of occuring.

Foundations of Research 8 Potential harms from research n n n Physical harm n

Foundations of Research 8 Potential harms from research n n n Physical harm n n Direct / intentional harms: ü Cold War radiation experiments, During the 1950 s – 60 s, in preparation for a potential nuclear war, the U. S. military exposed hundreds of service men and women to varying doses of radiation from nuclear tests. Soldiers (and civilians) were tested for radiation exposure effects, both to assess harms and to convince the public that nuclear war could be “safe” to civilian populations. n n n The Common Rule Minimize risks Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality Click for archival footage on You. Tube Soldiers being exposed to a nuclear explosion at the Nevada Test Site in 1951. http: //www. teoti. com/photography/123738 -historicalpictures. html Image: imgur. com, public domain

Foundations of Research 9 Potential harms from research n n Physical harm n n

Foundations of Research 9 Potential harms from research n n Physical harm n n Direct Harms: ü Cold War radiation experiments, ü Army psychoactive drug research n n The Common Rule Minimize risks Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality The U. S. army performed a variety of harmful drug studies before ethical standards were firmed in during the 1990 s. In the 1960 s through the 70 s service members and, in some cases, prisoners were given varying doses of LSD and other psychotropic drugs to test their possible use as psychoactive chemical weapons. Most of these exposures were with participants’ foreknowledge and consent. At other times drugs were slipped into food to test their effects when unanticipated. Click below for an excellent piece in The New Yorker.

Foundations of Research 10 Potential harms from research n n Physical harm n n

Foundations of Research 10 Potential harms from research n n Physical harm n n Direct Harms: ü Cold War radiation experiments, ü Army psychoactive drug research, ü Drug trials in prison populations n n The Common Rule Minimize risks Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality The great majority of psychoactive drugs used today – for sleep disorder, anxiety or depression, psychotic symptoms… - were initially tested on prisoners. Other testing administered known or potential toxins to prisoners to gauge their effects. Pharmaceutical companies would work with State officials to gain access to prisoners. Prisoners “volunteered” for the studies, although were rarely fully informed about the agents they were taking. The shameful history of this testing led to very strict regulations in the 1990 s. Today there is renewed debate about the prospect of prisoner testing. Click the image for a NYTimes. com article.

Foundations of Research 11 Potential harms from research n n Physical harm n n

Foundations of Research 11 Potential harms from research n n Physical harm n n Direct Harms: ü ü Cold War radiation experiments, Army psychoactive drug research, Drug trials in prison populations, Some forms of Animal research n n The Common Rule Minimize risks Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality Animal research has long been contentious for both research and commercial safety. As with prisoner research, animal testing has an ugly history, but has made invaluable contributions to human health. PETA, of course, opposes any animal testing (click image). NIH and organizations such as Foundation for Biomedical Research argue in Defense of responsible animal testing. Scientific American calls for ban on animal testing for Cosmetics. Many companies advertise their products as animal-testing free, although many use ingredients shown to be safe through prior animal work. The Scientist has an excellent series on animal testing, including the increasing number of alternatives to animals. Although the U. S. Food and Drug Administration does not require animal safety testing for cosmetics—a category that includes skin cream, perfume, makeup and shampoo— animal tests are still used. Credit: Thinkstock

Foundations of Research 12 Potential harms from research n The Common Rule Minimize risks

Foundations of Research 12 Potential harms from research n The Common Rule Minimize risks n n n For more perspectives on Animal research click the images. n Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality Second Thoughts of an Animal Researcher Is Animal Research Ever Ethical? Rebuttals and comments on Gluck JOHN P. GLUCK, New York Times, SEPT. 2, 2016 New York Times, SEPT. 12, 2016 University of California, Berkeley, 1967. Credit Henri Cartier. Bresson/Magnum Photos A University of Wisconsin-Madison researcher with a pregnant monkey infected with the Zika virus. Credit Scott Olson/Getty Images

Foundations of Research 13 Potential harms from research n n Physical harm n n

Foundations of Research 13 Potential harms from research n n Physical harm n n Direct Harms: Indirect harms: ü ü Cold War radiation experiments, Army psychoactive drug research, Drug trials in prison populations, Some forms of Animal research n n The Common Rule Minimize risks Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality ü Behavior induced by the experiment: Ø Experimental conditions that encourage risk, alcohol & drug use, smoking… Many studies on health behavior assess the effects of different conditions – say, modeling by attractive peers – on behaviors such as alcohol or tobacco use. These studies can be ethical if participants are carefully screened (e. g. , for no evidence of alcohol abuse…) and monitored during the experiment. Ethical constraints must be very strict in this area, particularly if deception is involved. (For example, in the “Balanced Placebo” design for examining alcohol-related behavior participants in one cell of the design expect to not receive an alcoholic beverage, but actually do. )

Foundations of Research 14 Potential harms from research n n n Simple inconvenience Withholding

Foundations of Research 14 Potential harms from research n n n Simple inconvenience Withholding care Physical harm n n n Psychological harm ü Discomfort or pain The Common Rule Minimize risks Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality Experiments that induce anxiety or negative moods. Many studies on issues such as emotions and alcohol use, coping with stress or change, or adjusting to novel environments can temporarily induce negative reactions or moods. These studies remain ethical given that they have carful: ü Screening, to eliminate highly vulnerable participants; ü Monitoring, to assess participants’ state during the study; ü Debriefing, to provide information and provide a chance to return to a normal state. (The same is true when administering alcohol…).

Foundations of Research 15 Potential harms from research n The Common Rule Minimize risks

Foundations of Research 15 Potential harms from research n The Common Rule Minimize risks n n Psychological harm ü Discomfort or pain ü Changed selfperception n Experiments that induce anxiety or negative n moods. n E. g. , leading participants to believe they were capable of harming others Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality One of the most famous research programs crossing this ethical line were the obedience studies conducted by Stanley Milgram in the 1960 s. Images: Simplypsychology. org/ Click image for an excellent overview from Simply. Psychology.

Foundations of Research Potential harms from research Research harms; changes in self perception. •

Foundations of Research Potential harms from research Research harms; changes in self perception. • In 1962, in the wake of the Nuremberg Trials of Nazi war crimes, Stanley Milgram wanted to discover how common citizens could be led to commit clearly evil acts. • He recruited people for a “Memory study”, where they thought they were serving as instructors or research assistants. • Milgram hired actors to portray the ostensible “Learners”. • Milgram thus used deception in his recruitment materials and study instructions. • Any deception study is ethically problematic because participants are not providing accurate informed consent. • Studies that use deception to hide hypotheses from participants typically have no other potential study harms. • Milgrams’ use of deception was unethical; Participants consented not knowing they would be subjected to substantial stress. 16

Foundations of Research Potential harms from research 17 Research harms; changes in self perception.

Foundations of Research Potential harms from research 17 Research harms; changes in self perception. • The “learner” was attached to (phony) electrodes, in a separate room from the “instructor”, who sat with the experimenter. • Each time the “learner” (an actor…) missed an item on a memory task, the “instructor”(the actual participant) was told to administer increasingly intense shocks. • The study assessed how high a shock participants would administer as the “learner” consistently gave wrong answer. • Or did they? A recent re-re-analysis suggests that participants were that disturbed thethought study. Many participants were obedient enough to take thenot shock to levels by they were lethal, while the “student” showed increasingly extreme discomfort. . • Bottom line: many ordinary people will harm others when commanded to by an authority. • Of course, participants then had to live with the knowledge that they were willing to follow instructions to that extreme. Click for Dar Williams’ song about the Milgram experiment and fascism in Indy Magazine.

Foundations of Research 18 Potential harms from research n n Research harms; changes in

Foundations of Research 18 Potential harms from research n n Research harms; changes in self perception. n n The Stanford Prison Study n The Common Rule Minimize risks Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality Click for the study web In 1971 the U. S. had been subject to several tragic prison riots. Political debate focused on site. whether criminals – the great majority Black and poor – were cognitively or psychologically “impaired” to the extent that they could not function in society. In contrast to “individual pathology” perspective, sociological research at the time described the effects of “total institutions” such as prisons or psychiatric hospitals, that themselves created deviant behavior. Phillip Zimbardo set out to test this experimentally, by constructing a realistic prison in the Stanford Psychology Dept. basement, and randomly assigning male undergraduates to the roles of prisoners or guards. Consistent with the hypothesis, the students – all affluent, white, well educated and screened for psychological health – acted little different than those at Attica State Prison, where riots had taken place. As with the Milgram experiment, participants were left with the knowledge that, whether prisoner or guard, they were capable arbitrary brutality.

Foundations of Research 19 Potential harms from research n The Common Rule Minimize risks

Foundations of Research 19 Potential harms from research n The Common Rule Minimize risks n n Psychological harm ü Discomfort or pain ü Changed selfperception ü Embarrassment n n n Experiments that induce anxiety or negative n moods. n E. g. , leading participants to believe they were capable of harming others Reasonable risks Equitable recruitment Informed consent Document consent Data monitoring Protect vulnerable participants, maintain confidentiality Loss of confidentiality or privacy Many studies address coping or risk behavior in populations that may be stigmatized. For example, the author conducted AIDS research among gay and bisexual men from the 1980’s forward. During much of that time being “outed” as gay/bisexual could lead to harassment, job loss, or actual violence. As more men (and women) became infected with HIV, that status became a second important source of stigma. Loss of confidentiality through sloppy record keeping, or even recognizing a research participant on the street, could have real adverse consequences. Research with potentially stigmatized people – whether due to poverty, criminal background, abuse history, or otherwise – requires very careful protections to ensure that vulnerable participants are not socially harmed just by being identified as a research participant.

Foundations of Research n n n 20 Potential harms from research Simple inconvenience Withholding

Foundations of Research n n n 20 Potential harms from research Simple inconvenience Withholding care Physical harm Psychological Harm Social or political harm The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Data monitoring for safety n Protect vulnerable participants & maintain confidentiality ü Portraying social groups in a negative light, e. g. : • Interpretations of research results that suggest lower intelligence among lower socio-economic status or minority participants; • “Pathologizing” minority groups by problem-focused research and reporting: o Research funding addressing poor, GLBTQ, minority and other populations typically focuses on problems: drug abuse, domestic problems, criminality…; o Although these are important topics, the consistent portrayal of a community in ‘problem’ terms can enhance negative stereotypes. ü Ignoring or relying too strongly on some groups: e. g. , women or minorities in clinical research.

Foundations of Research 21 Potential harms from research What makes research vulnerable to causing

Foundations of Research 21 Potential harms from research What makes research vulnerable to causing harm? n Strong financial pressure for study results: n n n Advancement in the research industry - promotions, recognition, getting grants or contracts ($) – requires that investigators consistently publish positive results. Simple bias or prejudice: n n n This creates substantial monetary pressure to produce positive results, potentially making ethical considerations secondary. Publication & grant pressure: n n The ability to market clinical interventions – new drugs, medical devices, behavioral programs – depends upon positive results of efficacy trials. Most researchers are Caucasian, with middle or upper-middle class backgrounds; The “Ivory Tower” syndrome; it is common for researchers have little direct experience with the populations they are studying. Lack of institutional controls n Active Institutional Review Board n Study monitors Up-front ethical reviews & ongoing monitoring for ethical compliance can prevent potential harms.

Foundations of Research 22 Potential harms from research The Common Rule Prevention of research

Foundations of Research 22 Potential harms from research The Common Rule Prevention of research harms: § Independent & rigorous Institutional Review Board [IRB]: • An IRB is a set of active researchers, who review and monitor proposals for their adherence to the common rule. § Diversity among investigators, research centers. § Study monitoring: • n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Data monitoring for safety n Protect vulnerable participants & maintain confidentiality We will see examples of good and poor monitoring below. § Careful pilot testing & monitoring of study manipulations § Informed consent and debriefing to: a) fully inform participants about the study b) eliminate any imposed state, e. g. temporary stress, negative affect… NIH. gov, public domain

Foundations of Research Reasonableness of risks Core issue: n Whether a risk to participants

Foundations of Research Reasonableness of risks Core issue: n Whether a risk to participants is reasonable rests on a Cost – benefit analysis n 23 Does the study present risks greater than every-day life? The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Data monitoring for safety n Protect vulnerable participants & maintain confidentiality • How much greater, for how long? n Are the potential harms – ‘costs’ – justified by the likely knowledge to be gained, i. e. , potential benefits? § Potentially harmful research may be justified if it provides invaluable data, e. g. : n Milgram obedience studies; n Zimbardo prison experiment; n Intrusive animal studies; n Studies that pay people to take medical or behavioral risks.

Foundations of Research 24 Reasonableness of risks § Research with little harm may still

Foundations of Research 24 Reasonableness of risks § Research with little harm may still be unjustified if it will not provide useful data: n n Research always requires time, effort, potential embarrassment…; Research that is trivial or incompetent may be inherently unethical for those reasons. § Weighing risks against potential benefits is difficult and complex… § We lack a scientific metric for evaluating degrees of potential harm. The Zimbardo Prison Study subjected college students to real stress… § The benefits of research are rarely guaranteed. …but provided valuable data about how social roles and the physical setting affect behavior. § Even research with scientific or applied benefit Click the image for an overview of the study. often has no direct benefit for the study participants themselves. © 1999 -2015, Philip G. Zimbardo, http: //www. prisonexp. org/the-story/

Foundations of Research n n The Common Rule n Minimize risks n Reasonable risks

Foundations of Research n n The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment The “Costs” of participation may not be fully understood by investigator; n Informed consent n Document consent n Data monitoring for safety Benefits may be overstated or not framed in terms of target population; n Protect vulnerable participants & maintain confidentiality Causes of unreasonable research risk: n 25 Reasonableness of risk Publication & grant pressure to recruit participants at any cost. Prevention: § Independent & rigorous Institutional Review Board [IRB]; § Community Advisory Board [CAB]; A board of people from the target population; § Pilot testing to assess the actual risks of the research. Shutter. Stock

Foundations of Research Participant Recruitment Equitable recruitment: n 26 People potentially affected by research

Foundations of Research Participant Recruitment Equitable recruitment: n 26 People potentially affected by research must: P Have the opportunity to join the study; The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Data monitoring for safety n Protect vulnerable participants & maintain confidentiality P Have the opportunity to withdraw once enrolled; P Not be coerced or deceived into enrollment. § How do we ensure that all social groups are represented? ü Some groups are less likely to enlist than are others; Ø How much should Investigators try to overcome peoples’ reluctance to join a study? Ø At what point does that become coercive? ü Some groups – drug abusers, those in poverty – will respond to even a small monetary incentive to join even high risk studies. Ø Is offering a monetary incentive to those individuals coercive? Ø Are they treated inequitably by not being offered money?

Foundations of Research Potential problems in recruitment: n Arbitrary bias in sampling n 27

Foundations of Research Potential problems in recruitment: n Arbitrary bias in sampling n 27 Participant Recruitment Excluding – or only including – groups for reasons unrelated to the research protocol: o Convenience or ease of recruitment, such as students, prisoners; The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Data monitoring for safety n Protect vulnerable participants & maintain confidentiality o Simple bias, such as toward men in medical research. n Using highly unrepresentative samples: o E. g. , over-reliance on students or “for hire” internet lists in social / behavioral research; o Over-reliance on poor people (who strongly respond to financial incentives) in medical or behavioral research.

Foundations of Research Potential problems in recruitment: n Arbitrary bias in sampling n n

Foundations of Research Potential problems in recruitment: n Arbitrary bias in sampling n n n 28 Participant Recruitment Excluding – or only including – groups for reasons unrelated to the research protocol. The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Data monitoring for safety n Protect vulnerable participants & maintain confidentiality Using highly unrepresentative samples: Coercive payments or incentives n n Culturally or socio-economically coercive: n Financial or similar incentives (e. g. , burial insurance during Tuskegee). n Provision of otherwise unavailable human or health services. Potential loss of benefits: n n n Recruitment by one’s own physician, teacher, supervisor… These create a dual role situation where the recruiter is also in a position of power or influence. Although there may not be direct coercion, the individual may fear a loss of good favor.

Foundations of Research Potential problems in recruitment: n Arbitrary bias in sampling n n

Foundations of Research Potential problems in recruitment: n Arbitrary bias in sampling n n 29 Participant Recruitment Excluding – or only including – groups for reasons unrelated to the research protocol. Using highly unrepresentative samples. The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Data monitoring for safety n Protect vulnerable participants & maintain confidentiality Coercive payments or incentives n Culturally or socio-economically coercive. n Potential loss of benefits. Deceptive descriptions of the study; n Simple misinformation about potential harms or potential benefits. n Therapeutic misconception: n n n Potential participants may not fully understand that they are enrolling in a research trial, and that the treatment is still unproven. Many participants (particularly with lower education) assume they are being enrolled to receive treatment, not to test a treatment. The informed consent must make this clear, and investigators have an ethical responsibility to ensure participants understand what a trial is.

Foundations of Research Potential problems in recruitment: n Arbitrary bias in sampling n n

Foundations of Research Potential problems in recruitment: n Arbitrary bias in sampling n n n 30 Participant Recruitment Excluding – or only including – groups for reasons unrelated to the research protocol. Using highly unrepresentative samples. The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Data monitoring for safety n Protect vulnerable participants & maintain confidentiality Coercive payments or incentives n Culturally or socio-economically coercive. n Potential loss of benefits. Deceptive descriptions of the study; n Simple misinformation about potential harms or potential benefits. n Therapeutic misconception. Ability to comprehend the research & provide informed consent; n Children, elderly, developmentally delayed, mentally ill … require extra precautions to ensure they are capable of consenting to participate. o Often this will consist of a person (i. e. , a relative, someone with medical power of attorney) who can consent on behalf of the participant. o This is particularly important for complex or long-term research protocols.

Foundations of Research Potential problems in recruitment: n Arbitrary bias in sampling n n

Foundations of Research Potential problems in recruitment: n Arbitrary bias in sampling n n n 31 Participant Recruitment Excluding – or only including – groups for reasons unrelated to the research protocol. Using highly unrepresentative samples. Preven The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Data monitoring for safety n Protect vulnerable participants & maintain confidentiality tion: or incentives Coercive payments n Lo cal socio-economically RB n Culturally or I represe & Funder’s coercive. require ntation n Potential loss of benefits. ments. n C for equ lear & c itable o mpreheof the study; docudescriptions Deceptive n sive in ment. formed coor n n Simple misinformation about potential harms benefits. nspotential Study ent monito und g to en erstand rin n Therapeutic misconception sure pa. the res rticipan earch p ts informed continu consent; Ability to comprehend the research roto& coprovide e to l & con s e n t. require extra precautions n Children, elderly, developmentally delayed, mentally ill … to ensure they are capable of consenting to participate. o Often this will consist of a person (i. e. , a relative, someone with medical power of attorney) who can consent on behalf of the participant. o This is particularly important for complex or long-term research protocols.

Foundations of Research 32 Informed consent ü Key elements of the Informed Consent document:

Foundations of Research 32 Informed consent ü Key elements of the Informed Consent document: § Purpose & procedures of the study. § Why the participant was recruited. § Study requirements and duration. The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Data monitoring for safety n Protect vulnerable participants & maintain confidentiality § Possible risks or harms, § The study is voluntary & the participant can withdraw at any time. § Any potential benefits or costs of participation. § Who to contact for information / concerns, including the IRB. § Written signature. ü How do we know the participant understood? § Administer consent quiz, or personal interview. ü How to document consent? § Written signature (with the name kept confidential). § For studies where participants are anonymous the IRB can waive written consent.

Foundations of Research 33 The Common Rule Informed consent Deception in experiments: ü How

Foundations of Research 33 The Common Rule Informed consent Deception in experiments: ü How do we provide informed consent if the participant cannot know the hypothesis? § The study must present no risks of harm; § The participant must be thoroughly debriefed after the study. n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Data monitoring for safety n Protect vulnerable participants & maintain confidentiality ü Deception can erode trust & confidence in social science. The consent document is one of the most closely examined issued in IRB ethical reviews.

Foundations of Research n Data and clinical trial monitoring Two key elements of Research

Foundations of Research n Data and clinical trial monitoring Two key elements of Research Monitoring: n Monitoring of clinical trials; n Studies of a new drug or treatment. n Behavioral intervention studies. n n 34 The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Monitor for safety n Protect vulnerable participants & maintain confidentiality Participants are followed over time with multiple study visits. Data Safety Monitoring Boards; n Independent bodies that oversee data collection and analyses.

Foundations of Research 35 Data and clinical trial monitoring Monitoring for safety: § Trial

Foundations of Research 35 Data and clinical trial monitoring Monitoring for safety: § Trial monitoring § Data Safety Monitoring Board The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Monitor for safety n n Investigators in clinical trials are required to monitor and Protect vulnerable participants & maintain confidentiality report any health or safety “adverse events”. n n Trial related – due to a feature of the trial protocol: n E. g. , heart complications during trials of weight loss drugs; n Can be deemphasized or ignored in trials testing products; n Often ignored in behavioral intervention studies. Non-Trial related; n Deaths are common during longitudinal studies of serious health problems: n n n Cancer, drug or alcohol abuse research. Other events may simply stem from everyday life, e. g. , auto accidents. of injection drug users Trial-related Serious Adverse Events may require a protocol change or may stop the study.

Foundations of Research n n 1992; Physicians begin prescribing a combination of Fenflouramine and

Foundations of Research n n 1992; Physicians begin prescribing a combination of Fenflouramine and Phentermine (“Fen-Phen”) for obesity, with no FDA approval. 1996; After clinical trials by the manufacturer the FDA approves Redux, a Fen-Phen drug. n n n Fen-Phen; a case study of failed trial Monitoring The manufacturer reported 4 cases of severe cardiac effects during trial monitoring, despite 41 having actually occurred. 36 The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Monitor for safety n Protect vulnerable participants & maintain confidentiality The Food and Drug Administration bypassed staff who had concerns, and approved the drug without a “black box” warning. n Pharmaceutical Companies can have substantial influence in FDA decisions n Wyeth spends $52 million promoting the drug, garnering $300 million in annual sales. n Time Magazine notes Fen-Phen as the hot new diet drug, but raises safety questions. 1997; 30 year-old woman dies of cardiac event after taking Fen-Phen for a month. n n n 1998; Mayo clinic finds multiple cases of cardiac problems in women taking Fen-Phen FDA receives 144 Adverse Event reports; 30% of patients taking Fen. Phen show cardiac abnormalities. Fen-Phen pulled from market. 2003; Forbes Magazine reports 153, 000 law suits against Wyeth, who pays out $13 billion in settlements. Click the image for the PBS Frontline documentary Fen Phen Nation

Foundations of Research n n Fen-Phen; a case study of failed trial Monitoring 1992;

Foundations of Research n n Fen-Phen; a case study of failed trial Monitoring 1992; Physicians begin prescribing a combination of Fenflouramine and Phentermine (“Fen-Phen”) for obesity, with no FDA approval. 1996; After clinical trials by the manufacturer the FDA approves Redux, a Fen-Phen drug. n The manufacturer reported 4 cases of severe cardiac effects during trial monitoring, despite 41 having actually Here a large company hadoccurred. a strong motive ü n 37 The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Monitor for safety n Protect vulnerable participants & maintain confidentiality for The Food and Drug monitoring; Administration bypassed whoare had a concerns, inadequate diet staff pills multiand approved the drug without a “black box” warning. billion dollar a year industry. Large Pharmaceutical Companies can have substantial influence in FDA n decisions n n n üWyeth Even though the Food and Drug spends $52 million promoting the drug, garnering $300 million in annual sales. Administration [FDA] was monitoring the trial, Time Magazine notes Fen-Phen as the hot new diet drug, but raises safety questions. the company intentionally data that 1997; 30 year-old woman dies of cardiac eventwithheld after taking Fen-Phen for a month. wouldn have stopped 1998 ; Mayo clinicapproval. finds multiple cases of cardiac problems in women taking Fen-Phen ü This is also example of industry research n FDA receives 144 Adverse Event reports; 30% of patients taking Fen- having Phen a corrupting on FDA decisions. show cardiac effect abnormalities. Fen-Phen pulled from market. n 2003; Forbes Magazine reports 153, 000 law suits against Wyeth, who pays out $13 billion in settlements. Click the image for the PBS Frontline documentary Fen Phen Nation

Foundations of Research 38 Data and clinical trial monitoring Monitoring for safety: § Trial

Foundations of Research 38 Data and clinical trial monitoring Monitoring for safety: § Trial monitoring § Data Safety Monitoring Board The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Monitor for safety n n Protect vulnerable participants & maintain confidentiality The DSMB monitors: n Trial integrity; is the research protocol being followed correctly. n “Stopping rules” for research risks or positive findings n Data integrity: n Ensures that data are collected in a valid fashion n Guards the data against “unblinding” of participants or investigators; n n n A trial is “double blind” when neither the participant nor the researcher know what group (experimental or control) the participant is in. This eliminates possible confounds stemming from expectations participants / researchers may have. The DSMB is entrusted with all the codes for experimental groups and “unblinds” participants and investigators only when the trial is over.

Foundations of Research n n 39 The Women’s Health Initiative; a case study of

Foundations of Research n n 39 The Women’s Health Initiative; a case study of “stopping rules” for a clinical trial. 1980 s-90 s: Millions of women use Hormone Replacement Therapy of estrogen plus progestin (E+P) to relive menopausal symptoms. 1991: NIH begins a study after observational data suggest that women using hormones have lower rates of heart disease. The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Monitor for safety n Protect vulnerable participants & maintain confidentiality 2002: The E+P part of the Initiative is stopped early after women show higher rates of heart attack, stroke and breast cancer. Millions of women abandon hormones overnight. § An ongoing trial examines women taking estrogen only (without progestin). § 2004: The estrogen only study is stopped one year early: participants show fewer breast cancers and only small increased risk of stroke.

Foundations of Research n n The Women’s Health Initiative; a case study of “stopping

Foundations of Research n n The Women’s Health Initiative; a case study of “stopping rules” for a clinical trial. 1980 s-90 s: Millions of women use Hormone Replacement Therapy of estrogen plus progestin (E+P) to relive menopausal symptoms. 1991: NIH begins a study after observational data suggest that women using hormones have lower rates of heart disease. Here is an e ways: n The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Monitor for safety n Protect vulnerable participants & maintain confidentiality f stoppin g rulesafter wowomen 2002: The E+P part of the Initiative is stopped early higher rking bshow ü The e o strogstroke th en & prand rates of heart attack, ogesbreast cancer. detected n xample o 40 adverse tin arm w a rom the he estro the DSM therapy. gen only B arm was strong p stopped ositive r because e thera s u l it showe t s t h py. at all pa d such rticipant s were a ll put on s stoppe d becau e f f e c ts f overnight. Millionsüof. Twomen abandon hormones se § An ongoing trial examines women taking estrogen only (without progestin). § 2004: The estrogen only study is stopped one year early: participants show fewer breast cancers and only small increased risk of stroke.

Foundations of Research How do we separate self-interest & political pressure from science? n

Foundations of Research How do we separate self-interest & political pressure from science? n n n 41 Thought questions: Monitoring for safety How much does military or corporate (pharmaceuticals, tobacco. . ) funding for research distort scientific findings? The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Monitor for safety n Protect vulnerable participants & maintain confidentiality How sensitive should scientists be to political or social pressures around their research, e. g. , n sexual behavior n climate change n stem cells n evolution n gun risks n economics & social policy How responsible are scientists for the social impact of their findings? Ø E. g. , negative portrayal of social “out-groups” Ø The use of empirically validated techniques for unethical practices

Foundations of Research 42 Vulnerability to coercion in research What is coercion in research?

Foundations of Research 42 Vulnerability to coercion in research What is coercion in research? Enrollment: § Joining a study that a reasonable person would see as harmful or exploitive. Continued participation: The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Monitor for safety n Protect vulnerable participants & confidentiality § Not recognizing harm or exploitation that emerges once the research begins; § Recognizing harms but not having the psychological or physical capacity to withdraw. What makes someone vulnerable to coercion in research? n Cognitive: Low capacity to think about and provide informed consent for participation: n Children, older adults; n Dementia or cognitive limitations, mentally ill, drug users.

Foundations of Research 43 Vulnerability to coercion in research What makes someone vulnerable to

Foundations of Research 43 Vulnerability to coercion in research What makes someone vulnerable to coercion in research? n Cognitive n Authority: liable to authorities who have a vested The Common Rule n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Monitor for safety n interest in your participation. n Prisoners: • n Protect vulnerable participants & confidentiality Even a small financial reward or the possibility or earlier parole can be extremely coercive. Medical patients, students and others with a Dual Role: • Patients / students may feel obligated to participate in a study conducted by their physician / Professor.

Foundations of Research 44 Vulnerability to coercion in research The Common Rule What makes

Foundations of Research 44 Vulnerability to coercion in research The Common Rule What makes someone vulnerable to coercion in research? n Cognitive n Authority n Deferential: participation due to deferential attitudes or n Minimize risks n Reasonable risks n Equitable recruitment n Informed consent n Document consent n Monitor for safety n Protect vulnerable participants & confidentiality cultural pressure rather than actual willingness. § In the 1980 s – 90 s the international gay community was deeply threated by HIV/AIDS. § Men were very willing to join clinical (e. g. , HIV vaccine) trials § n • From social responsibility, to help stop AIDS • From the Therapeutic Misconception that they would directly benefit. This deferential attitude could create vulnerability to participating in high risk research (although there is not evidence that happened). Medical: selected due to serious health-related condition for which there are no satisfactory remedies. n Poor / disadvantaged: lacking important social goods – money or health care – provided via research participation. See discussion above

Foundations of Research The Common Rule n Minimize risks n Risks must be reasonable

Foundations of Research The Common Rule n Minimize risks n Risks must be reasonable n Recruit participants equitably n Informed consent n Document consent n Monitor for safety n Protect vulnerable participants & maintain confidentiality 45

Foundations of Research The Common Rule § The common rule protects participants during the

Foundations of Research The Common Rule § The common rule protects participants during the process of research. n Minimize risks n Risks must be reasonable § Researchers also have n responsibility Recruit participants equitably for the use of their n Informed consent ü We discussed the problems with a negative n portrayal of research populations when Document consent n Monitor for safety ü The next case study shows the extreme of n results are published. use of research results. Protect vulnerableirresponsible participants & maintain confidentiality 46

Foundations of Research A case study of the unethical use of scientifically validated techniques.

Foundations of Research A case study of the unethical use of scientifically validated techniques. 47 The American Psychological Association [APA] actively contributed to the CIA program of “enhanced interrogation” (torture) until 2008. n n In 2014 the Senate released a report describing CIA “enhanced interrogations” during the Iraq war. They concluded that the CIA engaged in torture, and that the program was unsuccessful. n It did not contribute useful intelligence; n Therefore it had no justification. It had been widely known that APA-sanctioned Psychologists had helped design and administer the interrogation program. In 2015 an ethics panel of Psychologists and human rights activists issued a report on the deep involvement of the APA in the Bush era torture program. Click for NYTimes. com overview. An American soldier patrolling outside Abu Ghraib prison in 2005. The public disclosure of images of prisoners being abused there prompted debate about the way the United States was treating detainees. Credit John Moore/Getty Images, New York Times.

Foundations of Research n n 48 Case study of unethical use of scientifically validated

Foundations of Research n n 48 Case study of unethical use of scientifically validated techniques. Beginning in 2003, the APA worked closely with CIA Psychologists James Mitchell and Bruce Jessen to develop “enhanced interrogation” (I. e. physical and psychological torture) used at Guantanamo Bay and other “Black sites” around the world. APA worked with the Bush White House, State Department and CIA to tailor it’s ethical standards to allow Mitchell and other Psychologists to develop and conduct the CIA interrogation program. Mark Wilson/Getty Images More than a decade after George J. Tenet, then the From the New Yorks Times article. C. I. A. director, signed a secret order suspending the agency’s use of enhanced interrogation techniques, the American Psychological Association’s actions are coming under scrutiny. § The White House actually help write a 2005 APA policy statement allowing Psychologists’ assistance in “enhanced interrogation”. The secretive committee that prepared the document was comprised primarily of Military Psychologists. § Dr. Martin Seligman, past president of APA, contributed to the CIA effort by presenting his theory of Learned Helplessness as a method of extracting information. § From a 2004 CIA memo : "The goal of [harsh] interrogation is to create a state of learned helplessness and dependence conducive to the collection of intelligence. ”

Foundations of Research n Case study of unethical use of scientifically validated techniques. 49

Foundations of Research n Case study of unethical use of scientifically validated techniques. 49 The American Psychiatric Association & American Medical Associations refused to endorse their members’ involvement in interrogations: n n The CIA were of linked to torture & theory unlawful § interrogations The misuse scientific detention; and data can have very serious As helping professions, the Psychiatric and Medical consequences. communities refused to sanction or participate in torture. From the New Yorks Times article. § In it’s 2005 report, the APA ethics committee declared that, as a research organization, it was exempt from the ethical standards of “helpers”. n n This resembles the Bush Administration's declaration that those held at Guantanamo, Abu Graib and elsewhere were “detainees” in a battle zone, rather than “prisoners of war”, so the Geneva Convention guidelines for the ethical treatment of prisoners did not apply. After years of boycotts and protests, in 2008 APA finally altered its stance and accepted the “do no harm” ethical stance of helping professions. § See a chilling documentary on the American Psychological Association’s adoption of torture policy here.

Foundations of Research n 50 Case study 2: Researchers’ larger ethical responsibility What responsibility

Foundations of Research n 50 Case study 2: Researchers’ larger ethical responsibility What responsibility does a researcher have for the eventual use of a discovery? n What if a finding that may do great good may also do great harm? n How does the scientific community – and should the scientific community – control that? Matt Edge for The New York Times ˆ Dr. Jennifer A. Doudna. Three years ago, she helped make one of the most monumental discoveries in biology.

Foundations of Research n 51 Case study 2: Researchers’ larger ethical responsibility Click the

Foundations of Research n 51 Case study 2: Researchers’ larger ethical responsibility Click the image for an NYT piece about Dr. Jennifer Doudna, who discovered an easy method to alter DNA. n n She discovered an easy way to alter existing – or insert new - DNA sequences in the genome. These alterations would then be passed down to successive generations. This gene manipulation may someday be able to cure genetic diseases. It may also lead to “custom” genetic tampering of embryos, to create physical – and psychological – characteristics parents want in their children. Matt Edge for The New York Times ˆ Dr. Jennifer A. Doudna. Three years ago, she helped make one of the most monumental discoveries in biology. § The Chinese have already begun research in primate embryos. § This is particularly troubling, given the early stage of the research and the potential consequences for a commercial use of gene splicing to modify embryos. § Click here for a discussion of attempts to ban or limit this research.

Foundations of Research n 52 Case study 2: Researchers’ larger ethical responsibility Click the

Foundations of Research n 52 Case study 2: Researchers’ larger ethical responsibility Click the image for an NYT piece about Dr. Jennifer Doudna, who discovered an easy method to alter DNA. n n She discovered an easy way to alter existing – Dissemination of a technique to change something or insert new - DNA sequences in the genome. as fundamental as the human genome is potentially These alterations would then be passed down to dangerous. successive generations. n n ü At this stage the research is preliminary, so its This gene manipulation may someday be able to application would be particularly irresponsible. cure genetic diseases. ü also Even astothe research develops, errors It may lead “custom” genetic tampering of may lead to Matt Edge for The New York Times ˆ dire results. physical – and psychological – embryos, to create Dr. Jennifer A. Doudna. Three years ago, she helped make one of the characteristics parents their children. ü However, any want new in technology – physical, electronic, most monumental discoveries in biology. or biological – seems to always find a market. § The Chinese have already begun research in primate embryos. ü So, should researchers “go there” and work on topics § This is particularly troubling, given the early stage of the research and the such as gene tampering at all? potential consequences for a commercial use of gene splicing to modify embryos. § Click here for a discussion of attempts to ban or limit this research.

Foundations of Research Misleading, biased or fraudulent results. The Common Rule § Beyond potential

Foundations of Research Misleading, biased or fraudulent results. The Common Rule § Beyond potential harm to participants, social groups or society at large, a key form of research ethics is honesty in reasonable gathering and reporting results. n Minimize risks n Risks must be n Recruit participants equitably n Informed consent plagued by… n Document consentü Questionable or tenuous results, that n Monitor for safety n § Behavioral research has been cannot be repeated (replicated) by others, Intentional (if thoughtless) biases, Protect vulnerableüparticipants ü More rarely, outright fraud. & maintain confidentiality 53

Foundations of Research False Positives: questionable, tenuous, non-reproducible results. Quick note: Any research study

Foundations of Research False Positives: questionable, tenuous, non-reproducible results. Quick note: Any research study begins assuming the Null Hypothesis. Negative finding: finding any ‘results’ are by chance alone; there in nothing going on…) (Negative If the results are strong enough, the investigator can Reject the Null Hypothesis. finding the results are not just due to chance or a confound; (Positive finding: the hypothesis is confirmed, theory is supported) For obvious reasons positive results are the bread and butter of science, negative Results are considered to have little value. A false positive is a positive result due to to error, biases, or subtle (…not so subtle…) fraud. It appears that science generally, and the behavioral sciences in particular, are rife with false positives 54

Foundations of Research 55 False Positives § Inaccurate or non-reproducible (false positive) results often

Foundations of Research 55 False Positives § Inaccurate or non-reproducible (false positive) results often stem from confirmatory bias among investigators, sloppy work, or inappropriate statistics. ü Replication is a process where a study is repeated exactly, to ensure the results were not due to a fluke or the specific conditions in one lab (i. e. , an artifact). ü Converging studies address a topic using different methods or measures, to ensure the results are not simply due to the general method used. Ø Research journals (and grant awards) strongly favor new, exciting or innovative results. So, few replicating or converging studies are performed; it can be difficult to get them published. Ø A lot of work then goes ‘unchecked’ – and replicating studies that are conducted often fail to reproduce the original result. § The Reproducibility Project attempted to rigorously replicate 100 Ψ studies from 2008. Over 60% showed far weaker (or no) results when reexamined.

Foundations of Research 56 False Positives Positive Results Bias § A 2010 study in

Foundations of Research 56 False Positives Positive Results Bias § A 2010 study in Plo. S ONE showed over 70% of published articles in all science disciplines to have positive results. § Psychiatry and Psychology articles show > 90% positive results! § It is not remotely plausible that > 90% of our hypotheses are “true” and actually supported by objective research. Data from: Fanelli D (2010) “Positive” Results Increase Down the Hierarchy of the Sciences. PLo. S ONE 5(4): e 10068. doi: 10. 1371/journal. pone. 0010068 § Positive results bias can lead tenuous or simply false “findings” to be published, making our empirical base generally suspect.

Foundations of Research The varieties of research cheating 1. Tweaking 2. Stage managing 3.

Foundations of Research The varieties of research cheating 1. Tweaking 2. Stage managing 3. Cherry picking 4. P-hacking 5. Simple faking 57

Foundations of Research Misleading, biased or fraudulent results. 58 1. “Tweaking” study designs to

Foundations of Research Misleading, biased or fraudulent results. 58 1. “Tweaking” study designs to ensure positive results; ü Ideally, a study is designed to test the hypothesis: ü The null hypothesis and the alternate hypothesis (negative & positive results) should each have a fair chance of emerging. ü Unfortunately, many studies are conceived less to test a hypotheses than to prove or confirm one. ü Confirmatory bias can induce both intentional and unintentional biases in favor of positive results. ü This can be a consequence of a strong theory: ü The researcher is convinced (based on previous studies, with or without clear evidence) that theory is correct. ü The point of the study is then less to test whether theory is correct …than to find procedures that will empirically confirm it.

Foundations of Research Misleading, biased or fraudulent results. 59 1. “Tweaking” study designs to

Foundations of Research Misleading, biased or fraudulent results. 59 1. “Tweaking” study designs to ensure positive results; ü Ideally, a study is designed to test the hypothesis: ü Unfortunately, many studies are conceived less to test a hypotheses than to prove or confirm one. ü Confirmatory bias can induce both intentional and unintentional biases in favor of positive results. ü This can be a consequence of a strong theory: Researchers may repeat experiments, tweaking one element or another – the setting, the measurements, statistical analyses, participants – until they get the expected result. This powerful confirmatory bias is often not recognized as a problem.

Foundations of Research Misleading, biased or fraudulent results. 60 1. “Tweaking” study designs to

Foundations of Research Misleading, biased or fraudulent results. 60 1. “Tweaking” study designs to ensure positive results; ü Ideally, a study is designed to test the hypothesis: ü Unfortunately, many studies are conceived less to test a hypotheses than to prove or confirm one. ü Confirmatory bias can induce both intentional and unintentional biases in favor of positive results. ü This can be a consequence of a strong theory: Confirmatory bias in corporate drug trials: In analyses done in 2009 & 2010, 97. 4% of drug trials supported by the pharmaceutical industry reported positive findings. Only 68. 7% of those that weren't supported by drug manufacturers were as encouraging.

Foundations of Research Misleading, biased or fraudulent results. 2. Overly “Stage Managing” the experimental

Foundations of Research Misleading, biased or fraudulent results. 2. Overly “Stage Managing” the experimental setting. ü Many Psychology experiments are “stage managed”: ü Specific (manipulative? ) instructions or information; ü Stimuli in the environment… § Other people may model a certain behavior; § Your Week 3 reading discussed alcohol experiments that stage a “bump” to test hypotheses about how participants will react; § The Milgram experiments “staged” a learning task. ü Narrowing participants’ choices in how they can react to a stimulus; § Binary (“true”/”false”) measures that disallow nuanced responses. ü Stage managing can be crucial to testing behavioral hypotheses, by presenting more realistic stimuli. ü Experiments that overly stage manage can exert a powerful confirmatory bias by driving only certain responses. 61

Foundations of Research Misleading, biased or fraudulent results. 3. “Cherry Picking” results; publishing only

Foundations of Research Misleading, biased or fraudulent results. 3. “Cherry Picking” results; publishing only those that “work”. ü Ideally, a study sets out a clear, a priori hypothesis. § The hypothesis (or methods section) clearly specifies the variable(s) to be examined and the statistical approach. § The analysis and discussion address only the target variables or measures specified in the hypothesis. ü This ideal process is often not followed, particularly in the social sciences. § Researchers may examine a number of variables and write up only those that showed an “effect”; § Researchers may conduct multiple studies and publish only those providing confirmation of the hypothesis… 62

Foundations of Research Misleading, biased or fraudulent results. 63 “Cherry Picking” results; Selective choice

Foundations of Research Misleading, biased or fraudulent results. 63 “Cherry Picking” results; Selective choice of variables: ü Data sets often contain multiple variables. ü Investigators can cheat by testing a range of variables, identifying the specific measures that support the hypothesis, publishing only the results for those measures, and even re-writing the hypothesis post-hoc to make it seem as though the specific measure that “worked” was the one they had in mind all along. o For example, Bohannon’s fake chocolate study assessed multiple possible outcomes of a diet program; see the discussion below. ü This form of fraud is not always intentional or malicious; ü With a complex study we want to explore all aspects of the data. ü However, our bias toward positive results will draw attention to only some outcomes. ü As we saw in the Fen Phen scandal, company-sponsored researchers occasionally simply ignore negative outcome variables.

Foundations of Research “Cherry Picking” results; Multiple studies ü Pharmaceutical companies had been particularly

Foundations of Research “Cherry Picking” results; Multiple studies ü Pharmaceutical companies had been particularly implicated in this: o Companies would run multiple trials of a new drug, and submit only the successful outcomes to the FDA for approval. o The Food & Drug Administration (FDA) now requires submission and access to all drug trials. ü This has been called “the file drawer problem”; journals have little interest in negative results, so only positive outcomes get submitted for publication. ü Researchers rarely attempt to publish negative results. ü This is recently changing: o Researchers are increasingly aware of positive results bias, and journals are becoming more accepting of negative results. o Sophisticated statistical techniques allow writers or reviewers to estimate the extent of the file drawer problem in a research area 64

Foundations of Research “Cherry Picking” results; Multiple studies ü Pharmaceutical companies been particularly §

Foundations of Research “Cherry Picking” results; Multiple studies ü Pharmaceutical companies been particularly § To help resolvehad a number of these implicated in this: issues… o Companies would run multiple trials of a new drug, and submit only the successful outcomes to the FDA for approval. ü Positive results bias & cherry picking, o The Food & Drug Administration (FDA) now requires submission and access to all ü The file drawer problem, drug trials. ü Difficulty replicating studies, ü This has been called “the file drawer problem”; journals have little interest in negative results, so onlyconfirmatory positive outcomes ü Potential biases… get submitted for publication. § …researchers aretoincreasingly being more ü Researchers rarely attempt publish negative results. transparent, and making all the data ü This is recently changing: underlying their research publicly available. o Researchers are increasingly aware of positive results bias, and journals are becoming accepting of negative results. § Seemore a New York Times report here. o Sophisticated statistical techniques allow writers or reviewers to estimate the extent of the file drawer problem in a research area 65

Foundations of Research 66 Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD

Foundations of Research 66 Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD (2015) The Extent and Consequences of P-Hacking in Science. PLo. S Biol 13(3): e 1002106. doi: 10. 1371/journal. pbio. 1002106 Click for original article 4. P-hacking and False Positives Dr. Megan Head and colleagues examined 100, 000 research papers spanning scientific disciplines including medicine, biology and psychology. § Click here for a summary. § Head’s study found a high number of p-values that were only just over the traditional threshold that most scientists call statistically significant. § Quotes from the article (above): § “This suggests that some scientists adjust their experimental design, datasets or statistical methods until they get a result that crosses the significance threshold”. § “They might look at their results before an experiment is finished, or explore their data with lots of different statistical methods, without realizing that this can lead to bias. ” § “Journals, especially the top journals, are more likely to publish experiments with new, interesting results, creating incentive to produce results on demand. ” § The pressure for new, exciting results works against longer time frames in studies, which help us really understand, e. g. , the effect of a treatment. It almost rules out studies that attempt to replicate previous research.

Foundations of Research Misleading, biased or fraudulent results. 67 Ø Simple faking; ü Unfortunately,

Foundations of Research Misleading, biased or fraudulent results. 67 Ø Simple faking; ü Unfortunately, it is not that difficult to fake a study and garner substantial media attention of a phony (even silly) “finding”. ü How to create a fake study and get world-wide attention: John Bohannon’s fake chocolate weight loss study.

Foundations of Research 68 Misleading, biased or fraudulent results. § There is substantial pressure

Foundations of Research 68 Misleading, biased or fraudulent results. § There is substantial pressure – and temptation – for actual fraud. Ø Career advancement – and funding – requires finding positive results; Diederik Stapel, a Dutch social psychologist, perpetrated an audacious academic fraud by making up studies that told the world what it wanted to hear about human nature. Click for New York Times overview. Koos Breukel for The New York Times Ø Experiments must “work” to be published, to get drugs licensed, to apply for further funding… Ø As a consequence, U. S. Behavioral research shows a strong bias toward positive results. § A case study of research fraud Ø Simple faking; Deitrik Stapel successfully faked 20 years of data (click image). ü Stapel perpetrated the largest fraud in modern behavioral science. ü He made up results that were plausible and interesting (“sexy”, in his word), but close enough to real results to not be questioned ü He was aided by trust and lack of strong oversight in the research industry. ü Over a 20 -year fake research program he published 55 research papers, 11+ Doctoral dissertations, and secured substantial research funding.

Foundations of Research 69 Misleading, biased or fraudulent results. § Corporate research: major funder

Foundations of Research 69 Misleading, biased or fraudulent results. § Corporate research: major funder of junk science. Ø Companies spend significant amounts to produce junk science favorable to their products. Ø One study shows that research funded by the soft drink industry is 5 times (!) more likely to find no association between diet and obesity than is publically funded research. Click: Sugar industry pays scientists to produce “findings” that sweet foods are not harmful. Zoonar/P. Malyshev via Getty Images Ø Coca Cola has been particularly aggressive in this. • Concern over obesity has led people – particularly parents – to lessen their consumption of soft drinks, meaning that Coke is losing revenue. • Despite clear evidence that diet (particularly sugary drinks) is far more important to obesity than is exercise, Coke has mounted a campaign to convince the public of the opposite. • They have spent millions funding legitimate researchers to distort the literature and promote exercise as central to avoiding obesity, and to promote Coke as a healthy snack. An image from a video by the Coca-Cola Foundation. In November 2012, the foundation announced a $3 million grant to Chicago's Garfield Park Conservatory Alliance. The grant was intended to establish a wellness program. Shortly after, a bill to tax sugary soft drinks was dropped by the Chicago City Council. Click the image: NY Times overview of Coke’s funding of highly biased obesity research. Read an editorial here Image: NYTimes, August 9, 2015

Foundations of Research The consequences of biased research studies What are the consequences of

Foundations of Research The consequences of biased research studies What are the consequences of subtle (or not so subtle) cheating in research? § Compromised or poorly conducted research… § § § “Tweaking” study designs to ensure positive results, Cherry picking only positive results, Statistical “errors” favoring the hypothesis, Simple faking, Lax standards for evaluating results, § … can lead to the appearance (or gross overstatement) of positive results in data that actually do not support the hypothesis. § Evidence from journal publications shows Psychiatry / Psychology to have a very strong “positive results bias”: 70

Foundations of Research SUMMARY n n Overview The “Common Rule”: core guidelines n Minimize

Foundations of Research SUMMARY n n Overview The “Common Rule”: core guidelines n Minimize risks n Risks must be reasonable n Recruit participants equitably n Informed / Document consent n Monitor for safety n Protect vulnerable participants & maintain confidentiality Larger institutions have key roles in maintaining ethical behavior n n n Institutional Review Boards (IRBs) Organizations such as the APA can behave in clearly unethical fashion The “research industry” directly presses for questionable, mediocre or fraudulent research. 71

72 Foundations of Research q The Common Rule: Core criteria for ethical research q

72 Foundations of Research q The Common Rule: Core criteria for ethical research q The Belmont Report and the Informed Consent document Respect for persons Beneficence Justice

Foundations of Research Belmont Report (CITI training) 73 1. Respect For Persons Ø Right

Foundations of Research Belmont Report (CITI training) 73 1. Respect For Persons Ø Right to exercise autonomy & make informed choices. 2. Beneficence Ø Minimization of risk & maximization of social / individual benefit How much information should participants get from a blinded, randomized trial? See ethics of clinical trials 3. Justice Ø Research should not unduly involve groups unlikely to benefit from subsequent applications. Ø Include participants of all races & both genders Ø Members of target population on design & research team Ø Research & researchers contribute to study population studied Ø Communicate research results & develop programs/ interventions

Foundations of Research Optional section: Click to go to a detailed description of the

Foundations of Research Optional section: Click to go to a detailed description of the sections used in a complete informed consent document. I will not ask you to list these in the exam, but you will use this for your paper. Several key items you should know: • The research is completely voluntary • Participant must understand each element of the study. • Participant must willingly sign the informed consent document. 74

Foundations of Research n Why am I being asked to participate in this research?

Foundations of Research n Why am I being asked to participate in this research? n n n The informed consent document . . who is being recruited or selected? Why is this research being done? Overview Box: Brief descriptions. . ü ü procedures, purposes, potential risks & benefits, potential outcomes. n What is the purpose of this research? n What procedures are involved? n What are the potential risks and discomforts? n Are there benefits to taking part in this research? n What other options are there? . . . What happens if I decline participation? 75

Foundations of Research n Informed consent elements, 2 Will I be told about new

Foundations of Research n Informed consent elements, 2 Will I be told about new information that may affect my decision to participate? n What about privacy and confidentiality? n What if I am injured as a result of my participation? n What are the costs for participating in this research? n n . . . e. g. , for services, etc. Will I be reimbursed for any of my expenses for participation in this research? n Can I withdraw or be removed from the study? n Who should I contact if I have questions? n Signature of participant or legally authorized representative 76