Experimental designs The strongest of the research designs

  • Slides: 114
Download presentation
Experimental designs The strongest of the research designs Image: www. freeimages. co. uk 1

Experimental designs The strongest of the research designs Image: www. freeimages. co. uk 1 © 2006

Categories of research • Quantitative – Involves numerical data that result from taking measurements

Categories of research • Quantitative – Involves numerical data that result from taking measurements on subjects – Is objective – Deductive reasoning • Is used to test theories or ideas to determine whether or not they are true – The researcher is an objective observer Evidence-based Chiropractic 2 Image: www. freeimages. co. uk © 2006

Categories of research (cont. ) • Qualitative – Involves data derived from words e.

Categories of research (cont. ) • Qualitative – Involves data derived from words e. g. , questionnaires or interviews – Is subjective – Inductive reasoning • Reasoning based on observations which are used to create an idea or theory – The researcher actively involved at times Evidence-based Chiropractic 3 © 2006

Quantitative vs. qualitative research • Quantitative research employs the scientific method and is usually

Quantitative vs. qualitative research • Quantitative research employs the scientific method and is usually regarded at a higher level – But may have limited relevance to clinical practice because of strict methods • Qualitative research often leads to quantitative studies • Both forms of research are important Evidence-based Chiropractic 4 © 2006

Pragmatic and explanatory research • Pragmatic research – Used to verify the effectiveness of

Pragmatic and explanatory research • Pragmatic research – Used to verify the effectiveness of treatments • i. e. , whether they work under real-life conditions – Does not determine how or why the treatments work – Typically used to help make decisions about the effectiveness of new treatments compared with existing treatments Evidence-based Chiropractic 5 © 2006

Pragmatic and explanatory research (cont. ) • Explanatory research – Used to establish the

Pragmatic and explanatory research (cont. ) • Explanatory research – Used to establish the efficacy of treatments • i. e. , how they work under ideal conditions, as in a controlled experiment – Capable of answering questions about how and why treatments work – Strict methods involved are often very different from day-to-day clinical practice • Consequently, results may not be relevant to practitioners Evidence-based Chiropractic 6 © 2006

Pragmatic and explanatory research (cont. ) – Patient selection is more strict in explanatory

Pragmatic and explanatory research (cont. ) – Patient selection is more strict in explanatory studies – Patients are excluded because of things like co-morbid conditions, prior treatment, severity of the condition, age, etc. – This may be a disadvantage because it is not known whether the treatment will work for patients in everyday practice • Patients commonly present with many of the exclusion criteria Evidence-based Chiropractic 7 © 2006

Descriptive, relational, and causal research • Descriptive (observational) research – Observes and records various

Descriptive, relational, and causal research • Descriptive (observational) research – Observes and records various aspects of participants in a study – Descriptive statistics involved • Relational research – Considers relationships that may exist between variables – Correlation and regression Evidence-based Chiropractic 8 © 2006

Descriptive, relational, and causal research (cont. ) • Causal research – Explores whether an

Descriptive, relational, and causal research (cont. ) • Causal research – Explores whether an intervention causes or affects one or more outcome variables – The most demanding type of research that involves very detailed methods – Looks for statistically significant differences between groups Evidence-based Chiropractic 9 © 2006

Experimental and quasiexperimental research • Experimental research – Random assignment to groups is involved

Experimental and quasiexperimental research • Experimental research – Random assignment to groups is involved – Capable of determining cause-and-effect relationships • Quasi-experimental research – No random assignment – Provides much less evidence about causeand-effect relationships Evidence-based Chiropractic 10 © 2006

Experimental and quasiexperimental research (cont. ) • Non-experimental research – Does not involve random

Experimental and quasiexperimental research (cont. ) • Non-experimental research – Does not involve random assignment or even a comparison group – Merely involves the observation of one group before and after an intervention Evidence-based Chiropractic 11 © 2006

Research design notation • • R – random assignment O – observation or measure

Research design notation • • R – random assignment O – observation or measure X – treatment or intervention N – non-equivalent groups • The classic experiment – Randomization and 2 groups Evidence-based Chiropractic R R O O X O O Each row represents a group Time 12 © 2006

Research designs • A quasi-experiment – 2 groups but no randomization • Non-experiment –

Research designs • A quasi-experiment – 2 groups but no randomization • Non-experiment – Only 1 group Evidence-based Chiropractic 13 N N O O O X X O O O © 2006

Population • The units from which a sample is drawn – May include people,

Population • The units from which a sample is drawn – May include people, but can also consist of events or observations • It is rarely possible to include each and every unit of a population – Instead, a smaller number of units (a sample) are selected to represent the entire population • Defined as a subset of observations from a population Evidence-based Chiropractic 14 © 2006

Samples • Samples can permit inferences about what is happening in a population based

Samples • Samples can permit inferences about what is happening in a population based on what is observed in a sample • However, the sample must be representative of the population – Often achieved through random selection of the sample units whereby each unit of the population has an equal chance of being selected Evidence-based Chiropractic 15 © 2006

Sample selection A sample is selected Evidence-based Chiropractic 16 © 2006

Sample selection A sample is selected Evidence-based Chiropractic 16 © 2006

Samples (cont. ) • Population parameters that are estimated from random samples are known

Samples (cont. ) • Population parameters that are estimated from random samples are known as unbiased estimates • Random sampling is rarely employed in clinical trials – Patients are obtained using sequentially presenting patients or recruiting through advertisements – Referred to as convenience sampling Evidence-based Chiropractic 17 © 2006

Samples (cont. ) • Selection criteria in clinical trials – Patients are usually included

Samples (cont. ) • Selection criteria in clinical trials – Patients are usually included in a clinical trial only if they meet certain criteria – e. g. , severity of the condition, no secondary conditions, history, age, etc. • It is important to consider features of the population in a study when applying its results to a specific patient Evidence-based Chiropractic 18 © 2006

Random assignment • Clinical trials often employ random assignment (a. k. a. , randomization)

Random assignment • Clinical trials often employ random assignment (a. k. a. , randomization) – Refers to the way patients are assigned to groups • Used to make groups equivalent regarding prognostic factors (e. g. , pain levels) – Sometimes called probabilistic equivalence because there is still a chance the groups will be a different after randomization Evidence-based Chiropractic 19 © 2006

Random assignment (cont. ) • Blocking – Subjects are separated into homogeneous subgroups based

Random assignment (cont. ) • Blocking – Subjects are separated into homogeneous subgroups based on factors such as age or disease severity – Enhances comparison because the subgroups are more alike than the intact groups Evidence-based Chiropractic 20 © 2006

Random assignment (cont. ) • Stratified randomization – Intact groups are separated into subgroups

Random assignment (cont. ) • Stratified randomization – Intact groups are separated into subgroups based on prognostic factors – e. g. , trauma vs. non-trauma patients in a whiplash study Evidence-based Chiropractic 21 © 2006

Random assignment (cont. ) • Concealment – Assignment is often concealed from researchers to

Random assignment (cont. ) • Concealment – Assignment is often concealed from researchers to avoid the temptation of allotting patients with certain traits to groups that will receive special treatment • When concealment is inadequate, the apparent effects of the treatment may be distorted as much as or more than the size of the effect being investigated Evidence-based Chiropractic 22 © 2006

Sample size determination • Articles about clinical trials should discuss why the number of

Sample size determination • Articles about clinical trials should discuss why the number of subjects was chosen • Ethically important – Because no more subjects should be inconvenienced or put at risk than required to find a treatment effect • Economically – Extra resources required to include unnecessary subjects Evidence-based Chiropractic 23 © 2006

Sample size determination (cont. ) • Too few subjects reduces the power of a

Sample size determination (cont. ) • Too few subjects reduces the power of a study so that a treatment effect may not be noticeable when it actually is present • Extremely large samples may show statistically significant differences between groups that are so small they are not clinically important Evidence-based Chiropractic 24 © 2006

The randomized controlled trial (RCT) • Regarded as the ultimate research design in health

The randomized controlled trial (RCT) • Regarded as the ultimate research design in health care • The classic experiment Evidence-based Chiropractic 25 © 2006

Placebo • An inert substance or treatment – Compared to the active substance or

Placebo • An inert substance or treatment – Compared to the active substance or treatment in RCTs – Used in pharmaceutical trials to establish whether an active drug is more effective than a placebo – The drug and placebo groups are compared to determine if the drug resulted in a statistically significant treatment effect Evidence-based Chiropractic 26 © 2006

Sham • A non-therapeutic intervention that imitates the real treatment – Similar to placebo,

Sham • A non-therapeutic intervention that imitates the real treatment – Similar to placebo, but refers to something done rather than something taken – Patients should have a very difficult time telling the difference between a sham and the real treatment – A sham chiropractic manipulation is difficult to produce Evidence-based Chiropractic 27 © 2006

Treatment effect • The result that a treatment has on outcomes that is attributable

Treatment effect • The result that a treatment has on outcomes that is attributable specifically to the effect of the intervention • The difference between the mean outcomes observed in a treatment group and a control group Evidence-based Chiropractic 28 © 2006

Why patients improve • Natural history – Many acute and some chronic pain conditions

Why patients improve • Natural history – Many acute and some chronic pain conditions resolve on their own • Actual effect of the treatment • Nonspecific effects of the treatment – Linked to the treatment, but actually due to factors other than the active components of the treatment – Sometimes called placebo effects Evidence-based Chiropractic 29 © 2006

Components of treatment Evidence-based Chiropractic 30 © 2006

Components of treatment Evidence-based Chiropractic 30 © 2006

Effectiveness of a treatment • Both the placebo and treatment groups typically improve •

Effectiveness of a treatment • Both the placebo and treatment groups typically improve • The difference between groups at the conclusion of the study is what matters • The treatment is considered effective if the mean outcome of the treatment group is significantly better than the placebo group Evidence-based Chiropractic 31 © 2006

Bias • Systematic errors in a study that are caused by problems with –

Bias • Systematic errors in a study that are caused by problems with – The selection or assignment of patients to groups – The measurements involved in the study • Bias can render a study invalid, although all studies have at least some bias Evidence-based Chiropractic 32 © 2006

Hawthorne effect • People tend to react differently when participating in experiments • Researchers

Hawthorne effect • People tend to react differently when participating in experiments • Researchers found that the productivity of workers increased when they new they were involved in a study – True under a variety of conditions – Even conditions that should have reduced productivity Evidence-based Chiropractic 33 © 2006

Hawthorne effect (cont. ) • Behavior was more influenced by the attention researchers gave

Hawthorne effect (cont. ) • Behavior was more influenced by the attention researchers gave to the subjects than the effect of the interventions • The Hawthorne effect is a factor in all clinical studies Evidence-based Chiropractic 34 © 2006

Types of bias • Sampling bias (a. k. a, selection bias) – During the

Types of bias • Sampling bias (a. k. a, selection bias) – During the selection process, each person does not have an equal chance of being selected from the source population – Random selection is designed to take care of this problem – Results in systematic differences between groups in experimental studies as to factors of prognosis or response to treatment Evidence-based Chiropractic 35 © 2006

Types of bias (cont). – Random assignment with concealment is the best safeguard against

Types of bias (cont). – Random assignment with concealment is the best safeguard against selection bias in RCTs – The effect of selection bias is reduced by random assignment because it distributes the bias evenly between the treatment and control groups Evidence-based Chiropractic 36 © 2006

Types of bias (cont). • Experimenter (researcher) bias – Examining or treating doctors may

Types of bias (cont). • Experimenter (researcher) bias – Examining or treating doctors may influence a study’s results because of their expectancies or desires for a certain outcome – Blinding (masking) of researchers and study participants as to group assignment can diminish the effect of this bias – This bias can be divided into detection bias and performance bias Evidence-based Chiropractic 37 © 2006

Types of bias (cont). • Exclusion bias – Occurs when patients who drop out

Types of bias (cont). • Exclusion bias – Occurs when patients who drop out of a study are systematically different from subjects who remain • Perhaps dropouts were having a poor response to treatment • Would have changed the results of the study if they had remained Evidence-based Chiropractic 38 © 2006

Extraneous and confounding variables • In experiments, researchers are able to manipulate the explanatory

Extraneous and confounding variables • In experiments, researchers are able to manipulate the explanatory variables and then watch what happens to the outcome variable • Internal validity – The ability of an experiment to show that the explanatory variables actually caused the observed changes in the outcome variables Evidence-based Chiropractic 39 © 2006

Extraneous and confounding variables (cont. ) • Extraneous variables – Uncontrolled factors that can

Extraneous and confounding variables (cont. ) • Extraneous variables – Uncontrolled factors that can influence the relationship between variables in an experiment – They are not the variables that are being studied, yet they affect the outcome of the experiment – They are unwanted because they create error Evidence-based Chiropractic 40 © 2006

Extraneous and confounding variables (cont. ) – Error variance due to extraneous variables is

Extraneous and confounding variables (cont. ) – Error variance due to extraneous variables is distributed evenly between the groups when random assignment is utilized • Confounding variable – A type of extraneous variable that affects the explanatory variables differently • e. g. , it affects the treatment group but not the control group – Introduces systematic error into the study Evidence-based Chiropractic 41 © 2006

Extraneous and confounding variables (cont. ) – The effect of a confounding variable cannot

Extraneous and confounding variables (cont. ) – The effect of a confounding variable cannot be separated from the outcome variable Explanatory variable e. g. , manipulation Outcome variable e. g. , low back pain Confounding variable e. g. , groups receive manual vs. instrument manipulation Evidence-based Chiropractic 42 © 2006

Extraneous and confounding variables (cont. ) • Quasi-experimental designs are particularly susceptible to confounding

Extraneous and confounding variables (cont. ) • Quasi-experimental designs are particularly susceptible to confounding because the individual differences of subjects may act as confounding variables • For example – A quasi-experimental study that assigned headache patients with more severe pain to the treatment group Evidence-based Chiropractic 43 © 2006

Threats to internal validity • History – Participants are unintentionally exposed to some historical

Threats to internal validity • History – Participants are unintentionally exposed to some historical event during the research project which affects the results – For example • A statewide fitness campaign that coincides with a lower back pain study • Some of the subjects doing the exercises would likely affect the study’s outcome Evidence-based Chiropractic 44 © 2006

Threats to internal validity (cont. ) • Reliability of measures – Unreliable measures can

Threats to internal validity (cont. ) • Reliability of measures – Unreliable measures can invalidate a study – Possible causes • Faulty equipment, inconsistent instructions to study participants, unreliable training of examiners, fatigue or boredom of examiners, or examiners becoming more skilled at doing the test Evidence-based Chiropractic 45 © 2006

Threats to internal validity (cont. ) • Mortality – Subjects dropping out of studies

Threats to internal validity (cont. ) • Mortality – Subjects dropping out of studies – Drop-outs may be different from those who remain – Occurs for a variety of reasons • e. g. , poor response to treatment, exceptional response to treatment, adverse effects – Groups may not be equivalent as a result Evidence-based Chiropractic 46 © 2006

Threats to internal validity (cont. ) • Maturation – Changes that occur in study

Threats to internal validity (cont. ) • Maturation – Changes that occur in study participants as time passes that are not caused by the explanatory variables – e. g. , in a study investigating strength in children, they would most likely get stronger in time, even without exposure to the explanatory variables Evidence-based Chiropractic 47 © 2006

Threats to internal validity (cont. ) • Regression to the mean – Extreme scores

Threats to internal validity (cont. ) • Regression to the mean – Extreme scores at the beginning of a study that migrate toward the mean as time passes – Occurs because extreme symptoms tend to return to a more normal state on their own • i. e. , high initial patient scores are much more likely to move toward normality than to go even higher – Especially problematic when patients are selected based on high test values, while patients with low values are screened out Evidence-based Chiropractic 48 © 2006

Read and bring to class Monday 11/3/08 • Hoiriis et al. A Randomized Clinical

Read and bring to class Monday 11/3/08 • Hoiriis et al. A Randomized Clinical Trial Comparing Chiropractic Adjustments To Muscle Relaxants For Subacute Low Back Pain. JMPT 2004; 27: 388 -98 • Bakris, et al. Atlas vertebra realignment and achievement of arterial pressure goal in hypertensive patients: a pilot study. J Hum Hypertens. 2007 May; 21(5): 347 -52. Evidence-based Chiropractic 49 © 2006

External validity a. k. a. , generalizability • The extent results of a study

External validity a. k. a. , generalizability • The extent results of a study are applicable to other populations, other settings, and when implemented under different circumstances – Should be comparable regarding the intervention, age, condition severity, etc. • Relating to EBP – Are the results of a study applicable to the management of a particular patient? Evidence-based Chiropractic 50 © 2006

External validity (cont. ) • Meade et al. study – Office-based chiropractic care was

External validity (cont. ) • Meade et al. study – Office-based chiropractic care was compared with hospital-based physical therapy for low back pain – Chiropractic was found to be superior, but may have been related to patients being treated in private chiropractic offices versus out-patient PT departments at hospitals Evidence-based Chiropractic 51 © 2006

Internal validity vs. external validity Evidence-based Chiropractic 52 © 2006

Internal validity vs. external validity Evidence-based Chiropractic 52 © 2006

Group Mean vs. an Individual Patient • A RCT only considers the average of

Group Mean vs. an Individual Patient • A RCT only considers the average of a group of subjects • A given patient will NOT be average – Each patient is unique in some way regarding condition severity, secondary conditions, response to care, etc. • Each practitioner is unique with a whole arsenal of treatment options Evidence-based Chiropractic 53 © 2006

Research designs • The pretest-posttest randomized experimental design – Is the classic experiment design

Research designs • The pretest-posttest randomized experimental design – Is the classic experiment design mentioned earlier • The most commonly used design in research – Patients are randomized to treatment groups which drastically reduces the chance of bias Evidence-based Chiropractic 54 © 2006

Classic experiment design (cont. ) – Subjects are evaluated before and after the intervention

Classic experiment design (cont. ) – Subjects are evaluated before and after the intervention so that pre-treatment differences between groups can be considered • Groups are rarely exactly equivalent • Analysis of covariance (ANCOVA) test factors in pretreatment differences between groups as a covariate – Use of a control group allows separation of the active ingredient of the treatment effect from non-specific components Evidence-based Chiropractic 55 © 2006

ANCOVA test The ANCOVA test factors in pretreatment differences between groups as a covariate

ANCOVA test The ANCOVA test factors in pretreatment differences between groups as a covariate Evidence-based Chiropractic 56 © 2006

ANCOVA test • Statistically removes the effect of covariates from the analysis • Other

ANCOVA test • Statistically removes the effect of covariates from the analysis • Other variables can also be “adjusted for” using ANCOVA – e. g. , differences between groups regarding age or condition severity • Example report in journal article –. . . the effects of pre-treatment differences were adjusted for during analysis Evidence-based Chiropractic 57 © 2006

Two-group pretest-posttest design • Comparison with an alternate form of treatment – e. g.

Two-group pretest-posttest design • Comparison with an alternate form of treatment – e. g. , a new therapy is compared to an established therapy – Cannot determine whether a new treatment works better than no treatment R R Evidence-based Chiropractic O O X 1 X 2 58 O O © 2006

Post-test only randomized controlled trial • Groups cannot be compared after randomization because no

Post-test only randomized controlled trial • Groups cannot be compared after randomization because no pretest is used – It is a weaker design because of doubts about the success of randomization • Sometimes used when groups are large – Large groups are much more likely to be equivalent R R Evidence-based Chiropractic X 59 O O © 2006

Factorial design • Often used when several explanatory variables are involved in a study

Factorial design • Often used when several explanatory variables are involved in a study • Useful to determine if any interaction exists between the variables • Explanatory variables are categorized as – Factors (the major independent variables) – Levels (subgroups) Evidence-based Chiropractic 60 © 2006

Factorial design (cont. ) • Two factor by two level (2 X 2) factorial

Factorial design (cont. ) • Two factor by two level (2 X 2) factorial design X 11 X 21 Evidence-based Chiropractic 61 X 12 X 22 © 2006

Factorial design (cont. ) – Group 1 received Diversified technique and palpation as the

Factorial design (cont. ) – Group 1 received Diversified technique and palpation as the method of analysis – Group 2 Gonstead and palpation – Group 3 Diversified and x-ray – Group 4 Gonstead and x-ray Factorial design notation Evidence-based Chiropractic R R O O 62 X 11 X 12 X 21 X 22 O O © 2006

Crossover design • Treatment is provided to one group, while the other group receives

Crossover design • Treatment is provided to one group, while the other group receives a placebo or alternate treatment • Group assignments are switched at some point in time without the doctors’ or subjects’ knowledge • Each group receives both the active treatment and the alternate treatment Evidence-based Chiropractic 63 © 2006

Crossover design (cont. ) • Each subject acts as their own control, which can

Crossover design (cont. ) • Each subject acts as their own control, which can reduce the required sample size considerably Evidence-based Chiropractic 64 © 2006

Crossover design (cont. ) Crossover design notation R R O O Evidence-based Chiropractic X

Crossover design (cont. ) Crossover design notation R R O O Evidence-based Chiropractic X 1 X 2 O O Optional washout period 65 O O X 2 X 1 O O © 2006

Crossover design (cont. ) • Crossover design limitations – Carry-over effects • The therapeutic

Crossover design (cont. ) • Crossover design limitations – Carry-over effects • The therapeutic effects of the first intervention continue during the second intervention – High dropout rates • Because there are 2 or more periods of treatment • The negative effect is more harmful to the data analysis than other designs because each patient’s data is so important Evidence-based Chiropractic 66 © 2006

Crossover design (cont. ) – Treatment sequencing • Patients may respond differently when treatment

Crossover design (cont. ) – Treatment sequencing • Patients may respond differently when treatment 1 is given before treatment 2 than if the order is reversed – For example • A chronic neck pain study where treatment 1 is manipulation and treatment 2 is massage • Results may be different if treatment 2 is provided first because the massage may enable patients to receive a better effect from the manipulation Evidence-based Chiropractic 67 © 2006

Quasi-experimental designs • Very similar to the randomized designs, minus random assignment to groups

Quasi-experimental designs • Very similar to the randomized designs, minus random assignment to groups • The lack of randomization is a major factor that make claims about causality based on quasi-experimental evidence doubtful • On the other hand, a first-rate quasiexperiment can generate stronger evidence than a poorly conducted RCT Evidence-based Chiropractic 68 © 2006

Non-experimental designs • Do not utilize randomization or a comparison group • Are not

Non-experimental designs • Do not utilize randomization or a comparison group • Are not capable of determining the effect of an intervention • Includes – Survey and observational research – Case studies and case series Evidence-based Chiropractic 69 © 2006

Non-experimental designs (cont. ) • Non-experimental designs are low on the evidentiary scale –

Non-experimental designs (cont. ) • Non-experimental designs are low on the evidentiary scale – They are still quite valuable because they describe unfamiliar occurrences and often lead to more complex studies • Pretreatment measures may be taken, but usually one measure is involved X Evidence-based Chiropractic O 70 © 2006

Chiropractic interventions and experimental methods • Pharmaceutical experiments work well – Because it is

Chiropractic interventions and experimental methods • Pharmaceutical experiments work well – Because it is fairly easy to make an active pill and an identical looking placebo pill • Not so with chiropractic interventions – It is difficult to deceive doctors and patients – Sham adjustments are either so invasive they become therapeutic or so dissimilar from adjustments that patients know they are in the placebo group Evidence-based Chiropractic 71 © 2006

Chiropractic interventions and experimental methods (cont. ) – Patients may actually receive a treatment

Chiropractic interventions and experimental methods (cont. ) – Patients may actually receive a treatment effect when sham adjustments are too invasive – Conversely, they may not receive a placebo effect when they are aware of their inclusion in the placebo group Evidence-based Chiropractic 72 © 2006

Ethics in research • Biomedical research ethics came about as a result of abuse

Ethics in research • Biomedical research ethics came about as a result of abuse to research participants in the past – Nazis concentration camp experiments – The Tuskegee Syphilis Study – Research was often carried out on prisoners, orphans, or mentally ill; many times without the subjects’ knowledge Evidence-based Chiropractic 73 © 2006

Ethics in research (cont. ) • The Nuremberg Code (1947) – Developed from the

Ethics in research (cont. ) • The Nuremberg Code (1947) – Developed from the Nuremberg Military Tribunals – Had no legal authority (unenforceable) – Only dealt with research on healthy participants – Left the control of research entirely up to scientists Evidence-based Chiropractic 74 © 2006

Ethics in research (cont. ) • Declaration of Helsinki (1964) – Developed by the

Ethics in research (cont. ) • Declaration of Helsinki (1964) – Developed by the World Medical Association to remedy deficiencies of the Nuremberg Code – Has been amended five times, most recently in 2000 – Required medical researchers to clearly define study protocols – Protocols had to be submitted to independent ethical review committees (IRB) for approval Evidence-based Chiropractic 75 © 2006

Ethics in research (cont. ) • The Belmont Report (1979) – Included 3 principles

Ethics in research (cont. ) • The Belmont Report (1979) – Included 3 principles of research ethics • Autonomy – the principle of respect for persons; especially those with diminished autonomy • Beneficence – supports the well-being of research subjects by maximizing benefits while minimizing harms • Justice – the distribution of the benefits, burdens, and risks of research should be equitable between societal groups Evidence-based Chiropractic 76 © 2006

Informed consent • Nuremberg Code – “. . . the voluntary consent of the

Informed consent • Nuremberg Code – “. . . the voluntary consent of the human subject is absolutely essential. ” • Belmont Report – Respect for research participants – Research subjects must be given the opportunity to choose what shall or shall not happen to them Evidence-based Chiropractic 77 © 2006

Informed consent (cont. ) • Signed informed consent must be obtained from every research

Informed consent (cont. ) • Signed informed consent must be obtained from every research participant – However, it is a process, not just a form – The document should be thought of as a teaching tool, not as a legal instrument • The document must be written in “lay language” that is understandable to the people being asked to participate Evidence-based Chiropractic 78 © 2006

Informed consent (cont. ) • The consent document should explain – The purpose of

Informed consent (cont. ) • The consent document should explain – The purpose of the research – The benefits and risks – Confidentiality matters – A study contact person if questions arise – That participation is voluntary and subjects have the right to withdraw at any time – How adverse events are dealt with Evidence-based Chiropractic 79 © 2006

Placebo effects • Placebo is Latin for “I will please” • Refers to any

Placebo effects • Placebo is Latin for “I will please” • Refers to any type of treatment that is inert • Used in research trials to objectively test the efficacy of new treatments • One group is given the treatment, while another group (the control group) receives a placebo • Comparing the results from both groups should reveal the effects of the treatment Evidence-based Chiropractic 80 © 2006

Placebo American Heritage Dictionary 1. A substance containing no medication and prescribed or given

Placebo American Heritage Dictionary 1. A substance containing no medication and prescribed or given to reinforce a patient's expectation to get well 2. An inactive substance or preparation used as a control in an experiment or test to determine the effectiveness of a medicinal drug 3. Something of no intrinsic remedial value that is used to appease or reassure another Evidence-based Chiropractic 81 © 2006

Placebo effects (cont. ) • Interestingly, some people get better in the placebo group

Placebo effects (cont. ) • Interestingly, some people get better in the placebo group • This phenomenon is known as “the placebo effect” • The placebo effect is substantial – About one third of people taking placebos for a number of complaints will experience relief • The underlying mechanisms remain a mystery Evidence-based Chiropractic 82 © 2006

Sham • The term “sham treatment” is often used instead of placebo • Definition:

Sham • The term “sham treatment” is often used instead of placebo • Definition: – Something false or empty that is purported to be genuine; a spurious imitation Evidence-based Chiropractic 83 © 2006

Placebo effects (cont. ) • The placebo effect is triggered by the patient's belief

Placebo effects (cont. ) • The placebo effect is triggered by the patient's belief in the treatment and their expectation of feeling better • If symptoms are relieved by taking an inert substance or undergoing a dummy procedure, was the original illness imaginary? • No Evidence-based Chiropractic 84 © 2006

Factors that influence the placebo effect • Characteristics of the placebo – If the

Factors that influence the placebo effect • Characteristics of the placebo – If the pill (or treatment) looks genuine, the person taking it is more likely to believe that it contains active ingredients – Larger sized pills suggest a stronger dose than smaller pills, and taking two pills appears more potent than just one – Injections have a more powerful effect than pills Evidence-based Chiropractic 85 © 2006

Factors that influence (cont. ) • Attitude of the patient – If the person

Factors that influence (cont. ) • Attitude of the patient – If the person expects the treatment to work, the chances of a placebo effect are higher – However, the placebo effect may still take place even if the person is skeptical of success – The power of suggestion is probably at work here Evidence-based Chiropractic 86 © 2006

Factors that influence (cont. ) • Doctor-patient relationship – If the person trusts their

Factors that influence (cont. ) • Doctor-patient relationship – If the person trusts their health care practitioner, they are more likely to believe that the placebo will work – Chiropractors typically instill more trust in their patients, consequently critics have pointed to this as a likely explanation of our successes Evidence-based Chiropractic 87 © 2006

Placebo effects (cont. ) • Types of placebos – Pills are well-known for their

Placebo effects (cont. ) • Types of placebos – Pills are well-known for their placebo effect – However, a placebo can be any inert or “dummy” treatment – Special diets, exercise, physical therapy or surgery – Even chiropractic manipulation Evidence-based Chiropractic 88 © 2006

Psychic surgery Is actually produced by sleight of hand Animal tissue and blood are

Psychic surgery Is actually produced by sleight of hand Animal tissue and blood are used to give a realistic appearance, while a patient's fleshy midriff helps create the illusion that the surgeon's fingers have actually penetrated the body Still practiced today in Brazil and the Philippines Evidence-based Chiropractic 89 © 2006

How placebos work • Self-limiting disorders – Many conditions are self-limiting (e. g. ,

How placebos work • Self-limiting disorders – Many conditions are self-limiting (e. g. , common cold, some back or neck pain) – They will resolve on their own with or without treatment – Symptoms resolving is merely coincidence Evidence-based Chiropractic 90 © 2006

How placebos work (cont. ) • Remission – The symptoms of some disorders, such

How placebos work (cont. ) • Remission – The symptoms of some disorders, such as multiple sclerosis and lupus, may wax and wane – A remission during a course of placebos may be coincidence, and not due to the placebos at all Evidence-based Chiropractic 91 © 2006

How placebos work (cont. ) • Changes in behavior – The placebo may increase

How placebos work (cont. ) • Changes in behavior – The placebo may increase a person’s motivation to take better care of themselves, which may be responsible for the easing of their symptoms • Altered perception – The person’s interpretation of their symptoms may change with the expectation of feeling better. (e. g. , a sharp pain being reinterpreted as an uncomfortable tingling) Evidence-based Chiropractic 92 © 2006

How placebos work (cont. ) • Reduced anxiety – Taking the placebo and expecting

How placebos work (cont. ) • Reduced anxiety – Taking the placebo and expecting to feel better may soothe autonomic nervous system reducing levels of stress chemicals • Brain chemicals – Placebos may trigger the brain to release endorphins, the body's own natural painkillers Evidence-based Chiropractic 93 © 2006

How placebos work (cont. ) • Altered brain state – Research has shown that

How placebos work (cont. ) • Altered brain state – Research has shown that the brain responds to an imagined scene in much the same way it responds to an actual visualized scene. Placebos may help the brain to remember a time before the onset of symptoms, and then bring about physiological change – The so-called “remembered wellness” theory Evidence-based Chiropractic 94 © 2006

Placebo examples • A meta-analysis of studies of depressed individuals taking antidepressant medications suggests

Placebo examples • A meta-analysis of studies of depressed individuals taking antidepressant medications suggests that approximately: – One quarter of the drug response is due to the administration of an active medication – One half is a placebo effect to Prozacquarter but Hearing Placebo: A Meta–Listening The remaining is due to other Analysis of Antidepressant nonspecific factors Medication. Prevention & Treatment, Volume 1, Article 0002 a, June 26, 1998 Evidence-based Chiropractic 95 © 2006

Placebo examples (cont. ) • In a survey of surgery for lumbar disc disease,

Placebo examples (cont. ) • In a survey of surgery for lumbar disc disease, although no disc herniation was present in 346 patients (negative surgical exploration), complete relief of sciatica occurred in 37 percent and from back pain in 43 percent Evidence-based Chiropractic 96 © 2006

Placebo examples (cont. ) • Moseley et al did a double-blinded, randomized, placebo-controlled trial

Placebo examples (cont. ) • Moseley et al did a double-blinded, randomized, placebo-controlled trial to compare arthroscopic lavage and debridement vs. a sham procedure • They found that all three treatment groups fared equally: subjective symptomatic relief was reported, but no objective improvement in function in any of the groups Evidence-based Chiropractic 97 © 2006

Placebo examples (cont. ) • Forty years ago, a young Seattle cardiologist named Leonard

Placebo examples (cont. ) • Forty years ago, a young Seattle cardiologist named Leonard Cobb conducted a unique trial of a procedure then commonly used for angina, in which doctors made small incisions in the chest and tied knots in two arteries to try to increase blood flow to the heart. It was a popular technique and 90 percent of patients reported that it helped, but when Cobb compared it with placebo surgery in which he made incisions but did not tie off the arteries, the sham operations proved just as successful. The procedure, known as internal mammary ligation, was soon abandoned – "The Placebo Prescription" by Margaret Talbot, New York Times Magazine, January 9, 2000 Evidence-based Chiropractic 98 © 2006

Sham v. Pill • Kaptchuk et al. Sham device v inert pill: randomised controlled

Sham v. Pill • Kaptchuk et al. Sham device v inert pill: randomised controlled trial of two placebo treatments. BMJ 2006; 332: 391 -397. – Fake acupuncture and sugar pills were tested for their effect on relieving arm pain – Both groups noticed improvements, but fake acupuncture was significantly better – 25% of acupuncture group noticed side effects and 3 of the sugar pill group actually withdrew because of them Evidence-based Chiropractic 99 © 2006

Evidence-based Chiropractic 100 © 2006

Evidence-based Chiropractic 100 © 2006

Waiting list, where Hawthorne effects, natural progression of the disease, and regression to the

Waiting list, where Hawthorne effects, natural progression of the disease, and regression to the mean could be observed “Limited” interaction that involved placebo treatment plus minimal patient-practitioner interaction “Augmented” interaction that also involved a placebo treatment, plus a defined positive patient-practitioner relationship. Evidence-based Chiropractic Kaptchuk T. Components 101 of placebo effect: randomised controlled trial © in 2006 patients with irritable bowel syndrome. BMJ. 2008 May 3; 336

Placebo Death? Evidence-based Chiropractic 102 © 2006

Placebo Death? Evidence-based Chiropractic 102 © 2006

Natural history of a disease vs. placebo effect • The body has a natural

Natural history of a disease vs. placebo effect • The body has a natural ability to heal itself and people heal spontaneously, occasionally even when the illness is serious • Hard to differentiate from placebo effect • Cases of spontaneous remission sometimes end up being regarded as miracles Evidence-based Chiropractic 103 © 2006

Why is this important? • Placebo effects, disease natural history, and regression to the

Why is this important? • Placebo effects, disease natural history, and regression to the mean can result in high rates of good outcomes, which may be falsely ascribed to specific treatment effects • The true causes of improvements in pain after treatment remain unknown in the absence of independently evaluated randomized controlled trials Evidence-based Chiropractic 104 © 2006

Appraising research • The objective of appraisal is to assist readers in determining –

Appraising research • The objective of appraisal is to assist readers in determining – The extent a study is valid – Whether its findings are applicable to a given clinical situation • Involves judging the degree a study complied with proper research methodology Evidence-based Chiropractic 105 © 2006

Appraising research (cont. ) • Appraisal should be tailored to the particular research design

Appraising research (cont. ) • Appraisal should be tailored to the particular research design being evaluated • It is essential for readers to be familiar with the various designs that are commonly used Evidence-based Chiropractic 106 © 2006

Three issues to consider 1. Are the results of the study valid? – The

Three issues to consider 1. Are the results of the study valid? – The study methodology should be acceptable – Statements should be logical and supported by the study’s findings – Confounding factors 2. What are the results of the study? – If a treatment effect is present, consider whether it is clinically significant Evidence-based Chiropractic 107 © 2006

Three issues to consider (cont. ) 3. Will the results of the study help

Three issues to consider (cont. ) 3. Will the results of the study help my patients? – The study’s findings should be applicable to your practice or to one of your patients Evidence-based Chiropractic 108 © 2006

General Appraisal Checklist • Determine to what extent a study satisfies each point of

General Appraisal Checklist • Determine to what extent a study satisfies each point of the checklist – Yes Partly No • After appraisal, decide whether to use the information in your practice – Unfortunately, there are no strict rules to base this decision on – It is essentially a judgment call Evidence-based Chiropractic 109 © 2006

General Appraisal Checklist (cont. ) • The purpose of the study was conveyed plainly

General Appraisal Checklist (cont. ) • The purpose of the study was conveyed plainly and rationally. • A suitable literature review was presented that adequately covered the topic. • The research design was clearly described. • The research design was in concordance with the study question. • The research design was adequately implemented. Evidence-based Chiropractic 110 © 2006

General Appraisal Checklist (cont. ) • The target population was identified. • A determination

General Appraisal Checklist (cont. ) • The target population was identified. • A determination of the needed sample size was carried out. • The sample size was adequate. • Subjects were randomly assigned to groups. • Groups were equivalent. • Data collection methods were adequately described and were appropriate. • Demographic characteristics of the groups were presented. Evidence-based Chiropractic 111 © 2006

General Appraisal Checklist (cont. ) • The data were reported with sufficient detail. •

General Appraisal Checklist (cont. ) • The data were reported with sufficient detail. • Statistical tests were described and were appropriate for the type of data involved. • The discussion corresponded to and was supported by the data. • Discussion statements were reasonable and logical. • Conclusions were in agreement with the study’s predetermined purpose. Evidence-based Chiropractic 112 © 2006

General Appraisal Checklist (cont. ) • Study limitations were listed. • Was there any

General Appraisal Checklist (cont. ) • Study limitations were listed. • Was there any evidence of a “fishing expedition”? • References were accurate and appropriately used. • References were from respected evidence sources. Evidence-based Chiropractic 113 © 2006

General Appraisal Checklist (cont. ) • Overall impression about the trustworthiness of this article:

General Appraisal Checklist (cont. ) • Overall impression about the trustworthiness of this article: Good – Accept all or most of the article’s findings. Fair – Cautiously accept some of its findings. Poor – Do not accept any of its findings. • If acceptable, how will I apply the results to my patient/practice? – Consider generalizability and clinical significance issues Evidence-based Chiropractic 114 © 2006