Dr P P Venugopalan Dean Prof Dept of

Dr P. P. Venugopalan Dean & Prof. Dept of Community Medicine, Kannur Medical College

PITFALLS IN RESEARCH METHODS

• There are perils to treating patients , not as human beings but as means to some glorious end. • The Tuskegee study (1932) demonstrated the perils of treating patients , not as human beings but as a means to glorious end.

“Lies , damned lies and Medical Science” -Dr John Ioannidis

• Paul Kemmerer, Austrian Scientist • Committed suicide after his fraud experiment on toad to show “Lamarckian inheritence” using black injection into the hind legs of toads

• British scientist Charles Dawson and Arthur Smith Woodward 1912 • Skull of the so called “Piltdownman” in gravel pits in Sussex • Missing link between ape and man • Composite forgery-made of medieval human skull, 500 year old lower jaw of an Oranguttan and Chimpanzee fossil teeth • Detected after 40 years

• Dr William Summerlin • Transplantation Immunologist • Sloan Kettering Cancer Center, NY • Used marker pen to make black patches on fur of white mice in an attempt to prove his new skin graft technique , claiming no rejection • Painting mice = Fraud in research

• Dr Leon Essenberg before his death at 87 years made a statement to “Der. Spiegel” • ADHD is a fictitious disease, which they put together for the benefit of drug companies in the new disease classification in the American Psychiatry Associations DSM(Diagnostic and Statistical Manual of Mental Disease)

• Was it merely “discovery” of the specimen sent to them by Pasteur Institute in Paris? • American discovery of HIV virus ? ?

• 21 of 72 papers published in peer reviewed journals by Dr Scot S Reuben of the Baystate Medical Center in Springfield , Massachusetts , US • Found to contain fabricated data , during period 19962008. • Fabricated data involved drug celecoxib (Celebrix) Pfizer

• Professor Hwang Woo Suk • Seoul National University • With Gerald Schatten of University Pittsburg • Pennsylvania • Two papers in “Science” and one in “Nature” on human embryonic stem cell research and cloning of an Afghan hound

• Haruko Obokata • Riken center for Developmental Biology Kobe and Charles Vacanti • Anesthesiologist at Harvard Medical school in Boston • 2 papers in “Nature” on Jan 30 , 2014 • (National Institute of Health Website) • Duplicated images

• Dr Dipak Das-Professor in the department of Surgery, University of Connecticut, US • Fabricated and falsified data in over 100 instances in 26 articles. • Red wine to cardiovascular health • He has been found guilty of 145 counts of fabrication and falsification of data

• HPV Trial and PATH(Programme to Appropriate Technology and Health)in collaboration with Governments of Andhra Pradesh and Gujarat and ICMR on nearly 2350 girls in 10 -14 years age group in Khammam and Vadodara districts • 7 children died • 2 deaths in Khammam were unreported

• CNR Rao-Former Director Indian Institute of Science Bangalore and • SB Krupanidhi of Indian Institute of Science Bangalore • Issue of Journal - Advanced Materials • 22 July 2011 • Incorporated verbatim text from an earlier paper by different authors in Applied Physics Letters

• Dr Fazalurahman Khan Research Associate • Dr Swarnajit Singh Cameotra Environmental Biotechnology and Microbial Biochemistry Laboratory. Institute of Microbial Technology , Chandigarh Online journal of Hazardous Materials-June 15 , 2013 / Chemospheres /Journal of Petroleum-Environmental Biotechnology(Aug 21 and Nov 29, 2013) • Fabricated data

IMPORTANT CAUSES FOR REJECTION OF PAPERS FOR PUBLICATION STUDY - Not addressing specific and significant health/scientific issue - No claim as original study - Hypothesis not tested as proposed in the study - A different type of study should have been done - Faulty study method - Compromising on the original study protocol Adapted from Greenhalgh T, BMJ 1997; 15: 243 -6

IMPORTANT CAUSES FOR REJECTION OF PAPERS FOR PUBLICATION - contd - Inadequate sample size - Faulty control selection - Drawing unjustified conclusions - Conflict of interest among authors - Lack of clarity in the presentation of article itself Adapted from Greenhalgh T, BMJ 1997; 15: 243 -6

Systematic Error in medical research • Systematic error(also termed as bias)is any systematic error in design , data gathering , analysis , interpretation and dissemination of results that finally lead to under / over estimation of effects of a given exposure on a specific outcome

Systematic Error in medical research - Selection bias - Selected sample-not a rep sample of ref population. - Information bias - gathered information about exposure , outcome or both - not correct and there is an error in measurement.

TYPES OF SELECTION BIAS - Loss to follow up bias - Non response bias or Participation bias - Exclusion bias or Attrition bias - Disease spectrum bias - Volunteer or referral bias or Self selection bias - Incidence-prevalence bias or Neyman bias or Survival bias

TYPES OF SELECTION BIAS (contd) - Publication bias or Dissemination bias Citation bias Friend control bias Compensating bias Berkson’s bias

Types of Information Bias - Recall bias Wish bias or Wynder bias Detection bias Will Rogers Phenomenon Lead time bias Imperfect standard bias Incorporation bias Verification bias and Work up bias Interviewer / Observer bias

• In addition to selection / information bias , certain other phenomena such as Hawthorn effect and Ecological fallacy distract the finding from reality

• Increasing sample size would diminish the systematic errors and associated phenomena to a great extent, though all of these are not completely eradicable or controllable.

High quality epidemiologic study • Aim of any epidemiologic study – to estimate parameters as accurate as possible. • Accuracy-given parameter estimated with lowest possible error (random & systematic) -Valid-estimate with lowest systematic error -Precise – estimate with lowest random error

STATISTICAL ERRORS IN RESEARCH

Design of the study - Statement of aims Sample size Randomization errors Control selection Data analysis -Use of wrong statistical tests Statistical methods applied - Failure to specify or define all tests used clearly and correctly - Wrong names for statistical tests STATISTICAL ERRORS IN RESEARCH Presentation of study data - Inadequate graphical or numerical description of basic data. - Inappropriate and poor reporting of results Interpretation of results - Faulty and Poor interpretation of results

Conflict Of Interest • It exists when the welfare of the patient / research validity is influenced by financial gains / when a researcher feels defensive /faces ethical dilemmas / has competing interest

Conflict Of Interest • Dr Richard Horton , Editor Lancet • There appears to be a constant continuing conflict of interest during the peer reviewing process of journals and sponsors.

Conflict Of Interest A)Primary Interest - Health of patients - Integrity of research - Education of students B)Secondary Interest - Not necessarily illegimate in itself , may be necessary part of professional practice - Personal preferences - Commercial / political - Academic / financial concerns • Note-Secondary interest shouldn’t dominate over Primary interest

IMPACTS OF CONFLICT OF INTEREST - Patient care including informed consent or financial interests - Biomedical research like the distorting research priorities / compromising trial designs to suit institutional / sponsor interest / academic interest.

IMPACTS OF CONFLICT OF INTEREST(contd) • COI presents the risk of undue influence on professional adjustments , thus jeopardizing the integrity of scientific / academic investigations • the objective of medical education , the quality of patient care and the public trust in medicine and healthcare givers This is bound to adversely affect the public health

INSTITUITIONAL POLICY IN COI • Academic institutions conducting research in alliance with industries or commercial companies require a strong review committee to probe possible conflicts of interest between scientific responsibilities of researchers and business interests.

INSTITUITIONAL POLICY IN COI (contd) • Medical institutions to establish responsible and reasonable interest policies and procedures that require disclosure right of both , individual and institutional financial ties to industry. • This will reduce the risk of bias and loss of trust without damaging the fruitful collaborations with industry or similar other sponsors or funding agencies

“It is better to have an appropriate answer to a precise problem than to have an exact answer to a vague question”

“The quality of a research in laboratory is not determined by the elegance of the buildings or the furniture”

• A researcher measuring science in terms of dollars might be more tempted to plagiarize / fabricate data.

• Plagiarism is the “unattributed verbatim or nearly verbatim copying of sentences and paragraphs , which materially mislead the ordinary reader regarding the contributions of the author. ” • “Cut and paste plagiarism”

• Medical ghostwriting occurs when someone makes substantial contributions to a manuscript without attribution or disclosure • It is a scientific misconduct • Eg Wyeth pharma 26 papers in 18 journals between 1998 -2005 to promote “Prepromo”(hormone therapy drug) for menopausal women - Profound rise in breast cancer

“Toughest area of research is to make the outcomes / results akin to the outcomes / results as researcher likes / wants”

Manipulation of Research • Drug companies manipulating research to get favourable outcomes by not measuring critically important “hard” outcomes for patients • E. g. survival vs. death • Instead measure softer outcomes • E. g. self reporting of symptoms (“my pain is less now”)

Manipulation of Research (contd) • Statins reduce bad cholesterol rather than incidence of disease or death itself. • Other contributory factors promoting health ignored , while considering final favourable outcome. • Comparing new drugs against those already known to be inferior to others in the market.

Common pitfalls in research • Exaggeration of a result / selective reporting of results • Disregarding counter evidence • Falsification of data or experiment • Deliberate misinterpretation of an inference and its perpetration • Deliberate distortion of evidence • Publish in journals that are favourable to their interests

Common pitfalls in research(contd) • Testing products against placebo or low dose competitor drugs • Protocol violation • Cherry picking of data • Reporting measurements with unnecessary precision

Common pitfalls in research(contd) • Using figures and tables only to “store” data, rather than assist readers • Not accounting for all data and cases • Not defining “normal” or”abnormal” when reporting diagnostic test results

• In law a man is guilty when he violates the right of others. • In ethics , he is guilty if he only thinks of doing so. -Immanuel Kant

Research study • Not to have any vested interest • Not to exaggerate results • Not to disregard counter evidence • Not to misrepresent or falsify data / inference • Not to cut corner (Deliberate distortion of evidence) • Not for financial gains • Not harmful to participants and society at large

Research study (contd) • Not biased and / or free from wishful thinking • Not to violate copyright , patent , intellectual property rights • No “cut and paste plagiarism” • Not to publish same material in two different journals

Research study should be (contd) • • Transparent Of genuine nature Test result real and not cooked up / fabricated Honest Respect disclosure policy Gives credit to others , if their data is utilized Ethical handling of animals

Subjects should know • Nature , duration , purpose of the experiment / trial • Methods and means , which are to be conducted • Inconveniences and hazards reasonable to be expected • Effects upon their health or person as a result of participation • Right to withdraw from the experiment or trial

Subjects should know (contd) • Protection or preservation of health of subjects • Rights to compensation • Significance of written informed consent

Appropriateness of study design to the research • Clinical trial to test drug effectiveness • Cross sectional study for testing validity of diagnostics or screening tests or risk factor association • Longitudinal cohort study for prognostic studies • Case control study for causal association , especially rare illness

Clinical / Community Relevance • In the case of clinical studies, the potential value in the understanding, diagnosis, or management of a clinical condition or pathological state should be stated. • Funding agencies often now require a statement of community relevance – e. g. , how will the results be translated and disseminated to the target population or community.

Transparency in Research • Rosie Redfield -Microbiologist at University of British Coloumbia in Vancouver performing a novel method in research –Open Science.

ICMR GUIDELINES • Principles of accountability or transparency • Full disclosure is made by those associated with the research or experiment of each aspect of their interest in the research , and any conflict of interest that may exist

ICMR GUIDELINES (contd) • DECISION MAKING PROCESS • A member must voluntarily withdraw from the IEC while making a decision on an application , which evolves a conflict of interest , which should be indicated in writing to the chairperson prior to the review and should be recorded so in the minutes

ICMR GUIDELINES (contd) • Prospective participants in research should also be informed of the sponsorship of research , so that they can be aware of the potential for conflicts of interest and commercial aspects of the research.

LEGISLATIONS • Biomedical and Health Research Regulation Bill, 2013 • Drugs and Cosmetics Act • The Preconception and Prenatal Diagnostic Technique Act • The MTP Act • The Mental Health Act • The Human Organ Transplantation Act

• If a man will begin with certainities he shall end in doubts. • but if he will be content to begin with doubts , he will end in certainities -Francis Bacon

THANK YOU