Statistical data analysis and research methods BMI 504

Statistical data analysis and research methods BMI 504 Course 22903 – Spring 2017 Class 4 – Feb 22, 2017 Introduction to data analysis of quantitative and qualitative variables Werner CEUSTERS

C 4. Introduction to data analysis of quantitative and qualitative variables • Pre-class readings: 1. Savitri Abeyasekera. Quantitative analysis approaches to qualitative data: why, when and how? 2. John PA Ioannidis. Why Most Published Research Findings Are False. • Class structure: • interactive lecture: • application: analysis of C 2 in-class application test (if time) • Post-class assignment: none. • Assessment: 1. pre-lecture test on readings, 2. participation in interactive lecture.

Part 1 Interactive lecture

Steps in data analysis (1) 1. Specify the question you are asking. 2. Put the question in the form of a null hypothesis and alternative hypothesis. 3. Put the question in the form of a statistical null hypothesis and alternative hypothesis. 4. Determine which variables are relevant to the question. 5. Determine what kind of variable each one is. 6. Design an experiment that controls or randomizes the confounding variables. 7. Based on the number of variables, the kinds of variables, the expected fit to the parametric assumptions, and the hypothesis to be tested, choose the best statistical test to use. John H. Mc. Donald. The Handbook of Biological Statistics ©. 2014. http: //www. biostathandbook. com/

Steps in data analysis (2) 8. If possible, do a power analysis to determine a good sample size for the experiment. 9. Do the experiment. 10. Examine the data to see if it meets the assumptions of the statistical test you chose. If it doesn't, choose a more appropriate test. 11. Apply the statistical test you chose, and interpret the results. 12. Communicate your results effectively, usually with a graph or table. John H. Mc. Donald. The Handbook of Biological Statistics ©. 2014. http: //www. biostathandbook. com/

This class 1. Specify the question you are asking. 2. Put the question in the form of a null hypothesis and alternative hypothesis. 3. Put the question in the form of a statistical null hypothesis and alternative hypothesis. 4. Determine which variables are relevant to the question. 5. Determine what kind of variable each one is. 6. Design an experiment that controls or randomizes the confounding variables. 7. Based on the number of variables, the kinds of variables, the expected fit to the parametric assumptions, and the hypothesis to be tested, choose the best statistical test to use. John H. Mc. Donald. The Handbook of Biological Statistics ©. 2014. http: //www. biostathandbook. com/

Steps in data analysis (1. 1) 1. Specify the question you are asking: • Does physical exercise influences heart rate ? or ?

Steps in data analysis (1. 1) 1. Specify the question you are asking: • Does physical exercise influences heart rate ? or ? Does physical exercise increases heart rate ? • Does doing physical exercise influences heart rate while doing the exercise? • Does doing regular physical exercise influences heart rate at rest? •

Steps in data analysis (1. 2) 1. Specify the question you are asking. • Does doing physical exercise influences heart rate while doing the exercise? 2. Put the question in the form of a null hypothesis and alternative hypothesis: • Null hypothesis: • • Alternative hypothesis: •

Steps in data analysis (1. 2) 1. Specify the question you are asking. • Does doing physical exercise influences heart rate while doing the exercise? 2. Put the question in the form of a null hypothesis and alternative hypothesis: • Null hypothesis: • Physical exercise does not influence heart rate while doing the exercise. • Alternative hypothesis: • Physical exercise does influence heart rate while doing the exercise.

Steps in data analysis (1. 3) 1. Specify the question you are asking. • Does doing physical exercise influences heart rate while doing the exercise? 2. Put the question in the form of a null hypothesis and alternate hypothesis. • Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Put the question in the form of a statistical null hypothesis and alternative hypothesis.

Statistical hypothesis • A statistical hypothesis is an assumption about a ‘population parameter’, i. e. a measurable characteristic of a population, such as a mean or a standard deviation. • This assumption may or may not be true. • Hypothesis testing refers to the formal procedures used by statisticians to accept or reject statistical hypotheses. • The best way to determine whether a statistical hypothesis is true would be to examine the entire population. Since that is often impractical, researchers typically examine a random sample from the population. If sample data are not consistent with the statistical hypothesis, the hypothesis is rejected. http: //stattrek. com/hypothesis-testing. aspx

Two types of statistical hypotheses • Statistical null hypothesis. • the hypothesis that sample observations result purely from chance; • notation: H 0 • Statistical alternative hypothesis. • the hypothesis that sample observations are influenced by some non-random cause; • notation: H 1 or Ha. http: //stattrek. com/hypothesis-testing. aspx

Steps in data analysis (1. 3) 1. Specify the question you are asking. • Does doing physical exercise influences heart rate while doing the exercise? 2. Put the question in the form of a null hypothesis and alternate hypothesis. • Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Put the question in the form of a statistical null hypothesis and alternative hypothesis. • Statistical null hypothesis:

Steps in data analysis (1. 3) 1. Specify the question you are asking. • Does doing physical exercise influences heart rate while doing the exercise? 2. Put the question in the form of a null hypothesis and alternate hypothesis. • Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Put the question in the form of a statistical null hypothesis and alternative hypothesis. • Statistical null hypothesis: the average heart rate observed in a sample of individuals working out is the same as in a sample of individuals being at rest.

Steps in data analysis (1. 3) 1. Specify the question you are asking. increases • Does doing physical exercise influences heart rate while doing the exercise? 2. Put the question in the form of a null hypothesis and alternate hypothesis. increase • Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Put the question in the form of a statistical null hypothesis and alternative hypothesis. • Statistical null hypothesis: the average heart rate observed in … ?

Steps in data analysis (1. 3) 1. Specify the question you are asking. • Does doing physical exercise influences heart rate while doing the exercise? 2. Put the question in the form of a null hypothesis and alternate hypothesis. • Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Put the question in the form of a statistical null hypothesis and alternative hypothesis. • Statistical null hypothesis: the average heart rate observed in a sample of individuals working out is the same as in a sample of individuals being at rest. ?

Remember ontology: what is out there ? portions of reality relations configurations participation universals me participating in my life entities particulars continuants me organism occurrents my life

Data and Reality Referents Window on reality ? ? ‘Average heart rate’ References ?

Steps in data analysis (1. 4) 1. Specify the question you are asking. • Does doing physical exercise influences heart rate while doing the exercise? 2. Put the question in the form of a null hypothesis and alternate hypothesis. • Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Put the question in the form of a statistical null hypothesis and alternative hypothesis. • Statistical null hypothesis: the average heart rate observed in a sample of individuals working out is the same as in a sample of individuals being at rest. 4. Determine which variables are relevant to the question.

‘Variable’

How about this? http: //www. sjsu. edu/people/mark. vanselst/courses/psyc 120/s 2/Cosby-c 4 -Research-Fundamentals. pdf

Hint: would I like this? https: //www. khanacademy. org/math/algebra/introduction-to-algebra/alg 1 -intro-to-variables/v/what-is-a-variable

Why I don’t agree with this. http: //www. sjsu. edu/people/mark. vanselst/courses/psyc 120/s 2/Cosby-c 4 -Research-Fundamentals. pdf

‘Variable’ How the word should be used (my tentative definition grounded in realism-based ontology): Symbol used by an author to denote some pre-defined perspective P on portions of reality (Po. R) whereby P allows these Po. Rs to be described in a comparable and standardized way.

‘Pre-defined perspectives’: concept • ‘Concept’ • A perspective drawn from observing a number of Po. Rs in a relatively similar, comparable and standardized way.

‘Pre-defined perspectives’: concept • ‘Concept’ • A perspective drawn from observing a number of Po. Rs in a relatively similar, comparable and standardized way. ‘The critical term here is “observed”, because it means that there is a direct link between the concept (the abstraction) and its referents (the reality). For instance, we can observe a number of particular instances where individuals receive varying amounts of money for the work they have done over a given period of time. From these particulars we distill an abstraction and label it “income”. Similarly, we observe individuals and find some of them short, some tall and more of them in between; from these observations we generate the concept “height”. ’ James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

‘Pre-defined perspectives’: construct • ‘Construct’ • A perspective built from the logical combination of a number of concepts.

‘Pre-defined perspectives’: construct • ‘Construct’ • A perspective built from the logical combination of a number of concepts. James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002. Part 1, Chapter 2, Elements of Scientific Theories: Concepts and Definitions. P 12. http: //www. cios. org/readbook/rmc s/rmcs. htm

‘Perspective’ • ‘Construct’ • A perspective built from the logical combination of a number of concepts. • ‘Concept’ • A perspective drawn from observing a number of Po. Rs in a relatively similar, comparable and standardized way. • ‘Perspective’ • A Cognitive Representation resulting from an interpretive process driven by relatively systematic observations of a Po. R.

Cognitive Representation INFORMATION CONTENT ENTITY An ENTITY which is (1) GENERICALLY DEPENDENT on (2) some MATERIAL ENTITY and which is (3) concretized by a QUALITY (a) inhering in the MATERIAL ENTITY and (b) that is_about some PORTION OF REALITY INFORMATION QUALITY ENTITY A REPRESENTATION that is the concretization of some INFORMATION CONTENT ENTITY REPRESENTATION A QUALITY which is_about or is intended to be about a PORTION OF REALITY MENTAL QUALITY A QUALITY which specifically depends on an ANATOMICAL STRUCTURE in the cognitive system of an organism COGNITIVE REPRESENTATION A REPRESENTATION which is a MENTAL QUALITY Smith B, Ceusters W. Aboutness: Towards Foundations for the Information Artifact Ontology. In: Proceedings of the Sixth International Conference on Biomedical Ontology: July 27 -30, 2015; Lisboa, Portugal. 2015. Available at: http: //ceur-ws. org/Vol-1515/regular 10. pdf. Accessed 24 Aug 2016.

Pre-defined perspectives • Perspective label: • Short term that functions as a name to identify the perspective in communications about the research. • Theoretical definition: • Description that specifies what sort of Po. R the defining researcher carved out through the perspective. • Operational definition: • Description intended to allow other researchers • (a) to observe (in case of concepts) the intended Po. Rs from the very same perspective objectively or, • (b) to build (in case of constructs) the perspective in exactly the same way as intended. Adapted from: James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

Steps in data analysis (1. 4) 1. Research question: Does doing physical exercise influences heart rate while doing the exercise? 2. Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Statistical null hypothesis: the average heart rate observed in a sample of individuals working out is the same as in a sample of individuals being at rest. 4. Relevant variables:

Steps in data analysis (1. 4) 1. Research question: Does doing physical exercise influences heart rate while doing the exercise? 2. Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Statistical null hypothesis: the average heart rate observed in a sample of individuals working out is the same as in a sample of individuals being at rest. 4. Relevant variables: ‘heart rate’, ‘physical exercise’

Steps in data analysis (1. 5) 1. Research question: Does doing physical exercise influences heart rate while doing the exercise? 2. Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Statistical null hypothesis: the average heart rate observed in a sample of individuals working out is the same as in a sample of individuals being at rest. 4. Relevant variables: ‘heart rate’, ‘physical exercise’. 5. Determine what kind of variable each one is.

Kinds of variables • Can be classified along several axes, but most importantly: • Dependent versus independent, • Level of measurement, • Discrete or continuous.

Variables & level of measurement • Nominal (=categorical): • e. g. : shape = round, square, oval, … • Only comparison is (in)equality, • Special type: binary. • Ordinal (=rank): • Ranking order between values, • e. g. : like most, like somewhat, neutral, dislike somewhat, … • Interval: • Ranked with equal distance between values, • Null-point is arbitrary. • e. g. : temperature in Celsius or Fahrenheit. • Ratio: • Ratios between pairs of values are meaningful, • Absolute null-point. • e. g. : temperature in Kelvin.

Levels of measurement http: //www. mymarketresearchmethods. com/types-of-data-nominal-ordinal-interval-ratio/

Time ?

Time • Nominal: • AM versus PM (binary) • Ordinal: • Night – dawn – morning – noon – afternoon – dusk • Interval: • 12 AM, 1 AM, 2 AM, … • Ratio: • Take 12 AM as absolute zero.

Which one to select?

Pre-defined perspectives • Perspective label: • Short term that functions as a name to identify the perspective in communications about the research. • Theoretical definition: • Description that specifies what sort of Po. R the defining researcher carved out through the perspective. • Operational definition: • Description intended to allow other researchers • (a) to observe (in case of concepts) the intended Po. Rs from the very same perspective objectively or, • (b) to build (in case of constructs) the perspective in exactly the same way as intended. Adapted from: James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

Requirements for operational definitions • Must contain instructions as precisely as possible for observations/calculations to be done repeatable and objectively; • Must specify: • how observations are to be done, and calculations or logical combinations to be performed, • units of measurements, • level of measurement: • categorical , … ? • Correspondence between theoretical and operational definition establishes the validity of the measurements. Adapted from: James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

Example • Variable: SPPC • Label: Satisfaction with Primary Physician Communication • Operational definition 1 (nominal level of measurement) • Response to the question: Do you feel that most of the time your communication with your primary physician is satisfactory? • (a) YES or (b) NO • Operational definition 2 (ordinal level of measurement) • Response to the question: Do you feel that most of the time your communication with your primary physician is: (a) very satisfactory (b) somewhat satisfactory (c) neither Adapted from: (d) somewhat unsatisfactory (e) very unsatisfactory James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

Example • Variable: SPPC exhibits variation in values • Label: Satisfaction with Primary Physician Communication • Operational definition 1 (nominal level of measurement) • Response to the question: Do you feel that most of the time your communication with your primary physician is satisfactory? • (a) YES or (b) NO • Operational definition 2 (ordinal level of measurement) • Response to the question: Do you feel that most of the time your communication with your primary physician is: (a) very satisfactory (b) somewhat satisfactory (c) neither Adapted from: (d) somewhat unsatisfactory (e) very unsatisfactory James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

Relationship with reality Referents Window on reality References Portions of Reality ‘in focus’ unit of analysis Perspective Label Variable Values Definitions: Face validity Adapted from: Theoretical definition Operational definition Measurement validity James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

Operational definition Is. A …? A. Bandrowski et. Al. The Ontology for Biomedical Investigations. Plos. One. 2016. http: //dx. doi. org/10. 1371/journal. pone. 0154556. g 001

Operational definition Is. A …? Operational Definition

‘Interview age’ ?

‘Interview age’ ? https: //ndar. nih. gov/data_structure. html? short_name=grit 01

‘Interview age’ under the Grit perspective • Label: interview age • Theoretical definition (? ): Age in months at the time of the interview / test / sampling / imaging. • Operational definition (? ): Age is rounded to chronological month. If the research participant is 15 -days-old at time of interview, the appropriate value would be 0 months. If the participant is 16 -days-old, the value would be 1 month. Range: 0 -1260 [0 – 105 years] https: //ndar. nih. gov/data_structure. html? short_name=grit 01

https: //ndar. nih. gov/data_structure. html? short_name=grit 01

Steps in data analysis (1. 5) 1. Research question: Does doing physical exercise influences heart rate while doing the exercise? 2. Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Statistical null hypothesis: the average heart rate observed in a sample of individuals working out is the same as in a sample of individuals being at rest. 4. Relevant variables: ‘heart rate’, ‘physical exercise’. 5. Determine what kind of variable each one is:

Steps in data analysis (1. 5) 1. Research question: Does doing physical exercise influences heart rate while doing the exercise? 2. Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Statistical null hypothesis: the average heart rate observed in a sample of individuals working out is the same as in a sample of individuals being at rest. 4. Relevant variables: ‘heart rate’, ‘physical exercise’. 5. Determine what kind of variable each one is: • heart rate: ratio • physical exercise: ? ? ?

Operational definition for ‘heart rate’ • Variable: HR • Label: Heart rate • Operational definition: (interval level of measurement) • Heart rate of a subject connected to a heart rate monitor as displayed on that monitor at a specific moment in time. • Values: integers http: //www. surgeryencyclopedia. com/A-Ce/Cardiac-Monitor. html

Operational definition for ‘physical exercise’ • Variable: PE • Label: Physical exercise • Operational definition: (interval level of measurement) • Level of resistance at which the elliptical on which the subject is running is set at a specific moment in time. • Values: integers

Steps in data analysis (1. 6) 1. Research question: Does doing physical exercise influences heart rate while doing the exercise? 2. Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Statistical null hypothesis: the average heart rate observed in a sample of individuals working out is the same as in a sample of individuals being at rest. 4. Relevant variables: ‘heart rate’, ‘physical exercise’. 5. Determine what kind of variable each one is: • heart rate: ratio • physical exercise: ? ? ? 6. ‘Design an experiment that controls or randomizes the confounding variables’.

Relationships between variables 1. No relationship: ‘person length’ ‘body temperature’ temperature 2. Covariance relationship: a change in one variable is associated with a change in another variable (in whatever way) ‘person length’ + ‘person mass’ 3. Causal relationship: a change in one variable brings about a change in another variable (in whatever way) ‘physical exercise’ cause / independent variable + ‘heart rate’ effect / dependent variable

Experiment • Non-experimental design: • compare people who signed up for a program vs. people who did not sign up for any program. • people who sign up may be different from those who don’t sign up. • Experimental design: • Researcher randomly assigns people with alcoholism to treatment or no treatment • Determination of cause and effect variable.

Remember the definition of ‘research’

Remember the definition of ‘research’ keywords 1. careful or diligent search 2. studious inquiry or examination; especially : investigation or experimentation aimed at - the discovery and interpretation of facts, - revision of accepted theories or laws in the light of new facts, - or practical application of such new or revised theories or laws 3. the collecting of information about a particular subject http: //www. merriam-webster. com/dictionary/research

‘Scientific Laws’: a matter of relationships Referents Window on reality References Portions of Reality ‘in focus’ Perspective Variable Values How are these related? ‘Physical Exercise’ Working out Relationship in reality (‘a priori relationship’) Heart rate Relationships between variables ‘Heart rate’

Heart rate during exercise trained/untrained http: //the-cardiorespiratory-system. weebly. com/heart-rate. html

Remember: steps in data analysis (1. 3) 1. Specify the question you are asking. If the statistical null hypothesis is confirmed, is that • Does doing physical exercise influences heart rate while enough for the null hypothesis to be true? doingevidence the exercise? 2. Put the question in the form of a null hypothesis and alternate hypothesis. • Null hypothesis: Physical exercise does not influence heart rate while doing the exercise. 3. Put the question in the form of a statistical null hypothesis and alternative hypothesis. • Statistical null hypothesis: the average heart rate observed in a sample of individuals working out is the same as in a sample of individuals being at rest.

Heart rate during exercise trained/untrained These samples will likely produce the same average heart rate http: //the-cardiorespiratory-system. weebly. com/heart-rate. html

Observed relationships and reality Referents Window on reality + Physical exercise ‘Physical exercise’ + + Heart efficiency References + + Heart rate under effort Heart rate at rest ‘Heart rate’

Observed relationships and reality Referents Window on reality + Physical exercise ‘Physical exercise’ + + Heart efficiency theory References + + Heart rate under effort Heart rate at rest ‘Heart rate’

Observed relationships and reality Referents Window on reality References The lesser preselection of tested relationships, the less likely research results are true.

What else from is relevant here? 1. Where there is smaller Studies are lessflexibility likely in designs true when effect sizes are smaller and definitions; 2. Where there is greater flexibility in outcomes and analytical modes http: //the-cardiorespiratory-system. weebly. com/heart-rate. html

What else from is relevant here? Studies are less likely true when there is greater flexibility in designs, definitions, outcomes and analytical modes http: //the-cardiorespiratory-system. weebly. com/heart-rate. html

Requirements for causal relationships 1. Spatiotemporal contiguity: variables must be observed within units of analysis. Adapted from: James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

Requirements for causal relationships 1. Spatiotemporal contiguity: variables must be observed within units of analysis. The units of analysis within which effort intensity and heart rate are observed are … specific human beings (or animals)

Requirements for causal relationships 1. Spatiotemporal contiguity: variables must be observed within units of analysis. 2. Covariance: the values of two variables must shift together in some systematic way. Adapted from: James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

Requirements for causal relationships 1. Spatiotemporal contiguity: variables must be observed within units of analysis. 2. Covariance: the values of two variables must shift together in some systematic way. Ekblom, B. P. -O. Åstrand, B. Saltin, J. Stenberg and B. Wallström. Effect of training on circulatory response to exercise. J. Appl. Physiol. 24: 518 -528, 1968. Adapted from: James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

Requirements for causal relationships 1. Spatiotemporal contiguity: variables must be observed within units of analysis. 2. Covariance: the values of two variables must shift together in some systematic way. 3. Temporal ordering: a change in the cause variable must happen before the related change in the effect variable. 4. Necessary connection: A statement – the ‘theoretical linkage’ or ‘theoretical rationale’ providing the justification for the posited causation within theory. Adapted from: James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

‘Theoretical definition’ and ‘Theoretical linkage’ • The ‘theoretical definition’: • Is a …

‘Theoretical definition’ and ‘Theoretical linkage’ • The ‘theoretical definition’: • is a … description that specifies what sort of Po. R the defining researcher carved out through the perspective. • explains the nature of a Po. R observed through a perspective. • The ‘theoretical linkage’: • provides the justification for the posited causation within theory; • explains, as the product of a rational process, the nature of a relationship between two concepts/ constructs.

‘Operational Linkage’ • The operational linkage provides the means of testing the probable truth or falsity of the corresponding relationship between variables. • This distinguishes a scientific relationship, which must be shown to be true in the “measurement world” of observations, from an “a priori” relationship. • The operational linkage allows us to determine objectively if two concepts/constructs covary, by telling us how measured changes in one variable should be associated with measured changes in the other. • By determining if the two variables covary in the way described by the operational linkage, we can test the probable truth of the scientific relationship. Adapted from: James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

‘Operational Linkage’ http: //www. biostathandbook. com/linearregression. html This operational linkage tells us that a linear, positive relationship will be observed between the speed at which a person runs on an elliptical and the heart rate: as speed increases, heart rate will increase at a steady rate. Adapted from: James H. Watt and Sjef van den Berg. Research Methods For Communication Science. 2002 http: //www. cios. org/readbook/rmcs. htm

From operational linkage to hypothesis • The operational linkage predicts the way in which the measured variables should behave. Its test of truth goes beyond simple logical deduction, and must include observation of the real world. • The vehicles for this observation are the operational definitions of variables in the “measurement world. ” • We derive hypotheses from the operational linkages. • Hypotheses are verbal statements that specify how the variables, as defined by operational definitions, should be associated with one another if theoretical linkage is, in fact, correct. • If they don’t do so, they should be revised!

Pre-lecture reading test

Why Most Published Research Findings Are False. John P. A. Ioannidis. Why Most Published Research Findings Are False. PLOS Medicine 2005; 2(8): e 124 http: //robotics. tamu. edu /RSS 2015 Negative. Results /pmed. 0020124. pdf

A. The probability that a research claim is true may depend on: 1. 2. 3. 4. study power; bias; the number of other studies on the same question; the ratio of true to no relationships among the relationships probed in each scientific field. Answer form: Scoring: 1 x, 2 x, …, 4 x where x = T(rue)/F(alse)/D(on’t know) 4*roundup((TP-FP+TN-FN)/(TP+TN))

B. A research finding is less likely to be true: 1. 2. 3. 4. 5. 6. 7. 8. When the studies conducted in a field are smaller; When effect sizes are smaller; When there is a smaller number of tested relationships; When there is a lesser preselection of tested relationships; Where there is smaller flexibility in designs and definitions; Where there is greater flexibility in outcomes and analytical modes; When there is greater financial interest; When less teams are involved in a scientific field in chase of statistical significance. Answer form: Scoring: 1 x, 2 x, …, 8 x where x = T(rue)/F(alse)/D(on’t know) 8*roundup((TP-FP+TN-FN)/(TP+TN))

C. Which statements are true? 1. For most study designs and settings, it is more likely for a research claim to be false than true. 2. Research is most appropriately represented and summarized by p-values. 3. Repeated independent testing by different teams of investigators may lead to greater probabilities of the research findings being true. Answer form: Scoring: 1 x, 2 x, …, 3 x where x = T(rue)/F(alse)/D(on’t know) 3*roundup((TP-FP+TN-FN)/(TP+TN))

D. Why use quantitative approaches in qualitative research? 1. To draw meaningful results from a large body of qualitative data; 2. To separate out confounding factors; 3. To discover hidden features of phenomena that were not discussed in e. g. interviews of focus groups; 4. To allow the reporting of summary results in numerical terms to be given with a specified degree of confidence; Answer form: Scoring: 1 x, 2 x, …, 4 x where x = T(rue)/F(alse)/D(on’t know) 4*roundup((TP-FP+TN-FN)/(TP+TN))

E. Ranked products: which statements are true? Honest responses 1 = liked the most … 5 = liked the least Patient Dr 1 Dr 2 Dr 3 Dr 4 Dr 5 A 1 4 2 5 3 B 1 3 2 5 4 C 2 3 4 1 5 1. Dr. 1 is generally the most liked; 2. There is strong evidence that when A likes a specific doctor, B likes that one too; 3. It is possible that A likes Dr. 4 as much as C likes Dr. 4; 4. There is strong evidence that C likes Dr. 4 more over Dr. 3 than he likes Dr. 1 over Dr. 2; 5. There is strong evidence that C likes Dr. 4 more over Dr. 3 than he likes Dr. 3 over Dr. 5; Answer form: Scoring: 1 x, 2 x, …, 5 x where x = T(rue)/F(alse)/D(on’t know) 5*roundup((TP-FP+TN-FN)/(TP+TN))