Scientific Method Introduction Lecture 2 http xkcd com242

  • Slides: 22
Download presentation
Scientific Method — Introduction Lecture 2 http: //xkcd. com/242/

Scientific Method — Introduction Lecture 2 http: //xkcd. com/242/

What is science? United Kingdom • We’re interested in science as an active and

What is science? United Kingdom • We’re interested in science as an active and dynamic activity – how to ‘pursue and apply knowledge and understanding of the natural world’. We should learn do to do this ourselves and be able to evaluate claims by others. To do this, a key component is application of a ‘systematic methodology’ to help you avoid fooling yourself into believing something that is not well-supported • “The first principle [of doing science] is that you must not fool yourself—and you are the easiest person to fool. ” Richard Feynman, Caltech commencement address (1974) http: //www. sciencecouncil. org/definition

What is Science? • We want to do science to better understand the world

What is Science? • We want to do science to better understand the world – and perhaps predict how things will change (by themselves or if we intervene) or apply our understanding of the world to develop new technology (genetic engineering, computers…) • An important reason for wanting to define science is to help recognize science – the stuff that can be useful – from pseudoscience, which is often presented as being scientific, but is not • E. g. , one of each pair is scientific, the other is not (which is which and why? ): ◦ Astrology ◦ Astronomy ◦ Physics ◦ Psychics “Science is the pursuit and application of knowledge and understanding of the natural and social world following a systematic methodology based on evidence” …if this is a good definition of science, what aspect(s) do the two pseudosciences at left fail at?

mean = 69% range over skills = 58 -89% Score (%) Understand elements of

mean = 69% range over skills = 58 -89% Score (%) Understand elements of research design and impact on conclusions Skill Justify inferences, predictions, and conclusions based on quantitative data

Next number in sequence: 2, 3, 3, 5, 10, 13, 39, 43, 172, 177,

Next number in sequence: 2, 3, 3, 5, 10, 13, 39, 43, 172, 177, ? (Online) Rose and Petals game =2 =8 =0 = 14 = = (Allegedly, it took Bill Gates more than two hours to figure it out http: //www. borrett. id. au/computing/petals-bg. htm)

 • Pair up. • One person is given the rule to the following

• Pair up. • One person is given the rule to the following number sequence. The other should guess it. • The guesser should write down a series of three numbers and ask whether it does or does not adhere to the rule. • This can be repeated as many times as you’d like. The are no penalties for being ‘right’ or ‘wrong’ • When the guesser feels that they have determined the ‘rule’ of the game, they can submit that answer. An example of a series of numbers that adhere to the rule: 2, 4, 8

What is the “Scientific Method”? Scientific Method (sometimes called the Hypothetico. Deductive method): 1

What is the “Scientific Method”? Scientific Method (sometimes called the Hypothetico. Deductive method): 1 & 2. Induction: integrate observation/theory to come up with a research question that aims to explain how some aspect of the world probably works — ideally something that: (i) isn’t already understood and (ii) is worth the time it will take to understand. 3. Deduction: translate your question into a testable (i. e. , falsifiable) hypothesis. “If …[I do experiment X], then …[I will see result Y]”. The hypothesis should be falsifiable and specific (e. g. , if I change X by 5%, then Y will change by 10%). 4. Test the hypothesis through experiment. It is not trivial to devise a good test of a hypothesis—i. e. , one that can clearly separate predicted and alternative outcomes, and that avoids complicating factors. 5. Determine the fit of your experiment’s outcome to the predictions made by your hypothesis. This will almost certainly involve some kind of statistical comparison (e. g. , between treatment and control groups). 6. If the outcome of your experiment does not match your prediction, then: (i) your hypothesis is falsified and needs to be revised or (ii) your experiment was unsuitable and needs to be revised. o Bonus point if you can determine what’s wrong with this application of the scientific method: https: //www. youtube. com/watch? v=k 2 Mh. Ms. Ln 9 B 0

Homework (~10 -15 minutes) Sirum and Humburg (2011). Bioscene 37: 8 -16

Homework (~10 -15 minutes) Sirum and Humburg (2011). Bioscene 37: 8 -16

1. Developing a research question — Induction • Coming up with a good research

1. Developing a research question — Induction • Coming up with a good research question is the first, and a very important step in applying the scientific method • A good question integrates existing observations and theory to generate an overarching explanation • Often this process proceeds via induction — defined as: (i) proceeding from the specific to the general or (ii) a claim that a conclusion probably follows from a premise(s) – a statement from which a conclusion is drawn – if the premise is true • For example: Your house window is broken All animals have DNA Your TV is missing All animals have genetic material You’ve probably been burglarized DNA is probably the genetic material inductive conclusion

1. Developing a research question — Induction • You’ll often see an inductive argument

1. Developing a research question — Induction • You’ll often see an inductive argument presented as if the conclusion is true. Beware! An inductive conclusion is only the first step in the scientific method. By definition inductive conclusions are, at best, only probably true. • Some problems of induction (and deduction) that you need to keep in mind (more at: http: //cooperlab. org/biol 4397 - An Illustrated Book of Bad Arguments) – avoiding these kind of problems helps avoid wasting time testing bad conclusions … 1. Inductive leap: A conclusion that initially seems to be certain may not be supported as additional data is collected Every swan I’ve ever seen is white All swans are white http: //xkcd. com/605/ https: //www. rightpet. com/livestock-poultrydetail/black-swan

1. Developing a research question — Induction • 2. Some problems of induction (and

1. Developing a research question — Induction • 2. Some problems of induction (and deduction) that you need to keep in mind Affirming the consequent: Be aware that there will usually (always? ) be more than one explanation for a particular conclusion/observation. Just because ‘A’ *always* leads to ‘B’, it doesn’t follow that observing ‘B’ means that ‘A’ was the cause (perhaps ‘C’ also leads to ‘B’). If I have the flu, I will have a sore throat I have the flu might No! Having the flu is not the only way to have a sore throat

1. Developing a research question — Induction 3. Some problems of induction (and deduction)

1. Developing a research question — Induction 3. Some problems of induction (and deduction) that you need to keep in mind Correlation is not causation: Just because two variables are associated with one another, it doesn’t mean that the two variables directly interact with each other (so that changing one would necessarily change the other) Churches Crime in rate a city • Ice cream Bars in consumption a city http: //xkcd. com/552/

1. Developing a research question — Induction • 3. Some problems of induction (and

1. Developing a research question — Induction • 3. Some problems of induction (and deduction) that you need to keep in mind Correlation is not causation: Just because two variables are associated with one another, it doesn’t mean that the two variables directly interact with each other (so that changing one would necessarily change the other) Eric Cornell, who won the Nobel Prize in Physics in 2001, told Reuters: "I attribute essentially all my success to the very large amount of chocolate that I consume. Personally I feel that milk chocolate makes you stupid… dark chocolate is the way to go. It's one thing if you want a medicine or chemistry Nobel Prize but if you want a physics Nobel Prize it pretty much has got to be dark chocolate. " But when … contacted to elaborate on this comment, he changed his tune. "I deeply regret the rash remarks I made to the media. We scientists should strive to maintain objective neutrality and refrain from declaring our affiliation either with milk chocolate or with dark chocolate, " he said. "Now I ask that the media kindly respect my family's privacy in this difficult time. " New England J Med (2012) http: //www. bbc. com/news/magazine-20356613

1. Developing a research question — Induction • 4. Some problems of induction (and

1. Developing a research question — Induction • 4. Some problems of induction (and deduction) that you need to keep in mind Ascertainment bias (sampling bias): Observations we choose to consider in developing a conclusion may be biased in a way that supports our favorite conclusion (observations that can’t be explained by the conclusion may be ignored or the sampling may be carried out in a way that they are not even collected) nytimes. com http: //philosophy. hku. hk/think/stat/samples. php

Induction – stereotypes • Do national stereotypes (an inductive conclusion derived from many specific

Induction – stereotypes • Do national stereotypes (an inductive conclusion derived from many specific observations) have a ‘kernel of truth’? • Rate an average American for (1 -10 scale; world average = 5): Neuroticism Extroversion Agreeableness Openness Conscientiousness

ne ex uro optravticis m ag ennersio coreeaess n ns bl cie en nt

ne ex uro optravticis m ag ennersio coreeaess n ns bl cie en nt es io s us ne s s Testing an inductive conclusion… Standard personality inventory Character survey – judge an average person of your nationality/culture (very significant similarity in this measure among individuals of a culture) • Stereotypes – asking people to evaluate traits of people they interact with – bore no relation to results of a standard personality inventory test (NEO-PI-R) Terracciano et al. Science 2005

3. Deduction — developing a hypothesis • No matter how good a scientist is,

3. Deduction — developing a hypothesis • No matter how good a scientist is, his/her initial inductive conclusion cannot be accepted as true—even if the conclusion is properly constructed (i. e. , it doesn’t obviously have any of the problems we’ve discussed), a good scientist will recognize that it is just their best guess of several plausible possibilities • This is why it is necessary to devise a hypothesis that is based on our inductive conclusion and test a prediction that necessarily derives from this hypothesis — this is called deduction. If the prediction fails, the hypothesis is invalid. If the hypothesis is invalid, the inductive conclusion was wrong. (With caveats based on the quality of the test. ) • Feynman’s quote (“The first principle [of doing science] is that you must not fool yourself—and you are the easiest person to fool. ”) speaks to this point—after you’ve come up with a great inductive conclusion, it can be tempting to think you’ve finished. In fact, you’re just getting started, now you need to convince yourself that you haven’t been fooled!

3. Deduction — developing a hypothesis Observation Inductive conclusion (probably true) Green plants can

3. Deduction — developing a hypothesis Observation Inductive conclusion (probably true) Green plants can photosynthesize (convert light to chemical energy) • We can translate an inductive conclusion into a hypothesis — an ‘if, then’ statement about how you propose that the world works • A hypothesis should make a prediction that can be falsified — i. e. , shown to be incorrect by an experimental test. The more specific the hypothesis, the more likely it can be falsified (and the more impressive it is when it isn’t) The green pigment is involved in photosynthesis Deductive conclusion (necessarily true or your hypothesis is wrong) – aka, a hypothesis If…the green pigment is required for photosynthesis Then…preventing production of the green pigment will prevent photosynthesis A falsifiable prediction A mutation that prevents production of green pigment will prevent photosynthesis

3. Deduction — developing a hypothesis • Most scientists apply the principle of falsification

3. Deduction — developing a hypothesis • Most scientists apply the principle of falsification to test their hypotheses – an idea put forward by Sir Karl Popper in 1934 • Popper was concerned that some of the popular scientific theories of the early 20 th century were pseudoscience, not real science. In particular, he was worried that people who believed these theories found supporting evidence everywhere they looked – almost any observation could be interpreted in a way that confirmed theory, no observation would prove it wrong. • Popper describes meeting a famous psychologist, Alfred Adler (who had developed a theory of ‘individual psychology’): …I reported to [Adler] a case which to me did not seem particularly Adlerian, but which he found no difficulty in analyzing in terms of his theory of inferiority feelings. … Slightly shocked, I asked him how he could be so sure. "Because of my thousandfold experience, " he replied; whereupon I could not help saying: "And with this new case, I suppose, your experience has become thousand-one-fold. ” What I had in mind was that his previous observations may not have been much sounder than this new one; that each in its turn had been interpreted in the light of "previous experience, " and at the same time counted as additional confirmation. I could not think of any human behavior which could not be interpreted in terms of [Adler’s] theory. It was precisely this fact— that they [observations] always fitted, that they were always confirmed—which in the eyes of their admirers constituted the strongest argument in favor of these theories. It began to dawn on me that this apparent strength was in fact their weakness. why is this a weakness? Karl Popper, Conjectures and refutations (1963)

3. Deduction — developing a hypothesis • Popper’s idea was that for a hypothesis

3. Deduction — developing a hypothesis • Popper’s idea was that for a hypothesis to be useful, it has to make a prediction that can be shown to be wrong — that is, falsified (there needs to be a consequence to being wrong, you can’t weasel out of it by saying: “Well, actually, my hypothesis can explain that result, even though I wasn’t expecting it. Yippee. What a great and flexible hypothesis. ”) • The prediction may be difficult to test — perhaps not being possible at the current time. Nevertheless, that a hypothesis makes a prediction that can, at least in principle, be tested and falsified is generally considered to be essential for it to be considered scientific 48 yrs ~$13. 25 bn Prediction: existence of an as yet unobserved sub-atomic particle with a mass of ~125 Ge. V (note that the prediction is specific, a good thing) Phys. Lett. B 716: 1 -29 (2012) trbimg. com/img-52 b 49 c 18/turbine/la-sci-sn-top-science-stories-of-2013 -gallery--004/968

What’s the problem? https: //www. youtube. com/watch? v=X 2 xl. Qaims. Gg#t=20 s Witches

What’s the problem? https: //www. youtube. com/watch? v=X 2 xl. Qaims. Gg#t=20 s Witches burn Wood burns Witches are wood How do we know if the person is made of wood? Wood floats Ducks float Witches weigh the same as ducks