1 Outline Case Studies Testimonials Case studies as

1 Outline • Case Studies & Testimonials • Case studies as evidence • The placebo effect • The vividness issue • Correlation & Causation • Third variable problem • Directionality • Selection bias

2 Case studies & testimonials • Case studies • in-depth analyses of particular individuals • cannot be used to establish causal relations • Testimonials • positive reviews of a person or product

3 Case studies as evidence • • Historically important in Psychology Used by Freud Psychophysics Complex system can go wrong in complex ways – adequate description may require such lengthy observation that case study is only appropriate approach, at least initially

4 Case studies as evidence • Most useful in early stages of research in any area • Useful for development of theories & hypotheses • Observation may lead to identification of relevant variables that are controlling or influencing behavior - leading to theory building

5 Case studies as evidence • Less useful for testing theories • Case studies cannot rule out alternative hypotheses • Testing theories requires a manipulation – changing conditions under which people perform a task to see if variation in performance predicted by a theory occurs • That process lets you pit one theory against another, and see which (if either) wins

6 Case studies as evidence • The critical idea here is that there can be competing accounts of why something happened. • Our task as scientists is to compare competing accounts and select one.

7 Case studies as evidence • If we are to adjudicate between competing theories, we have to put them in jeopardy. • Jeopardy means that we give all the competing theories a chance to be wrong. • If one of them isn’t wrong in that situation, but the others are, then we reject those others and adopt the one that survived the test. • Manipulation based on theoretical predictions creates the required situation

8 Case studies as evidence • We have to discriminate between alternative accounts • If there isn’t a manipulation that lets you adjudicate between two theories – that is, if they don’t make different predictions about performance – then you don’t have two theories, you only have one. • So, whenever you truly have two theories, you must be able to test them. But not with case studies.

9 Case studies as evidence • The problem with case studies is that they don’t let you create the jeopardy situation • One reasonable account of any improvement in the condition of a case study subject is the placebo effect. • A second issue is that observations in case studies can be biased by the vividness effect.

10 Placebo Effect • Placebo effect: tendency to report that a treatment has helped, regardless of whether it has a real therapeutic effect. • Control group gets a fake treatment • Experimental group gets the real treatment

11 Placebo Effect • If the two groups show the same amount of improvement, then the improvement is due to placebo effect. • If control group shows any improvement, part of treatment effect is placebo effect.

12 Placebo effect • This is about the power of belief: e. g. , physicians say, when a new drug is available, “Hurry up and use it while it still works” • Hawthorne Effect – people’s performance got better when other people were being observed – possibly because “someone cares” • Turing Test – a computer program is intelligent if you can’t tell whether you’re talking to human or computer; problem – people see meaning

13 Placebo effects • Bower (1996): Placebo effect of Prozac twice as large as effect of the drug itself. • Greenberg & Fisher • The limits of biological treatments for psychological distress: comparisons with psychotherapy & panacea (1989) • From placebo to panacea: Putting psychiatric drugs to the test (1997)

14 Placebo effects • Benefits of psychoactive drugs are largely due to placebo effects • Brown (Sci. Am 1998): expectations have real, measurable effects – prescribe placebos!

15 Placebo effects- conclusions • Placebo effects are • You cannot rule out widespread, so they placebo effects in are always present as case studies. an alternative account • People who got the of why someone got benefit of placebo better when they got a effects may offer therapy. testimonials, not knowing that it wasn’t their therapy that made the difference.

16 The Vividness Issue • Retrieval from memory is influenced by vividness. • E. g. : which is safer, travel by car or travel by airplane? • Statistically, travel by plane is much safer, but many people think it is more dangerous. Why? • One account: reports of airplane crashes are vivid, so easily retrieved from memory – ease biases estimate of whether flying is safe.

17 The Vividness Issue • Stanovich: • <680 car, truck, & motorcycle accident deaths/wk in the U. S. • But (Stanovich says) people don’t call for action because they don’t recognize the problem • Lots of reports of one or two deaths aren’t as vivid as one report of an airplane crash with hundreds of deaths

18 Criticism: • In the case of plane vs. car travel, Stanovich doesn’t know that the traffic safety problem is not recognized – that’s just his opinion. • He only knows that people don’t respond as dramatically to traffic deaths as they would to similar numbers of plane crash deaths • But there’s a difference between not calling for investigations and not responding…

19 Criticism • Gerald Wilde, Queen’s University – homeostatic theory of risk: people set an acceptable risk rate for activities like driving • If driving conditions change in such a way as to move drivers away from their preferred rate, they adjust their behavior to move back to that rate.

20 Criticism • e. g. , better brakes? Drive faster. • speed limit reduced? Pay less attention • people are sensitive to driving risks – their behavior does vary in ways that are sensitive to risks • but they may not be aware of that variation in their behavior or be able to articulate those risks as statistics.

21 Criticism • Perhaps professors (including Stanovich) overvalue the ability to articulate knowledge and undervalue the ability to modulate behavior without such knowledge.

22 The Vividness Issue • In the new (8 th) edition • He now says that he of his book, Stanovich uses vivid examples has added a to make his points qualification to his memorable, but he comments about also provides citations vividness, to scientific evidence acknowledging that he for those points. The uses vivid examples latter constitute himself. “proof” of those points.

23 The Vividness Issue • Stanovich says that • But will this be the he uses vivid case? Will his examples only distinction between because they are “illustration” and memorable – so in his “proof” be as mind, presumably, the important to readers reader will be as it is to him? persuaded by the cited scientific studies, not the vivid examples

24 The Vividness Issue • Let’s distinguish between the ideas involved: 1. Do vivid examples prove claims? 2. Are ordinary people unsophisticated consumers of evidence? 3. Does vividness play any useful role in our thinking?

25 1. Do vivid examples prove claims? • No. As a technical point, Stanovich is quite right on this score.

2. Are ordinary people unsophisticated consumers of evidence? • No. As Wilde’s homeostatic theory of risk, and the evidence he reports, suggest, ordinary people respond rationally to changes in conditions, though they may not be able to articulate those changes or the basis for their responses. • The “wisdom of crowds” 26

3. Does vividness play any useful role in our thinking? • Our liking for vivid examples – which Stanovich himself takes advantage of – reflects a basic fact about human cognition: • Individual examples help humans to comprehend events or ideas that are otherwise beyond understanding. As Stalin said, “ 1 death is a tragedy. 1 million deaths is a statistic. ” 27

3. Does vividness play any useful role in our thinking? • As Stalin said, “ 1 death is a tragedy. 1 million deaths is a statistic. ” • To illustrate this point, let’s use as an example the Holocaust… 28

29 The Holocaust • 6 million Jews, and many other people (Gypsies, gays) were murdered in Germany and German-occupied territories during World War II. • A terrible thing – but how terrible? • We don’t have any measuring scale that lets us answer that question. • Does it help to learn about one victim?

30 Irène Némirovsky • Russian-born Jewish writer • Moved to France as a child, in 1919, following Russian Revolution • Published first novel in 1929; celebrated as a brilliant author • She was refused French citizenship in 1938. • Left Paris after German occupation of France in 1940, went to live in a rural village with husband & 2 daughters • Continued writing

31 Irène Némirovsky • Paper was scarce, so she wrote in tiny letters, with a magnifying glass • Arrested by French police in July 1942 in front of her two young daughters. • Wrote a novel about the German occupation of France • Told the daughters she was going away on a trip

32 Irène Némirovsky • Died of typhus in a • Daughters guarded by German concentration Catholic nuns & camp one month later priests through the war, and provided for • Husband arrested, by Irène’s publisher & died in Auschwitz gas a friend during and chamber in November after the war 1942

33 Irène Némirovsky • Daughters survived the war, kept a suitcase containing their mothers’ last writings. • 30 years later, daughter Dianne spent 20 years transcribing Irène’s novel Suite Francaise, working with a magnifying glass. • Book was published in France in 2004, and a tremendous success – though Némirovsky had been long forgotten. • Published in Canada in April 2007

34 Vividness • So, which helps you understand the horror of the Holocaust better – the statistic, “ 6 million murdered” – or the story of Irène Némirovsky? • Both are important, but in different ways • So, vividness can be an important aid to thought • And if vivid examples are filtered through many people’s evaluations, perhaps that process produces a useful conclusion

35 Correlation and Causation • Basic point: Correlation does not imply causation • Goldberger example: cause of pellagra • Goldberger used a manipulation to test causal accounts • He put the germ theory in jeopardy – and showed that it was wrong • He also put his diet theory in jeopardy – and showed that it was right.

36 Correlation and causation • Third variable problem • Directionality • Selection bias

37 Third variable problem • Relationship between two variables A and B might be produced by a third variable which influences both. • E. g. , correlation between # of births in German cities and # of storks in German cities, in 19 th century. • # Families # houses # chimneys # storks

38 Correlation and Causation • But note the example Stanovich uses: the relation between SES and going to university. • Correlation is still there even when academic aptitude effects have been removed statistically. • S says this finding “strikes at the heart of the merit-based goals of our society. ” • So, he’s concluding that SES controls university attendance, even though he admits that we don’t know the reason for the correlation.

39 Directionality • Poor reading example: researchers once thought that poor reading caused by improper eyemovements. • We now know that causation works in the other direction: poor reading skill produces impaired eyemovements.

40 Directionality • Self-esteem and • In self-esteem case, academic there is evidence that achievement – lots poor academic people have claimed achievement leads to that low self-esteem low SE and good leads to low academic achievement leads to high SE (other things being equal)

41 Selection bias • relationship between subject and environmental variables arises because people with different characteristics select different types of environments. • E. g. , Arizona and respiratory illness. • SAT scores & teacher salaries example