Foundations of Research Statistics 1 Statistics Knowledge Platos

Foundations of Research: Statistics 1 Statistics & Knowledge: Plato’s Allegory of the Cave ü Click “slide show” to start this presentation as a show. ü Remember: focus & think about each point; do not just passively click. © Dr. David J. Mc. Kirnan, 2014 The University of Illinois Chicago Mc. Kirnan. UIC@gmail. com Do not use or reproduce without permission Raphael [Public domain], from Wikimedia Commons

Foundations of Research: Statistics 2 The statistics module series 1. Introduction to statistics & number scales 2. The Z score and the normal distribution You are here 3. The logic of research; Plato's Allegory of the Cave 4. Testing hypotheses: The critical ratio 5. Calculating a t score 6. Testing t: The Central Limit Theorem 7. Correlations: Measures of association © Dr. David J. Mc. Kirnan, 2014 The University of Illinois Chicago Mc. Kirnan. UIC@gmail. com Do not use or reproduce without permission

Foundations of Research: Statistics Plato’s Allegory of the Cave. 3

Foundations of Research: Statistics Plato’s Allegory of the Cave 4 Socrates: And now, I said, let me show in a figure how far our nature is enlightened or unenlightened : "Behold ! , human beings living in a underground den, which has a mouth open towards the light and reaching all along the den. Here they have been from their childhood, and have their legs and necks chained so that they cannot move, and can only see before them, being prevented by the chains from turning round their heads. ” Plato's Allegory of the cave, Engraving of Jan Saenredam (1565 -1607) after a painting of Corneliszoon van Haarlem (1562 -1638)

Foundations of Research: Statistics 5 The allegory of the cave, 2 “Above and behind them a fire is blazing at a distance, and between the fire and the prisoners there is a raised way; and you will see, if you look, a low wall built along the way, like the screen which marionette players have in front of them, over which they show the puppets. “ Glaucon: "I see". "And do you see", I said, "men passing along the wall carrying all sorts of vessels, and statues and figures of animals made of wood and stone and various materials, which appear over the wall ? Some of them are talking, others silent. “

Foundations of Research: Statistics 6 The allegory of the cave, 3 Glaucon: "You have shown me a strange image, and they are strange prisoners". "Like ourselves", I replied. "And they see only their own shadows, or the shadows of one another, which the fire throws on the opposite wall of the cave ? " Glaucon: "True", he said. "How could they see anything but the shadows if they were never allowed to move their heads? "

Foundations of Research: Statistics 7 The allegory of the cave, 4 And of the objects which are being carried in like manner they would only see the shadows? " Glaucon: "Yes", he said. "And if they were able to converse with one another, would they not suppose that they were naming what was actually before them? " Glaucon: "Very true. “

Foundations of Research: Statistics 8 The allegory of the cave, 5 "And suppose further that the prison had an echo which came from the other side… would they not be sure to fancy then one of the passersby spoke that the voice which they heard came from the passing shadow ? " Glaucon: "No question", he replied. "To them", I said, "the truth would be literally nothing but the shadows of the images". Glaucon: "That is certain. " Click for the complete allegory

Foundations of Research: Statistics Plato’s Cave & Science What does Plato’s Allegory of the Cave tell us about reasoning (scientific and otherwise)? We cannot observe “nature” directly, we only see its manifestations or images: ü We are trapped in a world of immediate sensation; ü Our senses routinely deceive us (they have error). ü We cannot get outside our limited sensations to see the underlying “form” of nature 9

Foundations of Research: Statistics 10 Plato’s Cave and Scientific Reasoning: Core limitations of our knowledge about the world. 1. Theories (knowledge structures) address hypothetical constructs… We infer the underlying forms of natural processes 2. We study samples of people & places, and try to generalize to the larger population

Foundations of Research: Statistics Plato’s Cave: Hypothetical constructs 1. We study hypothetical constructs; basic “operating principles” of nature. e. g. evolution, gravity, learning, motivation… n n n Processes that we cannot “see” directly… …that underlie events that we can observe. We use rational analysis – theory – to deduce what the “form” of these processes must be, and how they work. 11

Foundations of Research: Statistics 12 Hypothetical constructs Gravity is a hypothetical construct: § We cannot ‘see’ gravity; § We only see what it does … stuff falls; § Our observations and theory tell us that gravity is highly lawful. Newton’s law tells us that: F= G M 1 M 2 r 2

13 Foundations of Research: Statistics Hypothetical constructs Gravity is a hypothetical construct: § We cannot ‘see’ gravity; § We only see what it does … stuff falls; § Our observations and theory tell us that gravity is highly lawful. Newton’s law tells us that: M 1 M 2 F= G r 2 F Force of gravity M 1 = G Gravitational constant (6. 7 x 10 -11) X The Mass of object 1 M 2 M Object 2 r 2 Distance (r) between the center or each object, squared

Foundations of Research: Statistics 14 Hypothetical constructs Gravity is a hypothetical construct: § We cannot ‘see’ gravity; § We only see what it does … stuff falls; § Our observations and theory tell us that gravity is highly lawful. Newton’s law tells us that: F= G M 1 M 2 r 2 § Our observations & theory about a construct such as gravity allow us to make and test very precise predictions (hypotheses). § Despite how well we can calculate & predict gravitational force, we cannot see or assess it directly. § We must infer it’s properties by testing hypotheses about it.

Foundations of Research: Statistics What is a hypothetical construct? n n We cannot actually look in the brain and “see” learning. What we can see are… n n These are different possible operational definitions of learning… 15 …changes in test scores …faster / better performance …changes in neuro-anatomy …other measurable outcomes.

Foundations of Research: Statistics What is a hypothetical construct? n n We cannot actually look in the brain and “see” learning. What we can see are… n …changes in test scores …faster / better performance …changes in neuro-anatomy n …other measurable outcomes. n n 16 We infer that we observe reflects the hypothetical construct “learning” We make these observations while testing a hypothesis about what might cause learning.

Foundations of Research: Statistics 17 Why not observe directly? Why can’t we just observe “nature” directly? ü We can only observe the effects of hypothetical constructs, not the processes themselves. ü Our theory helps us develop hypotheses about what we should observe if our theory is “correct”. What is my hypothesis?

Foundations of Research: Statistics 18 Constructs and hypotheses I theorize that attention facilitates learning Attention (These are my Hypothetical constructs) Learning Operational Definitions My Hypothesis requires that I operationally define these abstract constructs: I predict that: If eye tracking & task time are higher Then ü Eye tracking ü Quiz scores ü Time on task ü Paper quality scores & paper quality will be greater.

Foundations of Research: Statistics Constructs and hypotheses § If my hypothesis is supported by the data Attention Learning Inference 19 Inference ü Eye tracking ü Quiz scores ü Time on task ü Paper quality § I may infer that I have validly assessed (or manipulated) the constructs that underlie my theory. § Of course other processes may be operating. . § Or the results could simply be due to chance (error).

Foundations of Research: Statistics Why not observe directly? Why can’t we just observe “nature” directly? ü We can only observe the effects of hypothetical constructs, not the processes themselves. ü Our theory helps us develop hypotheses about what we should observe if our theory is “correct”. ü We test our hypotheses to infer how nature works. ü Our inferences contain error: we must estimate the probability that our results are due to “real” effects versus chance. ü The link from hypothetical constructs to empirical evidence can be deductive (“top-down”) or inductive (“bottoms-up”). 20

Foundations of Research: Statistics n The link between theory & data 21 Deductive reasoning is “top down”: Ø We begin with a strong theory or concept… Ø …then move to data collection to test / support it: Theory Deductive Hypothetical Constructs Research methods Inductive Not directly observed n Specific hypotheses (operational definitions) Empirical observations Inductive reasoning is “bottoms up”: Ø We begin with empirical observations… Ø …then develop or adapt a theory to explain them.

Foundations of Research: Statistics n The link between theory & data 22 Deductive reasoning is “top down”: Ø We begin with a strong theory or concept… Ø …then move to data collection to test / support it: I theorize that humans (mammals? ) store information and direct behavior via stable, implicit knowledge structures. Implicit Knowledge Structure (or cognitive schema) is a Hypothetical Construct. I deduce what effects schema should have on observable behavior I hypothesize that if cognitive schema are central, Deductive then participants will better recall information consistent with a cognitive schema I induce in the lab. I operationally define cognitive schema consistent with theory, Then gather experimental or observational data.

Foundations of Research: Statistics n The link between theory & data 23 Deductive reasoning is “top down”: Ø We begin with a strong theory or concept… Ø …then move to data collection to test / support it: Ø My study results (data) may then lead me to (inductively) modify or clarify my theory. I theorize that humans (mammals? ) store information and direct behavior via stable, implicit knowledge structures. Deductive Inductive I operationally define cognitive schema consistent with theory, Then gather experimental or observational data.

Foundations of Research: Statistics n The link between theory & data 24 Inductive reasoning is “bottoms up”: Ø We begin with empirical observations… Ø …then develop or adapt a theory to explain them. I develop a theory that people are affected more by threats to their sense of identity or personal security than by pragmatic concerns. I observe a regularity or anomaly in behavior; Deductive These different forms of threat are the hypothetical constructs I need to test. Since I think people should operate rationally Inductive I go on to develop concrete hypotheses and research methods. In recurrent elections many people vote against their own immediate economic interests. (that is, to maximize their immediate interest) I need to explain this empirical observation.

Foundations of Research: Statistics n The link between theory & data 25 Inductive reasoning is “bottoms up”: Ø We begin with empirical observations… Ø …then develop or adapt a theory to explain them. Ø My emerging theory allows me to (deductively) articulate hypotheses and gather systematic data. I develop a theory that people are affected more by threats to their sense of identity or personal security than by pragmatic concerns. Deductive I operationally define the I need to explain this key terms of theory, , , empirical observation. …then create variables that I measure or manipulate… Inductive …to gather more systematic data.

Foundations of Research: Statistics n The link between theory & data 26 Wherever the research begins, both deductive and inductive processes are important… Theory Deductive Hypothetical Constructs Research methods Inductive Not directly observed Specific hypotheses (operational definitions) Empirical observations In sum: § We develop theories – explanations of natural events – in the form of hypothetical constructs. § The interplay of theory and data is both inductive and deductive.

Foundations of Research: Statistics Generalizability 2. Theories are tested with samples, not the entire population. n n Just as we infer the hypothetical constructs underlying our observations, we infer how well those results generalize to: The larger population our sample is drawn from n n Other physical or social settings Other conditions or forms of the Independent Variable Other outcomes or forms of the Dependent Variable. As with all inferences, our generalizing beyond the experiment is probabilistic and is subject to error. 27

28 Foundations of Research: Statistics How well do your findings reflect the larger world? (outside of the experiment or study). Is the sample typical of the larger population? Does the outcome measure reflect how the phenomenon works in the “real world” The research Sample: The Dependent Variable The study structure & context The Independent Variable The research Setting: Is this typical of “real world” settings where the phenomenon occurs? Does the experimental manipulation (or measured predictor) actually create (validly assess…) the phenomenon you are interested in?

Foundations of Research: Statistics Inferring the natural world 29 ü We try to understand basic principles of nature; hypothetical constructs. ü We develop a theory that rests on one or several constructs. ü We derive a hypothesis that, if supported, suggests the constructs work the way we say they do. ü Data cannot ‘prove’ a theory; we infer – with error – how nature works. § Any research – experimental, observational, historical… – occurs in a particular time and place. § Just as we infer what nature is like from our evidence, we infer that our evidence is representative of the larger world.

Foundations of Research: Statistics Inferring the natural world 30 In a scientific perspective, all knowledge is provisional. Like the prisoners in the cave, we cannot look up and directly observe the underlying processes of nature. We infer what the underlying processes of nature must be by conducting and analyzing research. Of course there are potential sources of error or bias (threats to Internal Validity) in any study.

Foundations of Research: Statistics Inferring the natural world 31 Plato’s allegory of the cave underlies the concept of error in research. Our empirical data are like the shadows on the wall: • We know they do not give us a direct, unbiased view of reality; • We collect test hypotheses to infer what reality must be like; • We always adjust our “findings” by the prospect of bias or error.

Foundations of Research: Statistics Inferring the natural world 32 Plato’s allegory of the cave underlies the concept of error in research. Similarly, we estimate what the larger population is like by studying only a sample of it. Every feature of our study – the variables we manipulate or measure, the setting, the sample – may introduce bias or error.

Foundations of Research: Statistics 33 Inferring the natural world This is the core concept of the Critical Ratio. Our empirical observations: the strength of our research results. Critical ratio The amount of error in our observations or estimates. Statistically – and conceptually – we weight any observation about the world by the amount of error we assume to be occurring. This helps us think about the likelihood that we are “seeing” is inaccurate, or by chance alone. Inferential statistics – ‘Z’ scores, the t test and so on – all inherently weight the results by an estimate of error. We use this ratio to estimate the probability that we see reflects a “real” effect, or is just by chance alone.

Foundations of Research: Statistics 34 Critical Ratio Critical ratio Z is the most basic form of the Critical Ratio Z= Empirical observations: strength of our results. Error in our observations or estimates. = Distance of the score from the Mean Amount of variance in the sample = X-M Standard Deviation (S)

Foundations of Research: Statistics What do the terms of the Critical Ratio actually mean? 35 Results: Z: how much does one score differ from the rest of the scores in a sample? ü Mary Louise had statistics in a previous course. ü Did this help her in Psych 242? ü I compare her test score to the average (Mean, or M) of the scores for the complete class. ü If a previous stat course helped, her score should much higher than the class average. ü The difference between her and the class = strength of the results. Z= Empirical observations: strength of our results. Error in our observations or estimates.

Foundations of Research: Statistics What do the terms of the Critical Ratio actually mean? 36 Results: Z: how much does one score differ from the rest of the sample? ü Let’s look at the scores for the rest of the class… Number of People ü Here is the score for Mary Louise. X X Hmm. . Z= X X X X X ü Looks as though Mary Louise did well. X X ü There was a strong effect of her taking stats previoulsy. ML Scores Great Empirical observations: strength of our results. Error in our observations or estimates.

Foundations of Research: Statistics What do the terms of the Critical Ratio actually mean? 37 Error: ü These class scores did not have a lot of variance… ü … they are tightly clustered around the Mean. Number of People ü Mary Louise’s score was some distance better than the Mean. X X Hmm. . Z= X X X X X X ü So Mary Louise “did well” relative to a class that did not have a lot of variance (error) in their scores. ML Scores Great Empirical observations: strength of our results. Error in our observations or estimates. Strong result. Modest error variance

Foundations of Research: Statistics What do the terms of the Critical Ratio actually mean? 38 Error: ü Now imagine the rest of the class was all over the place… Number of People ü The M is the same … but scores do not form a tight cluster. ü Mary Louise still exceeds the M, but that is less impressive when a lot of scores are that high. X X X Hmm. . X X X X X ML X Scores Great Empirical observations: strength of our results. Z= Error in our observations or estimates. Strong result. Much more error variance

Foundations of Research: Statistics What do the terms of the Critical Ratio actually mean? 39 ü Mary Louise’s score is good in a distribution with low error variance. . . ü She will have a high Z Score. . . ü. . . Suggesting that she really was better than the rest of the class Z= Empirical observations: strength of our results. Error in our observations = How far is the score above / below the Mean (X – M) Error variance in the sample (Standard Deviation, or S) = Fairly far for Mary Louise Relatively low

Foundations of Research: Statistics What do the terms of the Critical Ratio actually mean? 40 ü Here Mary Louise’s score is in the same place; above the Mean. . ü But there is much more variance within the class. ü She will have a lower Z Score. . . ü. . . making it less clear that she really was better than the rest of the class Z= Empirical observations: strength of our results. Error in our observations = How far is the score above / below the Mean (X – M) Error variance in the sample (Standard Deviation, or S) = Above the Mean More error variance; many different scores in the class

Foundations of Research: Statistics What do the terms of the Critical Ratio actually mean? 41 ü This is the logic of how we understand the natural world. ü We make observations, either directly or via an experiment. . . ü. . . and try to understand how meaningful they are. ü “Meaningful” means we weight our results by the chance that we are simply seeing error or bias. ü This is the Critical Ratio. ü The Z score is a basic form of the critical ratio. ü You will use the t score later, which is just an elabortion of Z.

Foundations of Research: Statistics Inferring the natural world 42 Our empirical observations: the strength of our research results. Critical ratio The amount of error in our observations or estimates. Much of research methodology operates on the critical ratio: ü We design our studies to create strong empirical observations; § We choose predictor variables or experimental manipulations that maximize the power of our results; § We use measures that are maximally sensitive; § We try to have large enough samples for results to be robust.

Foundations of Research: Statistics Inferring the natural world 43 Our empirical observations: the strength of our research results. Critical ratio The amount of error in our observations or estimates. Much of research methodology operates on the critical ratio: ü We design our studies to create strong empirical observations. ü …and to minimize error; § We design our measures to be valid (to accurately reflect the construct we are studying)… § …and reliable, to provide similar answers over time. § We standardize and carefully control research procedures to minimize error variance (i. e. , differences among participants due to the procedures rather than the Independent Variable).

Foundations of Research: Statistics Inferring the natural world 44 As we move into inferential statistics and explore the critical ratio keep the allegory of the cave in mind. A core assumption of modern science is that we can never Know anything for certain; § As with the prisoners in Plato’s allegory, we cannot “see” truth directly. § All our knowledge is probabilistic; we are inferring what the world is really like by making systematic observations about what we can see.

Foundations of Research: Statistics Plato’s Cave and scientific logic 45 Plato’s allegory of the cave can be taken as a metaphor for the “problem” of knowledge; § Processes that govern the natural world are not easy for us to see directly, and. . § Almost invariably we are studying only a sample of the Summary population we are interested in. § As a result, we must infer how nature works by: ü Positing hypothetical constructs (the building blocks of a theory) that may explain natural processes ü Gathering data and testing hypotheses to infer whether our theory is correct. ü Gathering data requires that we operationally define our variables, by specifying measures or manipulations that express them. § All our observations and inferences are made with error: we never see the world perfectly clearly.

Foundations of Research: Statistics 46 Please go on to the next module: Testing hypotheses and the critical ratio. Raphael [Public domain], from Wikimedia Commons