Reasoning in Psychology Using Statistics Psychology 138 2018

Reasoning in Psychology Using Statistics Psychology 138 2018

• Quizzes 1 & 2 are posted – Quiz 2 due Fri (Feb 2) • Don’t forget Exam 1 is coming up (Feb 7) – In class part – multiple choice, closed book – In labs part – open book/notes • Today – Nuts and bolts of experimental & observational designs Announcements Social Science Reasoning Using Statistics

• Generally the process involves a number of steps: – identification of your research questions – identifying your variables – specify your hypotheses (how are the variables related to one another) – selecting a research design – collecting your data, analyzing your data – drawing conclusions from your data about your hypotheses Designing your research Social Science Reasoning Using Statistics

Today’s focus • Generally the process involves a number of steps: – identification of your research questions – identifying your variables – specify your hypotheses (how are the variables related to one another) – selecting a research design – collecting your data, analyzing your data – drawing conclusions from your data about your hypotheses Designing your research Social Science Reasoning Using Statistics

Context • • • Research design Measurement scale What is known about the distributions Social Science Reasoning Using Statistics Statistical tests • Which tests to conduct • How to set up your datafile Design Decision Tree

• Issue: What is the best way to study for a test? Cram the night before - or Spread your studying out over several nights • How could we investigate this question? – What research methods should we use? Selecting your research method Social Science Reasoning Using Statistics

• Issue: What is the best way to study for a test? Observational – involve examining variables as they already are Experimental – involve manipulation of variables • How could we investigate this question? – What research methods should we use? Selecting your research method Social Science Reasoning Using Statistics

• Issue: What is the best way to study for a test? Observational – Randomly select individuals – Watch & classify their study habits – See how they do on a test Experimental – Randomly select individuals – Randomly assign to groups • Crammed study group • Distributed study group – See how they do on a test Selecting your research method Social Science Reasoning Using Statistics

• Issue: What is the best way to study for a test? Observational – Randomly select individuals – Watch & classify their study habits – See how they do on a test • Here our analysis looks for a relationship between the explanatory (study method) and response (test performance) variables • The scale of measurement used for the variables is important Observational designs Social Science Reasoning Using Statistics

• Why random assignment? • To try to minimize bias, potential systematic differences between the people in the different groups Experimental – Randomly select individuals – Randomly assign to groups • Crammed study group • Distributed study group – See how they do on a test Part 1 of the movie “Every Statistical test starts with an appropriate selection of subjects” Experiments Social Science Reasoning Using Statistics

• Why 2 groups? – All experiments make some sort of comparison. Experimental – Randomly select individuals – Randomly assign to groups • Crammed study group • Distributed study group – See how they do on a test Experiments Social Science Reasoning Using Statistics

• Why 2 groups? – All experiments make some sort of comparison. If only one group: – There is natural variability in your DV. – With no comparison group, so hard to see if: change in the IV Causes change in the DV – Should not make causal claims • This is an observational design called “one-shot case study design” ? ? Experiment ? ? • Randomly select individuals • Tell 10 students to distribute their studying • See how they do on a test Special case: when our “comparison” group is the population Experiments Social Science Reasoning Using Statistics

• Why 2 groups? – All experiments make some sort of comparison. The systematic reduction and control of variability of the IV(s) allows us to examine whether they are sources of (causes) variability in the DV If two groups: – Still some natural variability in your DV. – But have constrained one source of the variability (crammed or distributed studying) – Can see if there is an effect of the manipulation change in the IV Causes change in the DV We will discuss these designs with two groups later in the semester Experiments Social Science Reasoning Using Statistics Experiment • Randomly select individuals • Randomly assign to groups – Crammed study group – Distributed study group • See how they do on a test

• Issue: What is the best way to study for a test? • Variables – Independent – Dependent – Extraneous • Control • Random • Confound Experiments Social Science Reasoning Using Statistics Experiment • Randomly select individuals • Randomly assign to groups – Crammed study group – Distributed study group • See how they do on a test

• Issue: What is the best way to study for a test? • Variables – Independent – Dependent – Extraneous • Control • Random • Confound Experiment – These are the variables that • are Randomly manipulated by the select experimenter individuals • Randomly • A number of ways to manipulate your IVassign to groups – Crammed study group – Event/Stimulus manipulations – manipulate – context, Distributed study group characteristics of the stimuli, etc. • See how they dogroups on a test – Instructional manipulations – different are given different instructions – Subject manipulations – there are (pre-existing mostly) differences between the subjects in the different conditions (typically results in quasiexperimental designs) – Will typically use a Categorical Scale – Nominal or Ordinal – names corresponding to different levels of the IV Experiments Social Science Reasoning Using Statistics

• Issue: What is the best way to study for a test? • Variables 1 IV (factor): study type – Independent – Dependent 2 levels: study Distributed study –Crammed Extraneous Study all in one night • Control Study over five nights • Random • Confound Experiments Social Science Reasoning Using Statistics Experiment • Randomly select individuals • Randomly assign to groups – Crammed study group – Distributed study group • See how they do on a test

• Issue: What is the best way to study for a test? • Variables Experiment – Independent • Randomly select – Dependent • These are the variables that are measured individualsby the experimenter, they are “dependent” onassign to groups – Extraneous • Randomly the independent variables • Control – Crammed study group • Random • For most of our designs these will – be Distributed study group measured on a Continuous Scale of how they do on a test • Confound • See Measurement (Ratio or Interval) Test performance Experiments Social Science Reasoning Using Statistics

• Issue: What is the best way to study for a test? • Variables Experiment – Independent – Dependent – Extraneous • Randomly select individuals • Randomly assign to groups – Holding things constant • Control – Crammed study group - Controls for excessive random variabilitystudy group • Random – Distributed • Confound • See how they do on a test Control Variable: total time studying Crammed study Distributed study Experiments Study for 5 hours Social Science Reasoning Using Statistics Study for 1 hour each night Total study time = 5 hours

• Issue: What is the best way to study for a test? • Variables – Independent – Dependent – Extraneous Experiment • Randomly select individuals • Randomly assign to groups • Control – Crammed study group • Random - May freely vary, to spread variability equally across – Distributed study group • Confound all experimental conditions • See how they do on a test Random variables: How much sleep they get What they eat for lunch Experiments Social Science Reasoning Using Statistics

• Issue: What is the best way to study for a test? • Variables Experiment – Independent – Dependent – Extraneous • Control • Random • Confound • Randomly select individuals • Randomly assign to groups – Crammed study group – Distributed study group – Other variables, that haven’t • been Seeaccounted how they for do on a test (manipulated, measured, randomized, controlled) that co-vary with the IV(s) and can impact changes in the dependent variable(s) Experiments Social Science Reasoning Using Statistics

• Issue: What is the best way to study for a test? • Possible Confound variable: Order of studied material, always start with vocab Variables Crammed study Distributed study Underlying third variable – Independent • Randomly select Experiment that varies with the IV – Dependent individuals – Extraneous • Randomly assign to groups Studied vocabulary Studied only • Control – Crammed study group and practice problems vocabulary • Random • Confound – Distributed study group Test performance Is this difference duethey to study • See how do onmethod? a test Or is it due to what was studied? 85% Experiments Social Science Reasoning Using Statistics 73%

• Which research method is best? - it depends Observational Advantages - May see patterns of complex behaviors - Good first step - May learn about something unexpected Disadvantages - Shouldn’t make causal claims - Directionality of the relationship isn’t known - Threats to internal validity due to lack of control - Sometimes the results are not reproducible Experimental - Precise control possible - Precise measurement possible - Can make causal claims - Artificial situations may restrict generalization to “real world” - Complex behaviors may be difficult to measure Selecting your research method Social Science Reasoning Using Statistics

Your design largely determines your statistical analysis Social Science Reasoning Using Statistics Design Decision Tree

Your design largely determines your statistical analysis Social Science Reasoning Using Statistics Design Decision Tree Which test do I use? • Students often find this one of the hardest parts of the course – the key is understanding the context SPSS

• Today’s lab – Practice developing and evaluating experiments • Questions? Statistics Learning Centre Choosing which statistical test to use (~10 mins) Erich Goldstein Choosing a Statistical Test (~13 mins) RStats. Institute Independent vs. Paired t Tests Setup in SPSS (~4 mins) Dr. Todd Grande Comparison of Independent-Samples T Test and Paired-Samples T Test using SPSS (~20 mins) Wrap up Social Science Reasoning Using Statistics