Lovely Lucid Logistics the analysis and graphic presentation

  • Slides: 21
Download presentation
Lovely Lucid Logistics the analysis and graphic presentation of effects of nominal and metric

Lovely Lucid Logistics the analysis and graphic presentation of effects of nominal and metric variables on binary outcomes Diana Eugenie Kornbrot Blended Learning Unit University of Hertfordshire d. e. kornbrot@herts. ac. uk 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 1

Abstract n n Logistic regression can be used to answer the same questions about

Abstract n n Logistic regression can be used to answer the same questions about binary variables that ANOVA and ANCOVA answer about metric variables. However, SPSS provides much less support for logistic regression. The Logistic Regression Procedure provides no equivalent of ANOVA Means Tables or Profile Plots. This presentation shows how to use a combination of SPSS Procedures to produce Tables and Graphs of predicted logit and probabilities as a function of categorical factor and metric covariate variables. Diagnostics for model fit NOT discussed n Merits own presentation 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 2

Acknowledgments n Lia Kvavilashvili n n n For all the prospective memory data Stimulating

Acknowledgments n Lia Kvavilashvili n n n For all the prospective memory data Stimulating theoretical discussion on content ESRC Project Grant 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 3

Goals n n Motivate Logistic regression Graphic Presentation of Logistic Model Results n n

Goals n n Motivate Logistic regression Graphic Presentation of Logistic Model Results n n Predictions n n Factors and Contrasts Application to Different Designs n n Logits and Probabilities as function explanatory variables Identification of statistically reliable effects n n Interpretation much easier from graphs Explanatory variables: 2 or 3 categorical Explanatory variables: 1 metric, 1 or 2 categorical Recommendations to Users of Logistic Regression Recommendations to SPSS 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 4

Why Logistic Analysis? n Need to analyse binary, i. e. 2 alternative, responses n

Why Logistic Analysis? n Need to analyse binary, i. e. 2 alternative, responses n n Errors: right, wrong Events: remembered, forgotten Success: grant awarded, grant rejected patient recovered, or not More than 1 categorical variable n n n Chi-square not sufficient Combination of metric and categorical explanatory variables Interactions matter 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 5

Why Interpretation of Results is a Problem n Analysis is on log (odds ratio)

Why Interpretation of Results is a Problem n Analysis is on log (odds ratio) or logits n n Need for Packages SPSS or other n n Lack of intuitive feel for logits Lack of intuitive feel for odds ratios for non-betters Probabilities are more ‘natural’? Can’t hand calculate, as no closed form answer SPSS Output n Primary output is in logits n No directly useful graphics output n BUT Save permits direct saving of probabilities no logits n ? No confidence levels on probabilities 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 6

Analysis n n n n GLM framework Effects assumed to be linear on logits

Analysis n n n n GLM framework Effects assumed to be linear on logits Model Goodness of Fit Test on – 2 Log. Likelihood, -2 LL Model Fitting Procedure n n Effect of Evauluation Criteria: SPSS uses Wald n n n SPSS uses Wald, other packages use deviance = -2 LL On factors and covariates On model parameters Other Packages Vary, all give Wald as minimum n JMP, SPSS, SAS, SYSTAT 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 7

Data Example: Prospective Memory n Does person have GOOD prospective memory n 5 or

Data Example: Prospective Memory n Does person have GOOD prospective memory n 5 or 6 occasions remembered from 6 opportunities n Model 1: task(action, event, time), age(4 categories) Model 2: task(action, event, time), age(4), intellect n Presentation Criteria n n n Easy to interpret > Graphics Predicted probability and logits Estimate of accuracy as part of results Tests for explanatory variable effects and contrasts 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 8

Model 1 using SPSS menus n Analyze > Regression > Binary Logistic n n

Model 1 using SPSS menus n Analyze > Regression > Binary Logistic n n n Dependent good# Covariates task#(cat) age#(cat) task#(cat)*age#(cat) Method Enter Categorical task#(deviation) age#(deviation) or task#(repeated) age#(repeated) !!!NOT indicator, the default!!! not a lot of people know that! Save probabilities, Cook’s, deviation Options CI for exp(B) 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 9

Model 1 Global Results n n n Model 1: task(action, event, time), age(4 categories)

Model 1 Global Results n n n Model 1: task(action, event, time), age(4 categories) Omnibus Test Significant = Good Model Summary Substantial variance accounted for 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 10

SPSS: Model 1 Parameters n n n Variable effect not salient No effects or

SPSS: Model 1 Parameters n n n Variable effect not salient No effects or standard errors for reference (last) Wald Estimates of s. e. may not be those that are needed? 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 11

SPSS Graphic Representation n Predicted Probabilities, pre_1 n n Directly Available from Save Logits

SPSS Graphic Representation n Predicted Probabilities, pre_1 n n Directly Available from Save Logits can be calculated n Compute > Transform n n n Graph > Interactive > Line plot n n Lgt = ln(pre_1/(1 -pre_1) NB Most other packages allow direct saving of logits Y axis X axis Colour predicted probability (mean) age# task# No interactions n So expect logit plots to be ‘more’ linear 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 12

SPSS: Logit & Probability Graphs Raw probability Logit ? ? looks more linear? ?

SPSS: Logit & Probability Graphs Raw probability Logit ? ? looks more linear? ? Confidence Levels? ? ? NOT in SPSS!!! 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 13

Confidence Levels n n Assume no extra-binomial dispersion Asymptotic for logit n n n

Confidence Levels n n Assume no extra-binomial dispersion Asymptotic for logit n n n se(lgt)2 = 1/Noccur - 1/Nnot occur Lower Confidence Level, 95%, LCL(lgt) = mean(lgt) -1. 96 se(lgt) Upper Confidence Level, 95%, LCL(lgt) = mean(lgt) +1. 96 se(lgt) Asymptotic for probability n n n Symmetric about mean(lgt) Asymmetric about mean(prob). Calculate from lgt CLs probability = exp(lgt)/[1+exp(lgt] LCL(prob) = exp(LCL(lgt)0/[1+exp(LCL(lgt)) UCL(prob) = exp(UCL(lgt)0/[1+exp(UCL(lgt)) Use EXCEL, can’t customise error bars in SPSS 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 14

EXCEL: Logit & Probability Graphs Raw probability Logit Errors are for each group. So

EXCEL: Logit & Probability Graphs Raw probability Logit Errors are for each group. So low power for interaction 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 15

Model 2 Using SPSS menus n Analyze > Regression > Binary Logistic n n

Model 2 Using SPSS menus n Analyze > Regression > Binary Logistic n n n n Dependent Covariates Method Categorical or Save Options good# task#(cat), age#(cat), intellec task#(cat)*age#(cat) task#(cat)*intellec*age#(cat) task#(cat)*age#(cat)*intellec Enter task#(deviation), age#(deviation) task#(repeated), age#(repeated probabilities, Cook’s, deviation CI for exp(B) 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 16

Model 2 Summary Omnibus=Whole Model LR chi 2(23)=82. 2, p=. 0000001 Various r 2

Model 2 Summary Omnibus=Whole Model LR chi 2(23)=82. 2, p=. 0000001 Various r 2 values n n n Mc. Fadden=. 36; Cox & Snell=. 37; Nagelkerke=. 51 Variable Effects n Source DF Wald chi^2 TASK 2 14. 03. 000899 AGE 3 3 4. 45. 217040 intellect 1 2. 87 TASK*AGE 6 6. 00 TASK*intellect 2 4. 32 AGE*intellect 3 5. 00 TASK*AGE*intellect 6 Wald Prob LR Chi^2 29. 70. 000000 4. 96. 174500. 089995 6. 03. 423621 14. 63. 115183 7. 73. 171542 7. 07 10. 52. 104480 LR Prob . 014101. 023371. 021003. 069614 21. 43 . 001532 Comparison of Variable Effects with different methods/packages n 1. Likelihood Ratio shows strong effects intellec + intellec interactions Used JMP-IN [even version 3, 5 is better for some things] 2. 3. Wald does NOT show these effect - WORRYING Model improvement with intellec: chi 2(12)=33. 3, p=. 00087 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 17

Model 2 Probability by Age • Not very clear! • Task effect: • Event

Model 2 Probability by Age • Not very clear! • Task effect: • Event has lower prob • Intellect: • Most groups: • Prob increase with intellec • 3 way interactions: • > 70, event; 61 -65 time • Prob decrease with intellec 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 18

Model 2 Logit by Age • Bit clearer! • Task effect: • Event has

Model 2 Logit by Age • Bit clearer! • Task effect: • Event has lower prob • Intellect: • Most groups: • Prob increase with intellec • Large: 71 -75 time, 76 -80 action • 3 way interactions: • > 70, event; 61 -65 time • Prob decrease with intellec 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 19

Summary & Recommendations n Recommend Logit analyses as a very important tool n Recommend

Summary & Recommendations n Recommend Logit analyses as a very important tool n Recommend Graphic displays toimprove interpretability n SPSS provides basic procedure n Limitations of SPSS n No direct predicted logit or probability Table or Graph Summary n Poor model diagnostics and power procedures n No direct group standard errors n No Maximum Likelihood estimates for explanatory variables n No mixed models n Other general packages are also DIRE - in different ways n Need simple tools for routine logistic applications n Can SPSS User Groups do anything? 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 20

References Agresti, A. (1990). Categorical data analyses. Chichester: Wiley. Agresti, A. (1996). Introduction to

References Agresti, A. (1990). Categorical data analyses. Chichester: Wiley. Agresti, A. (1996). Introduction to categorical data analyses. Chichester: Wiley. Agresti, A. , & Finley, B. (1997). Statistical methods for the social sciences (3 ed. ). Upper Saddle River, NJ: Prentice Hall. Agresti, A. , & Hartzel, J. (2000). Tutorial in biostatistics: strategies for comparing treatments on a binary response with mulit-centre data. Statistics in Medicine, 19, 1115 -1139. Everitt, B. , & Dunn, G. (2001). Applied multivariate data analysis (2 ed. ). London: Edward Arnold. Kornbrot, D. E. (2000, 17 -20 july 2000). Counting on prospective memory: Advantages of logistic and log linear models over ANOVA and correlations. Paper presented at the 1 st International Prospective Memory Conference, Hatfield, Hertfordshire, U. K. Kvavilashvili, L. , Kornbrot , D. E. , Mash , V. , Cockburn, J. , & Milne, A. (2000, 17 -20 july 2000). Remembering event-, time- and activity-based tasks in young, young-old and old-old people. Paper presented at the 1 st International Prospective Memory Conference, Hatfield, Hertfordshire, U. K. Lindsey, J. K. (1999). Models for repeated measurements (2 ed. ). Oxford: Oxford University Press. Sofroniou, N. , & Hutcheson, G. D. (2002). Confidence Intervals for the Predictions of Logistic Regression in the Presence and Absence of a Variance– Covariance Matrix. Understanding Statistics, 1(1), 3– 18. Tabachnick, B. G. , & Fidell, L. S. (1996). Using multivariate statistics (3 ed. ). New York: Harper Collins. 11 -Nov-05 Lucid Logistics: Kornbrot, Blended Learning Unit, University of Hertfordshire 21