Cognitive psychometrics and cognitive latent variable models Joachim

Cognitive psychometrics and cognitive latent variable models Joachim Vandekerckhove Department of Cognitive Sciences University of California, Irvine

Overview • The casting of covetous glances • Cognitive psychometrics and the diffusion model • Multilevel models • Explanatory modeling • Applications • Conclusions 2

CASTING COVETOUS GLANCES 3

Casting covetous glances • Lee Cronbach’s 1957 Presidential Address – Experimental psychology • Systematic manipulation of variables • Focus on difference in group means – Correlational psychology • Measure latent psychological constructs • Focus on preexisting differences between groups – “Time to cross-breed!” 4

We are free at last to look up from our own bedazzling treasure, to cast properly covetous glances upon the scientific wealth of our neighbor discipline. Trading has already been resumed, with benefit to both parties. 5

Casting covetous glances • Lee Cronbach’s 1957 Presidential Address – Experimental psychology • Systematic manipulation of variables • Focus on difference in group means – Correlational psychology • Measure latent psychological constructs • Focus on preexisting differences between groups – “Time to cross-breed!” • Aptitude-treatment interaction research 6

Casting covetous glances • Present day – Experimental psychology • Systematic manipulation of variables – Psychometrics • Measure latent psychological constructs – Mathematical psychology • Modeling cognitive processes (Parameters with interesting interpretations) – Time to cross-breed! • Cognitive psychometrics 7

COGNITIVE PSYCHOMETRICS AND THE DIFFUSION MODEL 8

Cognitive psychometrics • Use cognitive models as measurement model • Try to explain differences – between trials, manipulations and persons – e. g. by regressing the parameters on covariates • Three “model building blocks” for explanatory modeling (De Boeck & Wilson, 2004) – Random effects – Manifest predictors – Latent predictors 9

Cognitive psychometrics • Most common measurement model: Gaussian – Normal linear model (linear regression, ANOVA): Indexes p for persons, i for conditions – But often not a realistic model – Entirely unsuited for, say, choice RTs 10

Diffusion model • Wiener diffusion model – Process model for choice RT – Predicts RT and binary choice simultaneously 11

Diffusion model • Wiener diffusion model – Process model for choice RT – Predicts RT and binary choice simultaneously – Principle: Accumulation of information – Interpretable parameters • • Rate of information accumulation d Information needed a Initial bias b Nondecision time t 12

(For persons p, conditions i, and trials j. ) τ Evidence d a z = ba 0. 0 0. 125 0. 250 0. 375 0. 500 0. 625 0. 750 time

p(t) (For persons p, conditions i, and trials j. ) 0. 0 0. 125 0. 250 0. 375 0. 500 0. 625 0. 750 time

Diffusion model • Wiener diffusion model – Process model for choice RT – Predicts RT and binary choice simultaneously – Principle: Accumulation of information – Interpretable parameters • • Rate of information accumulation d Information needed a Initial bias b Nondecision time t 15

Diffusion model • Classical methods: many associated problems – Substantive issues • Almost completely descriptive – differences over persons/trials/conditions cannot be accounted for in the model unless we model each cell separately – Technical issues • Parameter estimation / Model comparison • Difficult to combine information across participants – Problem if many participants with few data each – Problem if items are presented only once (e. g. , words) 16

MULTILEVEL MODELING 17

Mixtures and mixing • 18

Mixtures and mixing Across-person distribution Person-specific, across-trial distribution qp Prediction for trial (pij) qp τ Evidence d a z = ba 0. 0 0. 125 0. 250 0. 375 0. 500 0. 625 0. 750 tpij

Mixtures and mixing • This is a dominance parameter 20

Mixtures and mixing Item component Person component gp li gp l i (into Wiener process) dpij 21

Mixtures and mixing • Addition of random effects – Allows for excess variability • Due to item differences • Due to person differences – Allows to build “levels of randomness” – Importantly, can be built on top of a diffusion model 22

EXPLANATORY MODELING 23

Explanatory modeling • Previous models were descriptive – Didn’t use covariates or variability over persons/items • Hierarchical models quantify variability – External factors can be used as predictors to explain the differences in parameter values (i. e. , reduce unexplained variance) – Latent factors can be used to explain covariance between people/items/. . . 24

Explanatory modeling • Use basic “building blocks” for modeling – Random/Fixed effects – Person/Item side – Hierarchical/Crossed – Use covariates (continuous/categorical/binary, manifest/latent) 25

Explanatory modeling • explaining variability in drift rate We translate the data into parameters, and explain variability through covariates

Implementation • Vandekerckhove, Tuerlinckx, & Lee (2011). Psychological Methods. 27

APPLICATION 1 THE LEUVEN NATURAL CONCEPTS DATA SET 28

Leuven data • Features – Features of natural language concepts • Typicality, goodness, generation frequency, age of acquisition, word frequency, familiarity, imageability • Choice response times – Category verification task • E. g. , “Is item dog a member of category mammals? ” – Partial overlap in used categories • Birds, fish, insects, mammals, reptiles, musical instruments, tools, vehicles 29

Leuven data • A sneak peek at the data. . . Person Person Person Person Person Person. . . 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Item 51 Item 52 Item 53 Item 54 Item 55 Item 56 Item 57 Item 58 Item 59 Item 60 0. 9848 0. 7019 0. 6202 0. 5771 0. 5245 0. 5999 0. 6108 0. 5517 0. 7580 0. 7072 0. 4902 0. 6593 0. 6788 0. 5178 0. 5833 0. 6885 0. 7425 0. 5630 0. 6716 0. 6673 -1. 4084 -0. 7109 -0. 5820 1. 7123 -0. 6528 -0. 8530 1. 6328 0. 6534 -1. 0110 -1. 0976 1. 3394 -1. 1068 -0. 5789 -0. 7130 -0. 6059 0. 8769 0. 7650 -1. 5076 -0. 8894 1. 0799 0. 8513 0. 9987 0. 6212 0. 6004 0. 4329 0. 7845 0. 7291 0. 5575 0. 7540 0. 6659 0. 5202 0. 6861 0. 4865 0. 5616 0. 5889 0. 6630 0. 7280 0. 6440 0. 6406 0. 7001 0. 7347 0. 7218 0. 6829 0. 9919 0. 8328 1. 0349 0. 8207 1. 0180 0. 7430 0. 6209 0. 5100 0. 6399 0. 4891 0. 5622 0. 5903 0. 6793 0. 8343 0. 5725 0. 8293 0. 8787 1. 0622 1. 4516 0. 5641 0. 6308 0. 9855 0. 7309 0. 8655 0. 5269 0. 9048 0. 7099 0. 5158 1. 2217 -0. 5767 0. 6386 1. 0032 1. 0409 1. 4173 0. 7045 0. 6546 0. 7200 0. 8811 0. 7224 0. 8666 0. 7991 0. 5314 1. 2436 1. 3248 -1. 9897 0. 7108 0. 8687 0. 5036 0. 7688 1. 4254 0. 8462 1. 1688 0. 6269 0. 7811 0. 6869 -1. 0920 -1. 0442 0. 8563 0. 7065 0. 5487 0. 6567 0. 4700 0. 6328 0. 6365 0. 6728 0. 8420 0. 7640 0. 5924 0. 7624 0. 5201 0. 6232 0. 5376 0. 6604 0. 6101 0. 6113 0. 7749 0. 6390 0. 9647 0. 8825 0. 7380 0. 7481 0. 4770 0. 6482 0. 6820 0. 7349 0. 6361 0. 6860 0. 5117 0. 6571 0. 5019 0. 5124 0. 5652 0. 7518 0. 6166 0. 5552 0. 6582 0. 7814 1. 4821 -0. 7484 0. 6441 0. 9260 0. 6685 1. 1973 1. 3392 0. 5815 -0. 8868 0. 9351 0. 5062 1. 1753 0. 9342 0. 5371 -0. 8289 0. 6824 0. 9403 1. 0313 0. 9315 0. 9698 0. 7379 0. 7842 0. 5972 0. 8520 -0. 5828 0. 6922 1. 0933 0. 6327 0. 8387 1. 2465 0. 6469 1. 0263 0. 5797 0. 8966 1. 3995 0. 9248 0. 6284 0. 6628 1. 2917 0. 6540

Leuven data • A sneak peek at the data – Importantly, each stimulus only shown to each participant once • Distributional analysis per cell is impossible • Cannot collapse over items • Shouldn’t collapse over persons hierarchical analysis – We want to disentangle person and item effects on drift rate – Crossed random effects design 31

Leuven data • Trial’s mean drift rate is a sum of person aptitude and item easiness Item easiness distribution may differ according to whether the item is a target or a distractor Indexes p for persons, i for words 32

Leuven data: Model Check Mean RT item Accuracy item 33

Leuven data • Some results Estimand Post. mean Pop. distr. of item easiness person aptitude 34

Leuven data •

Drift rate regression 36

37

Nondecision time regression 38

39

Leuven data • Drift rate results – Typicality a ‘significant’ predictor in most categories – No other predictor shows consistent pattern • Nondecision time – Word Length a ‘significant’ predictor in most categories – No other predictor shows consistent pattern 40

Conclusions • Leuven data – Variance in person aptitude small (≈ 0. 04) relative to variance in item easiness (≈ 0. 11) – Item easiness correlates with typicality – Nondecision time correlates with word length Vandekerckhove, Verheyen, & Tuerlinckx (2010). Acta Psychologica. 41

APPLICATION 2 EFFECTS OF VALENCE IN A PROACTIVE INTERFERENCE TASK 42

Factor analysis • Partials out underlying skills from one another and from task-specific abilities • Marginal distribution is often Gaussian Taskspecific abilities φ2 φ1 Skills b 1 b 2 b 3 b 4 b 5 Raw data points, normality assumed

Cognitive factor analysis • In many cognitive studies, the “score” on a lab task should be the participant’s rate of information processing (rather than just their accuracy) Taskspecific parameters φ2 φ1 Skills b 1 b 2 b 3 b 4 b 5 Performance in experimental tasks

Cognitive factor analysis Task-specific parameters Model predicted behavior φ2 φ1 Skills b 1 b 2 b 3 b 4 b 5

Identification constraints •

Proactive interference task • B A D C E? F E H interference G G? J I L K F?

Adding more covariates • Battery of personality covariates – CESD, SWL, RRS… • Processing deficits in dysphoric participants expected • Latent factor scores – Can project covariates into factor space and interpret factors and loadings

Model selection result • “Winning” model had F = 6 factors: – Intercept, detection (nonrecent), detection (recent), PI (neg), PI (pos), PI (neu) 49

Covariate loadings • Person covariates mapped into factor space 50

Covariates projected into 4 D space Vandekerckhove (2014). Journal of Mathematical Psychology.

Epilogue • Explanatory cognitive models – combine a realistic process model for choice and reaction time with random effects and explanatory covariates, and latent structures (all borrowed from psychometrics) – allow to analyze complex data sets in a statistically (and substantively) principled fashion with relative ease 52

Epilogue • Explanatory cognitive models – let researchers translate interesting substantive hypotheses into a model – allow simple statistics to be computed to answer complex research questions – often require fewer/less data compared to classical diffusion modeling techniques 53