Effort and Motivation in Psychosis Deanna M Barch
Effort and Motivation in Psychosis Deanna M. Barch Co-Director, Cognitive Control and Psychopathology Laboratory, Psychological & Brain Sciences, Psychiatry, and Radiology, Washington University 1
Disclosure of Interests • Source of Research Support: – NIMH, NIDA, NIGMS, NIH Blueprint • Consulting Relationships – None • Stock Equity >$10, 00 – None • Participation in Speaker’s Bureaus – None • Pharmaceutical Research Support: – None 2
Thank you to collaborators • Michael Frank (Brown) • Jim Gold (MPRC) • Angus Mac. Donald (U of Minn) • Cam Carter (UC Davis) • Dan Ragland (UC Davis) • Students and Postdocs – Adam Culbreth – Erin Dowd – Erin Moran 3
Heuristic model of components of motivation & goal-directed action 1. Hedonics - “liking” Opioid and GABA in BG, OFC RDo. C: Initial Response to Reward 2. Reward Prediction/Anticipation - “wanting” DA, BG and ACC RDo. C: Reward Anticipation / Reward Prediction Error 4. Represent, update, and maintain values OFC RDo. C: Reward Valuation Cost-Benefit Analysis 3. Implicit and Explicit Reinforcement Learning Implicit: DA, BG; Explicit ACC, OFC, DLPFC RDo. C: Reinforcement Learning 5. Compute effort relative to reward value DA, ACC RDo. C: Effort Valuation 6. Generate action plans to obtain valued outcomes DLPFC RDo. C: Action Selection/ Preference Based Decision. Making “Motivated” Behavioral Response Components of Reward to Goal Directed Behavior Translation Note: After Barch and Dowd (2010) 4
Pathway to motivational Impairments SCHIZOPHRENIA ✔Implicit Reinforcement Learning ✔Hedonics, ✔Reward Responsivity Reward Prediction Effort Allocation Reduced Motivated Behaviors ê Cognitive Control ê Explicit Reinforcement Learning 5
Pathway to motivational Impairments SCHIZOPHRENIA ✔Implicit Reinforcement Learning ✔Hedonics, ✔Reward Responsivity 6
“Implicit” Positive Reinforcement Learning Signal Detection Model (Bias vs. Dprime) Pizzagalli et al. , 2008 7
Implicit Positive Reinforcement Learning Intact in Schizophrenia BIAS Heerey & Gold 2008 8
And in schizoaffective … Sample CON: N=60 BP: N=50 SCZ: N=65 SCZAFF: N=53 Age, Sex and Parent Education Similar Barch et al. , 2018, Journal of Abnormal Psychology 9
Pathway to motivational Impairments SCHIZOPHRENIA ✔Implicit Reinforcement Learning ✔Hedonics, ✔Reward Responsivity Reward Prediction ê Cognitive Control ê Explicit Reinforcement Learning 10
Pathway to motivational Impairments SCHIZOPHRENIA ✔Implicit Reinforcement Learning ✔Hedonics, ✔Reward Responsivity Reward Prediction ê Cognitive Control ê Explicit Reinforcement Learning 18
Explicit Reinforcement Learning – not so good! PAIR TYPE SCZ Accuracy CONTROL CTL N = 37 SCZ N = 38 Dowd et al. , 2016, Biological Psychiatry: CNNI 19
Explicit Reinforcement Learning – not so good – especially for learning from positive outcomes Test Phase Transfer Measures Model Gain and Loss Learning Rates** *Gold et al. , saw same thing first Dowd et al. , 2016, Biological Psychiatry: CNNI ** After exclusion of 11 SCZ and 10 CON who failed to perform above chance 20
Prediction Error Related Activity Prediction error effects Positive Modulation Negative Modulation Deactivation with Positive Modulation Prediction Error Intact CON SCZ Prediction Error Modulation Dowd et al. , 2016, Biological Psychiatry: CNNI 21
Reduced Activity in Cognitive Control and Error Regions SCZ have reduced responses to “B” (bad) choices in at regions associated with cognitive control Superior Parietal Cortex Left Anterior PFC Dorsal Anterior Cingulate Thalamus Dowd et al. , 2016, Biological Psychiatry: CNNI 22
Relationship to Performance Dowd et al. , 2016, Biological Psychiatry: CNNI 23
Explicit Reinforcement Learning in SCZ Roshan Cools Reversal Learning Task Sample CON: N=40 SCZ: N=50 Age, Sex and Parent Education Similar Culbreth et al. , 2016, Biological Psychiatry: CNNI 24
Intact Prediction Error Signals in Striatum Using Model Based Analyses (Q-learning) Saw same intact striatal prediction errors in Dowd et al. , 2016, independent sample Culbreth et al. , 2016, Biological Psychiatry: CNNI 25
Activity in Cognitive Control and Error Regions Anterior Cingulate Thalamus Dorsal Parietal Insula Dorsal Prefrontal Orbital Frontal Activity in Frontal, Insular, Cingulate and Parietal Regions (but not thalamus, occipital, cerebellar, or putamen regions) predicts behavior and mediates group effect Culbreth et al. , 2016, Schizophrenia Bulletin 26
Pathway to motivational Impairments If not striatally mediated prediction error. . Then what? Cognitive Control, including Model Based Learning and Errors 27
Daw “Two Step” Task • Participants were told that they were space travelers in search of space treasure. • They needed to pick a spaceship to take them to a certain planet, where they would ask an alien for treasure. 28
Daw “Two Step” Task % • Payoff rates of second stage choices fluctuate to ensure learning throughout the task Red Planet % Purple Planet 70 • Second stage choice is probabilistically rewarded Stage 2 70 • Choice in first stage that leads probabilistically to each second stage state, “planets” Transition Stage 1 29
Model-Free Stage 2 % 70 % Red Planet 70 Purple Planet Transition Model-Based Stage 1 Participant Data Variable of interest: Is State A choice repeated or not given Previous Reward and Transition Type? Daw et al. , 2011 30
Healthy Control Choice Behavior Model-Free Stay Frequency 1 0. 75 Model-Based 0. 5 Common Reward Coefficient Intercept Reward Rarity Reward * Rarity Rare Reward Common Rare Unreward Estimate (SE) 1. 02 (0. 18) 0. 79 (0. 16) -0. 31 (0. 08) 0. 81 (0. 19) p-value <0. 001 Culbreth et al. , 2016, Journal of Abnormal Psychology Daw Participant Data 31
Schizophrenia Choice Behavior Stay Frequency 1 0. 75 Model-Free Model-Based 0. 5 Common Reward Coefficient Intercept Reward Rarity Reward * Rarity Rare Reward Common Rare Unreward Estimate (SE) 1. 02 (0. 16) 1. 22 (0. 23) -0. 10 (0. 06) 0. 25 (0. 15) p-value <0. 001 0. 09 0. 1 Culbreth et al. , 2016, Journal of Abnormal Psychology Daw Participant Data 32
Choice Behavior Stay Frequency 0. 9 0. 8 0. 7 CN SZ 0. 6 0. 5 Common Reward Rare Reward Common Unreward Culbreth et al. , 2016, Journal of Abnormal Psychology Rare Unreward 33
Other Way To Distinguish Trial Structure – Win or Not Win + Fixation 1000 msec Choice Period Up to 8000 msec OR WIN! + Fixation 1000 msec Choice Period Up to 8000 msec + Feedback Period 1000 msec Trial Structure – Not Lose or Lose Not a winner. Try again! Feedback Period 1000 msec + OR LOSE! Andra Geana, Michael Frank, Jim Gold and the CNTRACS Crew Keep your money! 34
Other Ways To Distinguish Andra Geana, Michael Frank, Jim Gold and the CNTRACS Crew 35
Other Ways To Distinguish Andra Geana, Michael Frank, Jim Gold and the CNTRACS Crew 36
Other Ways To Distinguish Andra Geana, Michael Frank, Jim Gold and the CNTRACS Crew 37
Other Ways To Distinguish (smoothed) Learning curves: average accuracy (across subs) for all learning trials (left) and average accuracy across subs AND across trials (right) Andra Geana, Michael Frank, Jim Gold and the CNTRACS Crew 38
Other Ways To Distinguish - Mixing param: eter weight of Q-learning vs Actor-Critic strategy (0 -1, closer to 0 means more A-C, closer to 1 means more Q-learning) - Beta: softmax parm (higher means more sensitive to reward differences) - Random Noise: value-independent decision-noise - Decay: memory decay between learning and test phase - Q 0: initial values for the stimuli, before any observed outcomes Andra Geana, Michael Frank, Jim Gold and the CNTRACS Crew 39
Pathway to motivational Impairments SCHIZOPHRENIA ✔Implicit Reinforcement Learning ✔Hedonics, ✔Reward Responsivity Reward Prediction Effort Allocation ê Cognitive Control ê Explicit Reinforcement Learning 40
Pathway to motivational Impairments EXPECTED VALUE OF COGNITIVE CONTROL Shenhav et al. , 2013; 2016 41
Effort Discounting Framework • Experience increasingly difficult levels of the N-Back • Make choices of repeating an easier version for less money, or a more difficult version for more money 1 -back 4 -back for for $1. 00 $2. 00 1 -back 4 -back $1. 50 $2. 00 1 -back 4 -back for for $1. 25 $2. 00 1 -back 4 -back $1. 38 $2. 00 for $1. 44 $2. 00 • Titrated the “ 1 back” over fivetrials (assumed indifference point) 42
Cognitive Effort Discounting Task SZ patients less willing to complete more demanding tasks for increased rewards. Group Beta: p = 0. 08; p < 0. 001 at 2 -Back Culbreth et al. , in submission Culbreth et al. , 2016 43
Area Under the Curve AUC = 1. 0 AUC = 0. 0 Culbreth et al. , in submission 44
Negative Symptom Associations Negative symptoms were trend-level associated with effort expenditure (r = -0. 39 p = 0. 07). Culbreth et al. 2020 (Culbreth et al. , 2016) 45
More Ecological Momentary Assessment 46
Ecological Momentary Assessment r = 0. 45 Moran et al. , 2017 r = 0. 27 47
Neural Correlates • Individuals made effort-based decisions in the scanner between cognitive tasks. • Putative difficulty of decisions varied creating “easy” and “hard” decisions • Whole-brain analyses; 3 T Siemens Skyra Connectome Scanner Sample CON: N=30 SCZ: N=28 Age, Sex and Parent Education Similar • Contrast: Hard (N = 48) > Easy Trials (N = 24) Easy Decision Hard Decision 1 -back 4 -back for $2. 00 1 -back 4 -back for $1. 00 $2. 00 Culbreth et al. , 2020 48
A priori ROIs – Hard Vs. Easy Choice Significant reduction in patients, but did not pass FDR Culbreth et al. , 2020 49
A priori ROIs – Hard Vs. Easy Choice Culbreth et al. , 2020 50
A priori ROIs – Hard Vs. Easy Choice Culbreth et al. , 2020 51
Pathways to motivational Impairments SCHIZOPHRENIA ✔Implicit Reinforcement Learning ✔Hedonics, ✔Reward Responsivity Reward Prediction ê Cognitive Control ê Explicit Reinforcement Learning ê Rewards to Guide Behavior Effort Allocation Reduced Motivated Behaviors 52
Lots of Open Questions/Challenges • Similar or different than other forms of psychopathology – Spoiler alert … likely different than depression related amotivation – Link more specificially to abilty to represent futur rewards • Developmental course of these deficits • Impact (or lack thereof …) of treatment • ECOLOGICAL VALIDITY!!!! – Most of the research is done with “rarified” paradigms – We and others have started to translate this into the types of everyday behaviors that are a problem in individuals with severe mental illness. – Do computational parameters relate better? Same? 53
- Slides: 46