The partial reinforcement extinction effect PREE Frode Svartdal

  • Slides: 59
Download presentation
The partial reinforcement extinction effect (PREE) Frode Svartdal University of Tromsø

The partial reinforcement extinction effect (PREE) Frode Svartdal University of Tromsø

Extinction: Basics n n Extinction is defined in terms of a reinforcement process Extinction

Extinction: Basics n n Extinction is defined in terms of a reinforcement process Extinction contingencies n n n The stimulus (SR or US) is discontinued The learning contingency is discontinued Extinction process n n The conditioned response is reduced (strength, frequency, etc. ) Relearning, … not forgetting

Extinction: Basics Operant conditioning Catania, 1984)

Extinction: Basics Operant conditioning Catania, 1984)

Extinction: Basics Classical conditioning

Extinction: Basics Classical conditioning

Factors affecting the extinction rate n n In general: Fast acquisition / high rate

Factors affecting the extinction rate n n In general: Fast acquisition / high rate of responding fast extinction Amount of reward n n Variability n n High fast extinction Stimulus Response Reinforcement = high ext. persistence Some forms of learning do not extinguish (easily) n Evaluative conditioning (e. g. , Diaz, Ruiz, & Beyens, 2005)

Factors affecting the extinction rate n Partial Reinforcement Extinction Effect n n Partial (Intermittent)

Factors affecting the extinction rate n Partial Reinforcement Extinction Effect n n Partial (Intermittent) Reinforcement (PRF) increased extinction response Continuous Reinforcement (CRF) reduced extinction persistence

First demonstrations Operant conditioning; free operant; rats; Skinner (1938) 50% 100% Classical conditioning; blink

First demonstrations Operant conditioning; free operant; rats; Skinner (1938) 50% 100% Classical conditioning; blink response; students; Humphreys (1939)

Free operant Ferster & Culbertson, 1975

Free operant Ferster & Culbertson, 1975

Free operant Compared to CRF: PRF • higher asymptotes • more persistent responding under

Free operant Compared to CRF: PRF • higher asymptotes • more persistent responding under extinction EXTINCTION PRF CRF

Rats, maze running speed under extinction (Weinstock, 1954) PRF (30%) CRF

Rats, maze running speed under extinction (Weinstock, 1954) PRF (30%) CRF

Classical conditioning (rats): PREE Extinction 25% PRF response rate LOWER than CRF response rate

Classical conditioning (rats): PREE Extinction 25% PRF response rate LOWER than CRF response rate 50% 100% 15%

Classical conditioning; eyelid; human subjects (Svartdal & Flaten, in prep. )

Classical conditioning; eyelid; human subjects (Svartdal & Flaten, in prep. )

Operant conditioning; humans; Svartdal, 2003, Exp. 4

Operant conditioning; humans; Svartdal, 2003, Exp. 4

Conclusions (… preliminary) n PREE is a very robust outcome n Measures & species

Conclusions (… preliminary) n PREE is a very robust outcome n Measures & species n n n Bar pressing, rats Maze running, rats Pecking, pigeons Blink reflex, humans, rabbits … Contingency n Operant/instrumental n n n Discrete trial Free operant Classical

But… n How general is the PREE? n n n Alternative methods of analysis

But… n How general is the PREE? n n n Alternative methods of analysis n n Reversed PREE observed under some conditions Generalized PREE observed under some conditions Nevin (1988): ”PREE is an artefact because of wrong method of analzing extinction performance” Response unit issue n PREE or not dependig on how the response is defined (Mowrer & Jones, 1945!

Reversed PREE What happens if the subject is exposed to a mixture of PRF

Reversed PREE What happens if the subject is exposed to a mixture of PRF and CRF contingencies?

Reversed PREE Pavlik & Carlton, 1965: Rats; bar pressing, free operant n n n

Reversed PREE Pavlik & Carlton, 1965: Rats; bar pressing, free operant n n n Gr. 1: Single contingency; CRF Gr. 2: Single contingency; PRF Gr. 3: Two signalled schedules alternated for the same subjects; CRF + PRF

Reversed PREE Conventional PREE

Reversed PREE Conventional PREE

Reversed PREE

Reversed PREE

Reversed PREE Pavlik & Carlton (1965): n n Single reinforcement schedules (CRF vs. PRF)

Reversed PREE Pavlik & Carlton (1965): n n Single reinforcement schedules (CRF vs. PRF) in betweengroups experiments PREE Two schedules (CRF vs. PRF) for the same subjects Reversed PREE Other research n n n Reversed PREE observed Generalized PREE (overall increased persistence, but no difference between conditions) Conventional PREE rarely if ever observed in within-subjects manipulations of CRF - PRF

PREE as a generalization: Ecological validity If applied to a situation with a very

PREE as a generalization: Ecological validity If applied to a situation with a very specific schecule for a specific behavior PREE Example: Single mother – child is begging for toys only from mom If applied to various situations with mixed contingencies Reversed PREE Generalized PREE Example: Mother and father – child begs for toys from both

PREE as a generalization n Relevance to ADHD n n Complex schedules RPREE or

PREE as a generalization n Relevance to ADHD n n Complex schedules RPREE or GPREE, not PREE Factors associated with slower learning slower extinction n n Attentional problems, difficulties with concentration, memory, … Would add to biological factors

Response unit issue

Response unit issue

Free operant responding: What is the response unit? Mowrer & Jones, 1945: What should

Free operant responding: What is the response unit? Mowrer & Jones, 1945: What should be counted as the response unit - single responses or the unit of responses required for reinforcement? n n Free-operant Intermittent reinforcemet, e. g. , FR 4

Response unit FR 4 Reinforced responses

Response unit FR 4 Reinforced responses

PREE Total responses Reversed PREE Total responses / reinforcement ratio

PREE Total responses Reversed PREE Total responses / reinforcement ratio

Nevin: PREE is an artefact

Nevin: PREE is an artefact

PREE: Alternative analyses Nevin, 1988: Behavioral momentum • ”RPREE” is the rule – the

PREE: Alternative analyses Nevin, 1988: Behavioral momentum • ”RPREE” is the rule – the response is stronger following CRF • • • in free-operant responding (but not in discrete-trial experiments) following extended training Extinction performance • • Traditional measure: Number of responses Nevin: Slope of the extinction curve

SHORT LONG Nevin, 1988 Absolute number of responses PREE Relative to initial ext response

SHORT LONG Nevin, 1988 Absolute number of responses PREE Relative to initial ext response level RPREE

PREE vs. RPREE – important variables n Dependent measure n n Type of situations

PREE vs. RPREE – important variables n Dependent measure n n Type of situations n n Free operant vs. discrete trial Complexity of situation n n No. of responses vs. relative change One vs. more schedules (e. g. , multiple schedule) Design n Between groups vs. within subjects

PREE typically observed Measure Number of responses Situation Discrete trial Schedule Single Design Between-groups

PREE typically observed Measure Number of responses Situation Discrete trial Schedule Single Design Between-groups manipulation of reinforcer rate Other CRF schedule must be 100%

PREE: My interests n n Interaction PREE & Reversed PREE Cognition (verbalization) related to

PREE: My interests n n Interaction PREE & Reversed PREE Cognition (verbalization) related to behavioral PREE

The experimental situation ”Computer responses” presented Left, right Subject responses recorded Left, right

The experimental situation ”Computer responses” presented Left, right Subject responses recorded Left, right

The experimental situation Task n Complete a four-response chain of responses started by the

The experimental situation Task n Complete a four-response chain of responses started by the computer n n ”Obtain as many correct answers as you can. ” Rules (depending on experiment) n n n Subject: R L Instructed task: Identify and apply the functional rule(s) n n E. g. : Computer: L R ”Repeat computer sequence” ”Reverse computer sequence” Feedback (visual, autitory) for correct answer; nothing happens if answer is incorrect

The experimental situation Manipulations (between groups and/or within groups) Rule Reverse (typically used) Repeat

The experimental situation Manipulations (between groups and/or within groups) Rule Reverse (typically used) Repeat Contingency CRF (100%) PRF (20 -60%)

The experimental situation n Reward rate manipulated n n n Between groups Within subjects

The experimental situation n Reward rate manipulated n n n Between groups Within subjects (multiple schedule) Discrete trial situation; fixed number of trials n n 180 acquisition trials 40 extinction trials

Conventinal PREE; operant responding; students; Svartdal, 2003, Exp. 4

Conventinal PREE; operant responding; students; Svartdal, 2003, Exp. 4

Reversed & conventional PREE; operant responding; students; Svartdal, 2000 Reversed PREE n n n

Reversed & conventional PREE; operant responding; students; Svartdal, 2000 Reversed PREE n n n Purpose: Explore the relationship between PREE and RPREE vs. RPREE: Contradiction or compatible effects? Method n n Independent groups: PRF and CRF Within: CRF and PRF

Svartdal, 2000 ctd. n n Multiple schedule, alternating Group 40/40 n n n PRF

Svartdal, 2000 ctd. n n Multiple schedule, alternating Group 40/40 n n n PRF Group 80/80 n n n Half trials (signalled): 40% Half trials (signalled): 80% ”CRF” Group 80/40 n n Half trials (signalled): 80% Half trials (signalled): 40% ”CRF” + PRF

* No. of responses: RPREE * Relative change: No difference PREE 80% 40%

* No. of responses: RPREE * Relative change: No difference PREE 80% 40%

Svartdal, 2000 ctd. n n Relationship between schedule components Simplest assumption: Modulation between component

Svartdal, 2000 ctd. n n Relationship between schedule components Simplest assumption: Modulation between component schedules: n 60% + context = 60% reference n 60% + context = 100% reduced persistence n 60% + context = 20% increaced persistence

Performance of a 60% schedule depending on other schedule = 100%, 60%, or 20%

Performance of a 60% schedule depending on other schedule = 100%, 60%, or 20% Svartdal, 2000

Svartdal, F. (2000). Persistence during extinction: Conventional and Reversed PREE under multiple schedules. Learning

Svartdal, F. (2000). Persistence during extinction: Conventional and Reversed PREE under multiple schedules. Learning and Motivation, 31, 21 -40.

Cognition in PREE • Currently: Strong cognitive arguments to interpret conditioning in terms of

Cognition in PREE • Currently: Strong cognitive arguments to interpret conditioning in terms of cognition • • • Classical conditioning: Lovibond & Shanks, 2002 Operant conditioning: Shanks & St John, 1994 Implicit learning doubted: Shanks, 2005 Extinction: Lovibond, 2004 Basic argument: CONTINGENCY CONSCIOUS APPREHENSION BEHAVIORAL CHANGE CONTINGENCY CONSCIOUS APPREHENSION NO BEHAVIORAL CHANGE • Large number of studies supporting this assumption

Cognition in PREE n n So, since the behvioral PREE is very robust, a

Cognition in PREE n n So, since the behvioral PREE is very robust, a ”cognitive PREE” must be easy to measure Basic prosedure: n Behavioral acquisition under 100% vs. 60% reinforcer rate n Measurement of verbalized PREE

Cognition in PREE Prediction of persistence: ”How likely is it that you will continue

Cognition in PREE Prediction of persistence: ”How likely is it that you will continue responding if reward no longer appears? ” Several experiments have demonstrated no sensitivity to learning history in predictions

3 extinction trials; immediate behavioral sensitivity No difference in predictions Svartdal & Silvera, in

3 extinction trials; immediate behavioral sensitivity No difference in predictions Svartdal & Silvera, in prep.

Cognition in PREE Retrospective judgments: ”How many responses did you emit after reward no

Cognition in PREE Retrospective judgments: ”How many responses did you emit after reward no longer appeared? ” Subjects are very accurate in descrbing their own behavior, including their own extinction persistence

Cognition in PREE Svartdal, F. (2003). Extinction after partial reinforcement: Predicted vs. judged persistence.

Cognition in PREE Svartdal, F. (2003). Extinction after partial reinforcement: Predicted vs. judged persistence. Scandinavian Journal of Psychology, 44, 55 -64.

Meta-cognitive PREE? n n We all have long experience with various contingencies Maybe a

Meta-cognitive PREE? n n We all have long experience with various contingencies Maybe a ”meta-cognition” evolves: n n Uncertain outcomes Persist Certain outcomes Quit

Meta-cognitive PREE? n Scenarioes presented to subjects, manipulation n Reliable outcome vs. Unreliable outcome

Meta-cognitive PREE? n Scenarioes presented to subjects, manipulation n Reliable outcome vs. Unreliable outcome Persistence judgments of behavior

Meta-cognitive PREE? Naive students: No effect of outcome manipulation

Meta-cognitive PREE? Naive students: No effect of outcome manipulation

Meta-cognitive PREE? Psychology students Naive students (have read about PREE)

Meta-cognitive PREE? Psychology students Naive students (have read about PREE)

Meta-cognitive PREE? Svartdal, F. (2000). Persistence during extinction: Are judgments of persistence affected by

Meta-cognitive PREE? Svartdal, F. (2000). Persistence during extinction: Are judgments of persistence affected by contingency information? Scandinavian Journal of Psychology, 41, 315 -328.

PREE: Theory Mowrer & Jones: Diskriminasjonshypotesen n PRF: n n n Læringbetingelsene ekstinksjonsbetingelsene Generalisering

PREE: Theory Mowrer & Jones: Diskriminasjonshypotesen n PRF: n n n Læringbetingelsene ekstinksjonsbetingelsene Generalisering til ekstinksjon CRF: n n Læringbetingelsene # ekstinksjonsbetingelsene Liten generalisering til ekstinksjon

PREE: Theory Amsel: Frustrasjonshypotesen n PRF: n n n Forventning om belønning frustrasjon når

PREE: Theory Amsel: Frustrasjonshypotesen n PRF: n n n Forventning om belønning frustrasjon når belønning uteblir Frustrasjons-cues assosieres med læringssituasjonen Under ekstinksjon: Frustrasjon pga uteblitt belønning Læringssituasjonen ekstinksjonssituasjonen CRF: n n Frustrasjon oppstår ikke under læring Læringssituasjonen # ekstinksjonssituasjonen

PREE: Theory Capaldi: Sequential hypothesis n PRF: n n Ikke-belønnede trials blir signal på

PREE: Theory Capaldi: Sequential hypothesis n PRF: n n Ikke-belønnede trials blir signal på at belønning snart vil følge: … N N N R … Dvs. : Det opparbeides en forventning om belønning når belønning uteblir Under ekstinksjon: Mange responser pga forventning om belønning CRF: n n Ingen erfaring med uteblitt belønning under læring Under ekstinksjon: Få responser

PREE: Theory n Status: n n n Discrete-trial-situasjonen n n Diskriminasjonshypotesen står svakt Amsels

PREE: Theory n Status: n n n Discrete-trial-situasjonen n n Diskriminasjonshypotesen står svakt Amsels hypotese står rimelig sterkt Capaldis hypotese står ganske sterkt Nevins modell: Ingen hypotese i vanlig forstand Capaldi og Amsel dominerende Fri-operant-situasjonen n Svak teoretisk forståelse

Evaluative conditioning n Neutral stimulus n n n ζ Neutral stimulus paired with US

Evaluative conditioning n Neutral stimulus n n n ζ Neutral stimulus paired with US n n E. g. , ζ + MURDER Extinction n ζ alone Test n ζ is evaluated in accordance with MURDER