Extinction of Conditioned Behaviour Extinction CS without US

Extinction of Conditioned Behaviour

Extinction • • • CS without US Response without outcome Not simply a reversal of acquisition Not the same as forgetting New learning An “inhibitory relationship”

Real Life • Common (and necessary!) occurrence • Adaptation to changed conditions • E. g. , Stop calling a friend after they stop returning calls

Effects of Extinction • • Continue original behaviour for a time Increase behaviour Vary behaviour E. g. , Call friend more; wait for friend after work.

Neuringer et al. (2001) • • Rats, operant chamber Two levers and a key Three responses in a row to receive food Group 1: had to vary response pattern Group 2: no variation required (yoked) Acquisition phase Put on extinction

Results • Variation in behaviour • Change in response rate

Emotional Effects • Frustration • Emotional reaction induced by withdrawal of expected reinforcer • Intensifies behaviour • Aggression

Tomie et al. (1993) • • • Rats Water deprived 3 min. VT-30 sec delivery of water 3 min. no water (ext. ), signaled by tone (S-) Target bite bar (plexiglass wrapped in tape) Target biting a sign of frustration in rats; readily produced by delivering aversive stimulus (e. g. , Azrin et al. 1968)

Results

Azrin et al. (1966) • Pigeons • Conditioned to peck key under alternating periods of food reinforcement and extinction • Restrained pigeon or stuffed pigeon model in chamber attacked during extinction

Design

Attack Behaviour • Cumulative records of 3 pigeons • Pen stepped up for each 1 sec. of attack

Length of Attacks • Average duration of attack post termination of food reinforcement for different pigeons

Stuffed Model • Remarkably similar behaviour

Number of Food Deliveries • Number of food reinforcements before extinction implemented • 0, 1, 3, 5, 10, or 30 food deliveries • Stuffed pigeon • Positive correlation between amount of food and attack duration

Data

Implications for Therapy • Extinction acts as aversive • Aversives create frustration • Frustration can produce aggressive behaviour • Directed against: therapist, anyone in proximity, self

Extinction and Original Learning • Not a reversal • Forgetting a different underlying process • Actually a new acquisition of learning

Disinhibition • Fully condition CS with US • Impose extinction protocol • Present novel stimulus along with extinguished CS • CR will reoccur

Why “Disinhibition”? • Pavlov’s terminology • Excitatory conditioning – Increase in excitatory strength • Extinction – Inhibition of excitatory conditioning – Net sum effect – Full extinction = excitation + inhibition = 0

• Disinhibition inhibits the “extinction inhibition” • Temporarily reduces strength of inhibition • Excitation + (inhibition + disinhibition) > 0 • A temporary effect

Parallels Dishabituation • Temporary return of habituated response without rest period • Three ways – New stimulus presented with habituated stimulus (e. g. , Graves & Thompson, 1970) – Change habituated stimulus (e. g. , Fisher, 1962 with the Coolidge effect) – Change context of habituation (e. g. , Schein & Hale, 1974) • Temporary sensitization process superimposed over habituation process

Spontaneous Recovery: Classical • • • Fully condition CS-US Fully extinguish CS Let some time pass Present CS Will get return of CR

Rescorla (1997) • Goal tracking for two different CSs • Full conditioning for all groups followed by full extinction • CS-Rest: test session 8 days after extinction • CS-No Rest: test session immediately after extinction • Pre-CS: control group

Results

Spontaneous Recovery: Operant • • Condition three term contingency Extinguish response Allow time to pass Response will return in presence of SD

Rescorla (1996) • Rats • Responses (lever press or nose poke) acquired, then extinguished • R-Rest: tested 7 days post-extinction • R-No rest: tested shortly after extinction

Results

Renewal • Recovery of acquisition of performance when context cues present during extinction are changed • If extinction is learning another three term contingency, then changing the cues eliminates the SD for extinction • Think of this in terms of stimulus control

Bouton & King (1983) • • Rats press lever food Tone (CS) paired with footshock Training in two chambers Post training, 20 extinction trials – Group 1 in original (A) chamber – Group 2 in novel (B) chamber – Group 3 had no extinction (control) • All groups tested for response in chamber A

Results

Renewal Also In: • • Classical appetitive conditioning Conditioned inhibition Instrumental conditioning Physiological states, such as from drugs can also act as the SD for extinction that can be renewed (e. g. , Bouton et al. , 1990)

Reinstatement • Recovery of the excitatory responding to an extinguished stimulus produced by exposure to the US • Example: fear of flying – Extinguish fear through therapy – Have one frightening flying experience – Phobia re-established to high level

Bouton (1984) • Conditioned suppression in rats • US = shock Phase 1 Phase 2 Reinstatement Test CS --> weak US (weak CR) No treatment US, same CS--> (weak CR) context as test weak CR CS --> weak US (weak CR) No treatment US, different CS --> (weak CR) context as test weak CR CS --> strong US (strong CR) Extinction (weak CR) US, same CS --> context as test strong CR CS --> strong US (strong CR) Extinction (weak CR) US, different CS --> context as test weak CR • Boulton suggests reinstatement may be subset of renewal (US activates context cues)

Sensitivity to US Devaluation • Utilize US devaluation to determine if CS-US association persists through extinction • Show S-R and R-O association maintained post-extinction

Study Design • • Two CS (light, tone), two US (food, sucrose) Counterbalanced across subjects US devaluation via Li. Cl Note: need to recondition extinguished CSs to different US to get measurable CR in test Phase 1 Phase 2 Phase 3 Test L --> food T --> sucrose L extinguished T extinguished food devalued L…? T…? L --> food T --> sucrose

Results Not. Dev CR Strength Not. Dev Light Tone Dev Extinguished Not extinguished • Extinguished group shows weaker CR • Within groups, devalued stimulus shows even weaker CR; pre-extinguished CS-US association still affected

Study Design • Different responses, different outcomes • Note: recondition R 1 & R 2 to new O predevaluation Phase 1 Phase 2 Outcome Devaluation Test R 1 --> O 1 R 1 Ext O 1 --> Li. Cl R 1 vs. R 2 & R 3 vs. R 4 R 2 --> O 2 R 2 Ext O 1 --> Li. Cl R 1 vs. R 2 & R 3 vs. R 4 R 3 --> O 1 --> Li. Cl R 1 vs. R 2 & R 3 vs. R 4 --> O 2 O 1 --> Li. Cl R 1 vs. R 2 & R 3 vs. R 4

Results Resposnes Not. Dev. O 2 Dev. Extinguished Not Extinguished O 1

Enhancing Extinction • So extinction doesn’t actually eliminate prior learning • Sometimes extinguished response comes back • Techniques to minimize return of extinguished learning

Number & Timing • More extinction trials! • Space extinction trials closer together (massed) rather than spread out (spaced) – Works with aversive conditioning; don’t really know about appetitive conditioning yet

Reducing Spontaneous Recovery • Repeat periods of rest and testing – Less recovery with each successive cycle • Manipulating interval between acquisition and extinction – Fear conditioning study found less spontaneous recovery with shorter interval – Appetitive conditioning study found the opposite • Present cues associated with extinction – Reactivates extinction performance

Reducing Renewal • Conduct extinction in multiple settings – Increases stimulus generalization • Present SD for extinction during renewal

Compound Extinction Stimuli • Present two stimuli undergoing extinction simultaneously

Rescorla (2006) • Rats • Three stimlui: Light, Noise, Tone • Acquisition of lever pressing (VI 30 sec. ) in presence of stimuli • Extinguish each of the stimuli • Compound extinction phase – Light with one of auditory stimuli; other auditory alone

Results Response Rate Elevated responding (summation of subthreshold responding remaining to L & A 1) No recovery of A 1; compound extinction increased A 1’s extinction Extinction Light Substantial spontaneous recovery of A 2 Auditory 1 Compound extinction Auditory 2 Test (6 days later) Light & Auditory 1

What is Learned in Extinction • S-O and R-O associations not eliminated • Current research suggests an inhibitory S-R association • Extinction effects will be highly specific to the context in which the response was extinguished – E. g. , if you never got birthday presents on your birthday as a kid, you won’t be disappointed if you don’t get presents as an adult

Rescorla (1993) • 1. Discrimination training (nose poke --> food) whenever Light or Noise present • 2. Lever press & chain pull (R 1 & R 2) --> food – No S-R association b/t L or N with R 1 or R 2 • 3. Extinction of N: R 1 and L: R 2 – Establishes inhibitory S-R associations • 4. Test – N: R 1 vs. R 2… more R 2 responding – L: R 1 vs. R 2… more R 1 responding • Can’t be due to S-O or R-O effects; has to be S-R

S-R • Think back to our discussion of Central Emotional States • Decline in responding in extinction linked to frustration due to not getting what you expected • Leads to some seemingly odd effects

Overtraining Extinction Effect • The more acquisition trials, the greater the expectancy of reward, hence the greater the frustration when extinction introduced • Produces more rapid extinction • Odd, because you’d expect that more training results in a stronger response that is more resistant to extinction

Magnitude Reinforcement Extinction Effect • More rapid extinction if trained with larger rather than smaller magnitude reinforcer • Expectancy of greater reward produces greater frustration when it is not forthcoming

Partial Reinforcement Extinction Effect • Continuous reinforcement (FR-1) • Intermittent reinforcement (everything else) • Extinction faster on CRF • The less likely each response is of being reinforced, the more difficult it is to extinguish

Discrimination Hypothesis • Easier to notice start of extinction if on FR 1 than e. g. , FR 10 • But PREE may not be due to difficulty noticing start of extinction • Jenkins (1962) – Two groups of pigeons, 1 st on CRF, 2 nd on PRF – Both put on CRF, then immediately on extinction – 2 nd group took longer to extinguish • Effects of PRF long-lasting; “don’t give up in face of failure”?

Frustration Theory • Intermittent reinforcement has rewarded and nonrewarded responses • Rewarded responses motivating • Non-rewarded responses frustrating • Typically some variation in when you get reinforced; sometimes what you expect will be a non-reinforced response produces a reinforcer • Hence, frustrated responses lead to future expectation of reward • On CRF can’t learn to respond when expecting nonreward

Sequential Theory • Memory concepts • You can remember whether or not a response was reinforced in recent past • In PRF training, non-reinforced responses become an “SD” for performing the response • Depends on sequencing – E. g. , R N N R R N R • With experience, learn to respond when remembering not having been reinforced for recent responses

Behavioural Momentum • Analogy with Newtonian physics – Momentum = mass*speed – “A body in motion tends to stay in motion. ” • Behaviour that has a lot of momentum will be hard to disrupt through manipulation • Studied using multiple schedules of reinforcement – Two or more components, each with its own SD and reinforcement schedule – Add disruption (e. g. , extra food between components, extinction, novel salient stimuli, etc. )

Behavioural Momentum • Response rate – Often (but not always) unrelated to behavioural momentum • Rate of reinforcement – Higher produces more momentum; less susceptible to disruption – Seems to be most important element • Implies S-O associations regulate behavioural momentum – Adding non-contingent reinforcers to one schedule increases its behavioural momentum

Mace et al. (1990) VI 60… more reinforcement… more momentum