Indirect Acting Contingencies Feedback PSY 6450 Unit 4
Indirect Acting Contingencies & Feedback PSY 6450 Unit 4 Schedule: Wednesday, 10/03 and Monday, 10/08: Lecture Wednesday, 10/10: Exam (MAC opening event) Monday, 10/15: ME 1 study objectives handed out Monday, 10/22: ME 1 Questions over ME 1 process? 1
Introduction: indirect acting contingencies n n n When dealing with changing the behavior of human adults, certainly workers, we are usually dealing with indirect acting contingencies, not direct acting contingencies. In order for consequences to affect behavior directly, they must occur within 60 seconds or so after the behavior (molecular vs. molar perspective) You often hear: n n If workers go on strike and management gives in, management has reinforced workers for going on strike The sales commission that sales rep receive once a month reinforces their sales behaviors 2
Molecular vs. molar, NFE § § Molecular perspective: Consequences must follow behavior immediately (or almost immediately) in order for them to affect behavior (temporal contiguity) Molar perspective: Consequences do not have to be temporally contiguous; only correlated with behavior over and over again – the delay between the behavior and the consequence is not important 3
The Western Way § Western way has historically been molecular § § Michael, Malott, Fuqua, Van Houten, Mc. Gee, Johnson, Dickinson That’s why Michael and Malott go to such effort to distinguish between direct and indirect acting contingencies and analyze molecular variables that may account for the effectiveness of long-delayed consequences (Reno & FIT, same as WMU) 4
The Western Way In 2010, Bradley and Poling surveyed the Board of Editors of the main BA journals (TAVB, 2010) They used Michael’s grant writing example from this unit Respondents did not agree with respect to whether temporal contiguity was important (table with data next) 5
Michael’s Example A person applies for a research grant and then six months later gets a letter in the mail informing him he has gotten the grant. The person then writes more grants. Is the grant money reinforcement for writing the grant? 6
Results: Money is a reinforcer 7
Results: Money is a reinforcer, but dependent on verbal behavior (of those that said true) 8
Conclusions n Majority of members of the BOEs of behavioral journals are willing to call delayed consequences reinforcers: Range 47%-71% n n Majority of those who are willing to call delayed consequences reinforcers maintain they are mediated by verbal behavior n n This is not my position; my position is the minority position Delayed consequences are reinforcers even if they are mediated by verbal behavior Over 1/3, however, maintain that delayed consequences are direct acting (molar perspective and correlational-based law of effect) n No need to distinguish between direct acting and indirect acting contingencies; no need to invoke verbal mediation 9
SO 1 A: Three positions regarding delayed consequences n Delayed consequences are not reinforcers/punishers even if mediated by verbal behavior n n Delayed consequences are reinforcers/punishers even if mediated by verbal behavior n n Molecular I don’t know how to label this perspective! Delayed consequences are direct acting reinforcers/punishers n Molar 10
Molecular vs. Molar n n These results are very similar to the results of a study conducted by Schlinger, Blakely, Fillhard, & Poling conducted 20 years ago (TAVB, 1991) Interesting questions are raised by this: n Our field does not agree on one of the very basic concepts: do consequences have to be temporally contiguous? n Most people maintain that a consequence does not have to be temporally contiguous to be called a reinforcer/punisher as long as a functional relation can be established, however, most also believe effects of delayed consequences are due to verbal behavior n Some said you needed to define “immediate” – within 5 -60 seconds wasn’t “immediate” 11
SO 1 B: Malott, reinforcement vs. reinforcer n n Malott on reinforcement (molecular) In order to be called a direct acting contingency, the consequence must follow the behavior immediately Malott on reinforcer A stimulus can be called a reinforcer even if it is delayed if, and only if, it would increase behavior if it immediately followed it Michael on reinforcer A stimulus can only be a reinforcer if it immediately follows a behavior – that is reinforcers are context specific. Thus, while Malott would agree with Michael that the grant writing example is not an example of reinforcement, unlike Michael, he is willing to call the money a reinforcer if it would increase some behavior when presented immediately after it 12
Molecular vs. Molar in OBM § Strong advocates for molar § § Many simply ignore the issue but often analyze things from a molar perspective § § Mawhinney & Hantula That is, they talk about a consequence being a direct “reinforcer” even it is delayed by a week, month, etc. Ultimately, it’s an experimental question § Nonhuman data indicate that delayed consequences can affect behavior, but the delays have not exceeded 45 s – and, in addition, that you begin to “lose” behavior even after 2 s (6, 4, 4, 3 rats of 8, 0”, 15”, 30”, 45”; Synerski, Laraway, Poling, 2005) 13
How do molecular advocates deal with the effectiveness of molar interventions? n n The effectiveness of molar interventions is due to changes in the molecular contingencies When you change the molar contingencies you also change the molecular contingencies. The effectiveness of molar interventions is mediated by verbal behavior (rule-governed behavior) This can account for the variability in performance when you implement interventions with adults – individuals may formulate different “rules”. For this reason, I also believe that it is very important for all OBM practitioners to understand verbal behavior. It may not be necessary for practitioners to understand VB as long as they collect data and evaluate their interventions but it sure can prevent some theoretical misconceptions! 14
SO 3: Michael, indirect contingencies n n Michael makes a very complex argument with respect to the grant writing example. Example A person applies for a research grant and then six months later gets a letter in the mail informing him he has gotten the grant. And then the person writes more grants. Many would call getting the grant money reinforcement for writing the grant. But Michael says, “NO! It is not, even though grant writing may increase. ” (not going to talk about SO 2) 15
SO 3, cont. n n n I want to go through his argument carefully and head off a problem Michael is NOT arguing that the delay is the main problem. Many Malottians say that - and while that is a problem, it is not Michael’s main argument. (he wants to convince molar folk as well, so he is not arguing the point simply on the delay issue – on the molecular vs. molar argument I just presented but they are related) Rather, Michael is basing his argument on the “automaticity of reinforcement. ” n That is, that operant conditioning is automatic - if a reinforcer follows a behavior, that behavior will increase in the future. 16
Michael’s example again A researcher writes a research grant and six months later receives a letter indicating he received the money. As a result, he writes more grants. Now assume the researcher writes a research grant and six months later receives a letter telling him he has received the same amount of money as an inheritance. 17
Michael’s argument n n I think we would all agree (as would Michael) that the inheritance money would not increase grant writing But, Michael’s point is if operant reinforcement was at work, that is, if we were dealing with direct acting contingencies, then both the grant money and the inheritance money should increase grant writing If money is a reinforcer, then it should increase grant writing regardless of whether it was money from a granting agency or an inheritance Why? Because of the automaticity of reinforcement. A reinforcer will increase any behavior it follows - the organism doesn’t have to understand “why” he/she is getting it - all that is important is that the Sr follows behavior 18
SO 3, cont. n Others would say: n n (the molar perspective) Those situations are very different! Receiving grant money is causally related to writing the grant while the inheritance money is not. Therefore, it IS simple reinforcement How does a person “know” the money is causally related to the grant? n n A lot of other behaviors occurred in between Without a complex verbal repertoire (indirect acting cntg), the grant money would not increase grant writing 19
Michael’s unstated underlying argument n n Reinforcement increases behavior when consequences are causally related to the behavior, when there is an ifthen relationship between behavior and the consequence However, we also know that “adventitious” (noncontingent) reinforcement also increases behavior n n n Reinforcement that just “happens” to follow a behavior will increase that behavior as well That is, behavior can be accidentally or “adventitiously” reinforced SO 3 B: Why/how is the following example related to Michael’s argument about grant writing? 20
Contingent Reinforcement: FR 1 Reinforcement Schedule R (pigeon pecks floor) –––––––> SR (food) Pecking the floor increases in frequency Adventitious Reinforcement: FT 20” Reinforcement Schedule R (pigeon pecks floor) –––––––> SR (food) Pecking the floor increases in frequency even though there is not a causal/contingent relationship between the response and the reinforcer (From pigeon’s perspective - doesn’t matter. Take home point: if a consequence is a reinforcer it will increase the frequency of the response even though it is not contingent - again, because operant conditioning is automatic. The organism doesn’t have 21 to know or be aware of the relationship between R and Sr)
SO 3 C: Analogous contingencies Contingent reinforcement: (assume FR 1 for pigeon) R (pigeon pecks floor) ––––––> SR (food) R (researcher writes grant) –––––––> Sr (grant money) Adventitious Reinforcement: (assume FT for pigeon) R (pigeon pecks floor) ––––––> SR (food) R (researcher writes grant) –––––––> Sr (inheritance) Given that pecking the floor increases in both of the above, if we are dealing with direct acting contingencies, so too should grant writing. But it doesn’t in the second situation - hence we cannot be dealing with the basic principles of behavior. (Automaticity, not delay is mn point) 22
SO 4: Direct and indirect effects n n Michael then provides examples from OBM where procedures are likely to influence behavior, but consequences are too remote to be “simple” reinforcement/punishment Makes the point that the indirect effects are the ones we are interested in - the direct effects are not very interesting n Monetary bonus for sales reps for meeting quota, and checks are placed in the mailboxes on Friday afternoon n Indirect effect, increase sales n Direct effect, look in mailbox more frequently on Friday afternoons, or walks more quickly to the mailbox on Friday afternoon (picked on OBM because of me!) 23
SO 4: More examples: Direct and indirect effects n A wrestler who is too heavy and needs to make weight for a meet eats small meals one day. The next morning he steps on the scales and sees he has lost weight. n n n Direct effect? Indirect effect? Weekly lottery for attendance. When workers arrive on time to work, they receive a “lottery ticket” that is placed in a hat and the drawing is held at the end of the week. n n Direct effects? Indirect effect? 24
SO 5: 3 clues that an effect is indirect n n Delay - if the consequence is delayed by more than 60 seconds Preconsequence increase in behavior n n Tell workers in advance that we are going to implement a feedback program and they increase performance before getting the feedback Ask a worker to stay late and finish a project and that you will take the worker to lunch the next day if he/she stays - and the worker stays late. 25
SO 5: 3 clues, cont. n Large change in behavior as a result of a single delivery of a consequence: direct effects tend to occur gradually n n Praise a worker for mopping up oil spills on the floor, and the worker then does it from then on Criticize a worker for smoking near flammable material - he never does it again Exception is that we do know that intense punishment can immediately suppress behavior, but is does have to be intense. (I am not going to talk about SO 6 - straightforward) 26
SO 7: Three reasons why are we successful even if we talk about indirect effects as direct effects n Some OBM interventions do involve direct acting contingencies n n Praise that immediately follows behavior On-line feedback systems/measurement in mfg (1 st and 3 rd are quite straightforward - students have trouble with the second. ) 27
SO 7: Second reason why are we successful even if we talk about indirect effects as direct effects n “We don’t get distracted by by inner directedness, rather we look to the environment for causal variables and manipulate those variables. ” n n Examples - related to SO 7 B In BBS, we don’t try to change “attitudes” by posting signs about how important safety is. Consequate performance. n n n Problem in past re “posting signs”: We may post signs, but not to change attitudes and we don’t rely on them. We don’t try to change “satisfaction” in order to increase productivity We don’t try to change a person’s self-esteem to change “motivation” to increase productivity (set goals, provide feedback Posting signs is a very common practice: safety and quality – to change attitudes, only as prompts - antecedents. ) 28
SO 7: Third reason why are we successful even if we talk about indirect effects as direct effects n Our methdology and empiricism: may be the most important reason n Objective measurement of performance/behavior Record/measure behavior over time as it occurs in the workplace As a result, we constantly assess whether our interventions are actually working n Daniels, “If you can’t measure it, you can’t manage it. ” n 50% of experimental articles in JAP used self-report measures to assess the effects of their interventions - we know how unreliable verbal reports are 29
Peterson article: FEEDBACK 30
Intro to Peterson: Feedback n Both Peterson and Balcazar et al. make the same major points: n n n Feedback is NOT a principle of behavior Yet we often talk about it as if it were Peterson n The term “feedback” has become professional slang 31
SO 8: Why is it inappropriate to talk in the abstract about what function fbk serves? n n n When people address the behavioral function of feedback generally or in the “abstract, ” they usually say that it works because it is an SD or Sr, which Peterson says in incorrect Feedback, information about past performance can potentially serve any number of behavioral functions depending upon the situation It is first and foremost a physical stimulus and only that. Thus, it can have any or all of the possible effects of any stimulus in a particular situation n MO, conditioned reinforcer, conditioned punisher, SD, CS 32
SO 9: Why can’t feedback be either an Sr or SD in most settings? n n Peterson also states that feedback cannot be either an Sr or an SD in most settings He gives two reasons for each, but deals with them in a different order than I am going to n n First, why can’t feedback be a reinforcer? Second, why can’t feedback be an SD? 33
SO 9: First reason why fdbk typically cannot be a reinforcer n A reinforcer must immediately follow behavior. In most applied settings, feedback is too delayed. R ––––––> Feedback Too much delay More than 60 sec (straightforward, the next one often causes problems) 34
SO 9: Second reason why fdbk typically cannot be a reinforcer n A reinforcer must be contingent upon performance (adventitious Sr aside). Feedback is not usually contingent upon performance. Reinforcement R (lever press) –––> SR (food) No R (no lever press) –––> No SR Feedback R (decrease electric use) –––> Feedback No R (do not decrease use) –––> Feedback In most situations, you get feedback whether or not you emit the appropriate target response, hence the feedback is not contingent upon performance and cannot be reinforcement (contingent? If-then relationship; but I’ll come back to this in SO 9, to see whether this analysis is correct) 35
SO 9: First reason why fdbk typically cannot be an SD n An SD must evoke a behavior within 60 sec. Feedback typically does not. n n But first, this may be a new concept for you Confusion about Malott’s definition, which is the definition many of you have learned n An SD is a stimulus in the presence of which a response is reinforced or punished n Although not stated because of his study cards, I assure you Malott agrees with the above n Again, it is the molecular position 36
SO 9: Finally! The first reason feedback typically cannot be an SD SD SD: R No more than 60 sec Feedback: R More than 60 sec Be careful! It is not the delay between the response and consequence that is relevant here (as it was for the argument about why feedback usually cannot be a reinforcer). Rather, it is the delay between feedback as an antecedent stimulus and the response! Not R –––> Feedback, but Feedback –––> R Delay 37
SO 9: Second reason feedback typically cannot be an SD n For an SD, the response must be reinforced in the presence of that stimulus but not in its absence. With feedback there isn’t usually an S∆. SD SD (light): R (lever press) –––> SR (food) S∆ (no light): (lever press) –––> No SR (no food) Feedback: R (use less electricity) –––> Sr (lower bill) No feedback: R (use less electricity) –––> Sr (lower bill) (for exam, not sufficient just to say, no S∆, explain; ) 38
SO 10: Is Peterson right in saying feedback is not contingent on performance? Feedback is not contingent on performance: R (decrease electric use) –––> Feedback No R (do not decrease use) –––> Feedback But is it? Feedback is not a unitary stimulus. Doesn’t feedback differ depending upon whether performance is good or bad? R (decrease electric use) ––> Feedback No R (no decrease/increase) ––> Feedback Are those two feedback graphs the same stimulus? So, isn’t feedback contingent upon performance after all? (one of the reasons why feedback can’t be a reinforcer; click, then click again, feeedback diagrams: lecture) 39
SO 11: Dickinson’s additional reason why feedback typically can’t be an SD or Sr n n n Peterson’s reasons focus on the feedback itself However, Dickinson’s additional reason is: What we typically reward (reinforce? ) in OBM are not behaviors, they are accomplishments. We reward the number of parts produced, the total amount of sales, etc. You cannot reinforce “accomplishments”; you can only reinforce behaviors. Not for the exam, but this is one of the reasons why Gilbert’s Human Competence was so controversial when it was published and why many behavior analysts at the time stated that “OBMers” were not, in fact, behavior analysts. 40
Balcazar et al. article FEEDBACK 41
SO 14: Cautionary note about conclusions from review studies n Conclusions are based on the structural analyses of procedures that varied along many different dimensions n n n In Alvero et al. only 30% of the studies used feedback alone Van. Stelle et al. found that in the past decade, different types of feedback were used in all of the feedback studies Only systematic experimental comparisons can ultimately determine whether feedback procedures affect performance differently (skipping 12&13’ not one isolated the effects of a particular type of fdbck or typically from other components; ) 42
Cautionary note about conclusions from review studies, cont. but NFE n In the Alvero et al. review, there were small Ns in many of the categories (you can only review the studies that have been published) n n For example, 4 or fewer feedback applications in 9 of the 12 content categories (the type of information provided on the feedback display) The results of three empirical investigations have conflicted with the conclusions of the reviews n n n So et al. (2013): Daily vs. weekly feedback Goltz et al. (1989): Group vs. Individual + group Van. Stelle (2012): Graphic individual, graphic Individual + group average, graphic display of the performance of all performers (Daily as good as weekly; Group as good or better than individual – 2 experimental studies differed; So again, my main point is just to be cautious with conclusions from these type of review studies) 43
SO 16: Why is it that if reinforcement already exists, fdb might improve performance further? n Feedback may have discriminative control and evoke higher levels of performance due to generalization of past reinforcement contingencies (discriminative control refers to feedback as an SD – an antecedent variable – probably verbally mediated in most cases) That is, in the past, when feedback has been introduced (as an antecedent), you have been reinforced for performing well and criticized/punished for performing poorly. Thus, in the current situation, as long as performance isn’t maxed out, it may evoke higher levels of performance as an antecedent stimulus. n Once the worker is performing better due to the feedback, the higher performance results in more rewards (due to the already existing differential reinforcement system), and the greater rewards then sustain the higher performance over time (skipping 15 straightforward; already a differential reinforcement system in place but no fdb If you add feedback why might performance increase further - assuming, of course, it hasn’t maxed out: two parts!) 44
SO 16: Example n n Assume that workers are being paid a per piece incentive for each widget they assemble. The more widgets they produce, the more money they make. But, they are not getting feedback Now, weekly feedback is introduced n n Feedback increases the number of widgets produced because of generalization of past contingencies (stimulus generalization, SD) They earn more money now because they are producing more widgets than before, and the extra earnings now sustain their higher levels of performance (both points are important for the analysis!) 45
SO 16: What evidence do we have that feedback may enhance the effectiveness of extant rewards? n Einarsson, Y. (2016): Lab study n n n n Participants: 80 undergraduates, all received piece-rate pay DV: number of tasks completed (data entry task) IV: no feedback, graphic individual feedback, and graphic social comparison feedback Experimental design: Between group design, 5 45 -min sessions Results: Both graphic individual and graphic social comparison feedback significantly increased performance in comparison to the no feedback group Comment: type of feedback, graphic may have affected these results because in the past, objective written feedback did not enhance incented performance; more research is required Stongest applied data are from a Union National Bank case study - never published (a few studies that have directly examined this, but no convincing data due to methodological issues or the type of feedback used until…not published yet; not even submitted yet ; to my knowledge, The first study to show that feedback can/does enhance incented performance) 46
Union National Bank, Little Rock, AR n n n When: early to mid 1980 s Who: Abernathy, Mc. Nally, Mc. Adams, & Dierks Job: Proof operator at bank n n DV: number of items entered per machine hour n n Get checks and deposit slips from all bank branches and, using a proofing machine, put the numbers on the bottom so they can be automatically entered into the bank’s computer Industry standard: 1, 000 items per hour Phases n n Baseline Weekly graphed feedback 47
UNB cont. n Phases, cont. n Incentive 1 n n Incentive 2 n n One piece rate: 1500 items per machine hour Higher piece rate: 2000 items per machine hour Higher yet: 2500 items per machine hour Above plus higher rate: 3500 items per machine hour No feedback, but incentives (new supervisor) Reinstate feedback with Incentive 2 conditions Last three phases represent a very nice reversal, incentives with and without feedback 48
Union National Bank: Proof Operators Fdbk 1, 800 Baseline 1, 065 ~Industry standard Inct 1 2, 700 Inct 2 3, 500 Incentive w/o fdbk Fdbk restored Why reversal? Notice: Baseline in weeks Remaining in months! Almost 7 years of data! (note that it took four months before performance declined w/o feedback - may not see it in shorter studies) 49
SO 17: Effectiveness of feedback alone and with tangible rewards, Balcazar et al. n n n % of articles in which performance consistently improved n Feedback alone: ~30% n Feedback with tangible rewards: ~90% Take home point: Feedback works because it is correlated with a differential reward system But what about when it works alone? n n Feedback may be linked to a differential reward system even though that reward system was not explicitly designed by the researcher/practitioner Performance may have been measured for too short of time to see a possible decrease over time - feedback would be expected to increase performance temporarily when first introduced 50
SO 18: Why feedback may be more effective when provided by supervisor n n 18 A: First reason Feedback on how well workers are doing may prompt supervisors to provide differential consequences for good and bad performance when supervisors hadn’t done that before So… 51
SO 18 A, cont. For the supervisor: Antecedent: R ––––––��> Sr Good feedback Praises worker “Thanks!” about worker’s perf More good work Antecedent: R ––––––��> Sr Bad feedback Criticizes worker “I’ll do better” about worker’s perf Prompts worker Better work Thus, feedback may change the consequences that the supervisor provides to the workers. 52
SO 18 B: from lecture SO 18 B: Second reason (not in SOs) n The supervisor has more control over the employee’s rewards than others, and therefore the feedback is more strongly correlated/paired with (a) more rewards and (b) higher value rewards when provided by the supervisor, and thus will function as a more effective antecedent and consequence. 53
SO 19: Balcazar et al. ’s fundamental conclusion n If no system of functional, differential consequences exist, there is probably no point in establishing a feedback system. Effort would be better spent developing procedures (contingent rewards) for wanted behaviors n In other words, you should not develop and implement a feedback system until and unless you develop and implement a system of functional, differential rewards! (or determine one already exists) n This has become a problem in the field n 37% of articles reviewed by Balcazar et al. used feedback alone (review covered 1974 -1984, approximately) n 29% of articles reviewed by Alvero et al. used feedback alone (review covered 1985 -1998) (This is straightforward, yet students seem to have trouble with it) 54
SO 20: If feedback is established independently of functional, differential rewards, what type appears to be the most effective? n Feedback that is: n Graphic n n Confirmed by Wilk & Redmon (1998) Confirmed by Austin, Weatherly, & Gravina (2005) Provided at least once a week Combined with tangible rewards n This is interesting - basically the authors are saying you need to implement a reward system along with the feedback, so I am not at all sure it is a logical answer to the question that was asked, but you should include it in your answer. 55
SOs 21 -23: Johnson (2013) Intro n n n Johnson conducted a study to determine the relative effects of objective feedback, evaluative feedback, and objective combined with evaluative feedback Objective feedback: information about your performance absent any type of evaluation Evaluative feedback: information/stimuli about how well you are performing absent specific data/numbers n Praise/criticism from supervisor or someone else (you are doing great, super job, not so good) (important study that helps explain when/why feedback will be effective; could well help explain some of the differences of feedback studies – will tie into two studies next unit – results were Suggestive, but not as definitive) 56
Background/rationale of study n Johnson, Huitema, & Dickinson (2008) n 2 X 2 between group factorial design: 4 groups n n n Hourly pay without feedback Hourly pay with feedback Incentive pay without feedback Incentive pay with feedback Somewhat surprisingly, although incentives increased performance in comparison to hourly pay (proving the incentives were functional rewards); feedback did not improve performance when Ps were paid hourly or monetary incentives (basically, that’s what we did for Dr. Johnson’s thesis) 57
Background/rationale of study n What could account for the fact that feedback did not have any effect? n We decided it probably was due to the fact that we used objective feedback, without any evaluation n n Just reported the number of accurately completed forms using a data entry task Most feedback procedures, at least those used in OBM involve both an objective and evaluative component n Numbers plus praise from supervisor; numbers plus average group data; numbers plus goals, etc. (So, Johnson’s study specifically compared the relative effectiveness of objective feedback, evaluative feedback, and feedback that had both components ) 58
Methodology: 4 Groups (no pay) n n No feedback Objective feedback n n Data entry task: how many records/checks were completed accurately Evaluative feedback n Evaluative statements without objective feedback based on whether their performance was excellent, good, average, or poor n n Really impressive, wow; impressive, one of the better performers; normal, typical, standard; under standard, below what the average person does, somewhat low Combined objective and evaluative feedback (summarize this – read the details on your own) 59
SO 22: Results n n n Objective feedback: 17% above no fdbk Evaluative feedback: 17% above no fdbk Combined: 30% above no feedback Statistical significance (NFE): Combined > objective, evaluative, no fdbk Objective & Evaluative Alone > no fdbk (personally, I think it’s particularly interesting that evaluative feedback was no better than objective – an applied study you will be reading next week, Gaetani & Johnson found that to be the case as well, but due to the design of the applied study, results weren’t as definitive. ) 60
SO 23: Results differ from Johnson et al. (2008) n n In this study, objective feedback increased performance by 17% In Johnson et al. (2008), objective feedback had no effect 61
SO 23 A, B, &C: Results differ from Johnson et al. (2008) n Possible reason for the difference? n n n In this study, objective feedback was provided by a person In Johnson et al. (2008), objective feedback was provided by a computer Why may this be important? n Objective feedback may not be as effective when delivered by electronic devices (computers, smart phones, i. Pads, etc. ) as when delivered by a person (supervisor) 62
SO 23 A, B, &C: Results differ from Johnson et al. (2008) n Why might objective feedback delivered by a person be more effective? n Evaluation is implied when a person provides the feedback but such evaluation is not present with an electronic device When we receive feedback on our performance from another individual, it is typically the case that person is evaluating our performance, even though the person might not say that explicitly. Not so with electronic devices. 63
Human-delivered vs. electronicallydelivered feedback n Poster at ABAI May, 2015 n n n Chae, S. , Moon, K. , Lee, K. , & Oah, S. , Chung-Ang University Simulation study: Human-delivered feedback increased performance more than feedback delivered by email Initial presentation at ABAI, May 2016 n n Garrett Warrilow thesis: (Johnson’s lab, ongoing) Simulation: All types of feedback improved performance in comparison to a no feedback group, but face-to-face human feedback did not increase performance significantly more than computer-delivered feedback or cell phone text (Warrilow, face-to-face did appear to improve performance a bit more; jury is still out on this 64 one)
Another Interesting potential study (NFE) n 2 x 2 between group factorial design n n Human-delivered objective feedback Human-delivered evaluative feedback Electronically-delivered objective feedback Electronically-delivered evaluative feedback But a bit dangerous because you may not get any Differences and you can’t confirm a null hypothesis: Should probably have a no feedback group as well. 65
Yet another interesting potential study: graphic versus written feedback (NFE) n 2 x 2 group between-group design n n Human-delivered objective graphic feedback Human-delivered objective written feedback Computer-delivered objective graphic feedback Computer-delivered objective written feedback (then, of course, repeat with evaluative component) 66
If time: Current trends, discussion of social comparison feedback (NFE) n Some applied studies have used a type of social comparison feedback in which the performance of all employees is displayed publicly, usually by name n n This would be similar to my putting your names on the grade sheet. A recent trend in business and industry is gamification: there are many start-up software companies offering gamification to companies In gamification, employees have on-line access to the following performance data: (a) their own, (b) their peers, identified by name, (c) their department/unit, and (d) department/unit data for different stores (assuming multiple stores/locations) 67
If time: Current trends, discussion of social comparison feedback n n n Sears is an example of this with their “Game. On” Presentations by CLG at ABAI on visual management: all the rage in engineering process improvement programs/systems The performance data re social comparison feedback with named peers: n n OBM studies have shown that in each study in which peer to peer feedback is provided, performance has improved substantially Van. Stelle’s dissertation showed that graphic feedback with the performance data of all peers, with names, increased performance more than graphic feedback that displayed the individual’s performance and group’s average performance, which increased performance more than graphic feedback that displayed only the individual’s performance 68
If time: Current trends, discussion of social comparison feedback n The performance data re social comparison feedback with named peers: n n n OBM studies have shown that in each study in which peer to peer feedback is provided, performance has improved substantially Van. Stelle’s dissertation (2012) showed that graphic feedback with the performance data of all peers, with names, increased performance more than graphic feedback that displayed the individual’s performance and group’s average performance, which increased performance more than graphic feedback that displayed only the individual’s performance (hourly pay) Einarsson’s thesis (2016) showed that graphic feedback with performance data of all peers, with names, increased performance more than graphic individual feedback (incentive pay) 69
If time: Current trends, discussion of social comparison feedback n Einarsson’s dissertation (2018) examined four groups: n n n SCF with hourly pay SCF with incentive pay Tiered goals (5 goals) with hourly pay Tiered goals with incentive pay Results indicted that (a) incentive pay increased performance in comparison to hourly pay, and (b) SCF increased performance in comparison to tiered goals The four groups ranked as follows in terms of performance from highest to lowest: n n n SCF with incentive pay SCF with hourly pay = tiered goals with hourly pay Tiered goals with hourly pay 70
If time: Current trends, discussion of social comparison feedback n The social validity data re social comparison feedback with named peers: n n n In the few OBM field studies that assessed satisfaction, employees did not find peer-to-peer social comparison feedback aversive In Van. Stelle (2012) and Einarsson (2016, 2018), participants in this group did not rate the feedback procedure as any more aversive than the other types of feedback procedures In a company that has adopted gamification, a behavior analyst who I respect very much, told me that the employees found it a lot of fun and engaged in a lot of fun “trash talk” with each other n The system is a criterion-referenced, not normative, so just like in my classes everyone can receive an A, which can promote “friendly competition” rather than “vicious competition” 71
If time: Current trends, discussion of social comparison feedback n Alternatively, in a study by Steigleder et al. (1978), when given the option of opting out of competition (dyads with performance feedback with respect to who has the highest score – no other contingencies on performance): n n Participants who were led to believe that they were in the 90 th percentile of performers opted out Participants who were led to believe that they were in the 20 th percentile opted out more quickly and acquired the opt out response more quickly Steigleder, M. K. , Weiss, R. F. , Cramer, R. E. , & Feinberg, R. A. (1978). Motivating And reinforcing functions of competitive behavior. Journal of Personality and Social Psychology, 36(11), 1291 -1301. 72
If time: Current trends, discussion of social comparison feedback n What do you think? n n n Is it aversive to be compared to peers by name? Have times changed with social networking so that individuals of your generation don’t find this type of competition and social comparison aversive? Were the “artificial” contingencies/constraints of the two lab studies (Van. Stelle and Einarsson) responsible for participants reporting that the SC feedback was OK n n n Participants did not know one another Rewards were not competitive based on rankings No evaluative comments were made in either study by the RAs 73
If time: Current trends, discussion of social comparison feedback n Einarsson, Schipper, & Dickinson: ABAI 2018 discussion of talk n n Ludwig defended SCF as not aversive, no big deal Johnson: it may depend on a lot of factors, but one should not assume it is aversive Dickinson: agreed with Johnson that there are probably exceptions depending upon the situation and history of individuals (athletes and sales representatives – OK; lone workers probably OK to compare against each other) but argued that in the typical work environment it is likely to cause “tension” between and among workers Interestingly, at ABAI anyway, the women seemed more opposed to it than the males. 74
Questions? ? 75
Effects of Objective Feedback Study Effective Vs. No FB? Chapanis (1964) No Crowell et. al (1988) Yes Johnson, Dickinson, Huitema (2008) No Johnson (2013) Yes
Effects of Objective Feedback Study Effective Vs. No FB? Modality Chapanis (1964) No Counter Crowell (1988) Yes Posted Charts Johnson, Dickinson, Huitema (2008) No Computer Johnson (2013) Yes Face to Face
- Slides: 77