Collective Mistrust of Alarms James P Bliss Ph

Collective Mistrust of Alarms James P. Bliss, Ph. D. Susan Sidone Holly Mason Old Dominion University

Collective Mistrust of Alarms - A Few Thoughts Before We Begin. . . l Novelty of this project – Not Automation per se: Alarms inform, do not control – Alarms convey system state to operator – May help to “push the envelope” of etiquette research “Mistrust/distrust” may be different with alarms l Simplistic paradigm deceptive - multiple trust components involved l Information accessibility, technology improvements means operators expect more from alarm systems l Operator mental models very important l Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms Introduction l Investigations of Individual Alarm Mistrust – – Aviation (Bliss, 1997) Mining (Mallett et al. , 1992) Ship Handling (Kerstholt et al. , 1996) Driving (Nohre et al. , 1998). General Findings: People Reacted Slower, Less Frequently, Less Appropriately to Unreliable Alarms. l No Studies of the Impact of Marginally Reliable Alarm Signals on Teams of Operators l Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms Introduction l Teamed Alarm Reactions – – l Aviation Critical Care Units Nuclear Power Plants Air Traffic Control Centers To Effectively React to Alarms, Team Members Must – – Share Information Troubleshoot Systems Determine Relative Signal Priority Allocate and Coordinate Reaction Responsibility Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms Introduction Team Member Interdependence Often Varies with the Task and the Environment (Thompson, 1967). l Dependent Teams React to Alarms More Appropriately, More Slowly (Bliss et al. , 2002) l Implications of Teamed Alarm Reactions for Human-Automation Etiquette l – Human-Alarm Trust – Human-Human(Alarm) Trust Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms Introduction l Goals of the Current Research: – Investigate Reactions of Dependent and Independent Teams to Alarm Signals of Various Reliability Levels. – Determine How Collateral Alarm Systems Mediate Alarm Mistrust. l Approach: – Dual-Task Approach (Damos, 1991). – Independent Variables Manipulated Using a 2 X 3 Mixed Design. – Dyads Reacted to Two Separate Alarm Systems. l l Bliss, Sidone, & Mason, 2002 Temperature Alarm Reliability = 80% true alarms). Pressure Alarm Reliability Fluctuated (40%, 60% or 80%).

Collective Mistrust of Alarms Experimental Design l Interdependence Manipulated Between Two Groups – Dependent Team Members Required Interaction to React Appropriately; Independent Team Members Did Not. l Pressure Alarm Reliability Manipulated Within Groups – Pressure Alarms Were 40%, 60%, and 80% Reliable During Sequential Task Sessions. Temperature Alarms Were 80% Reliable. l Dependent Measures – Ongoing Task: Gauge Monitoring Accuracy, Tracking Error. – Alarm Task: Reaction Speed, Appropriateness; Response Frequency. Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms Method Participants: 40 student dyads from Old Dominion University (18 -43 yrs) worked for course credit and the chance for a monetary performance bonus. l Primary Task: Multi-Attribute Task (MAT) battery (Comstock & Arnegard, 1992) presented to each member. l – – Dual-Axis Compensatory Tracking Gauge Monitoring Resource Management Participants Performed the MAT Back-to-Back Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms – MAT Battery Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms Method l Auditory and Visual Alarms: Digitized Fire Bell From a Boeing 757/767 simulator. – Alarms Occurred 90° to the Side of the Primary Task. – Alarm Procedure: Determine Whether Corresponding MAT Gauges Are Out of Tolerance. If so, Reset Gauges and respond to the alarm. If Not, Cancel the alarm and resume the primary task. l Interdependent team members had to communicate because they shared the out-of-tolerance gauges. Independent team members monitored all gauges. Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms Procedure Informed Consent Form l Experimental Instructions - Dependent team members told to communicate. l MAT Task Practice l – Individual 120 -second sessions (Each Subtask) – Combined 200 -second session (MAT and Alarms) l Three experimental sessions – Ten alarms presented during each session. – Pressure alarm reliability randomly counterbalanced – Participants Knew Alarm System Reliability Before They Began l Debriefing, dismissal. Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms – Results (Response Frequency) Response Frequency to Temp Alarms • No Interaction (p>. 05) • Linear main effect, F(1, 38)=129. 600, p<. 001. • Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms – Results (Reaction Appropriateness) • Significant Interaction, F(2, 76)=10. 193, p<. 001. • Main Effect for Interdependence, F(1, 38)= 4. 000, p=. 05. • Quadratic Main Effect for Reliability, F(1, 38)=19, 563, p<. 001. Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms – Results (Reaction Time) No significant interaction • No Interdependence main effect • Linear Reliability Main Effect, F(1, 38)=8. 181, p=. 007. • NOTE: No Primary Task Differences • Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms Discussion l Results Similar to Past Efforts, Except for Lack of Primary Task Differences. – Multiple alarm systems may have led participants to rethink their trust levels, a reflection of workload (Bliss & Dunn, 2000). l Alarm designers should consider the effects of multiple alarm systems on operator behavior. – Recognize that complex reaction responsibilities may cause cognitive load as team members adjust trust levels. Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms Discussion l Dimensions of Trust in This Experiment – Basic Trust of the Experimenter (Human-Human) – Trust of the Primary (MAT) Task (Human-Computer) – Trust of the Alarm Task (Human-Computer) Manipulated by the Experimenter – Trust of Teammates (Human-Human) l Questionable in this Experiment, Due to documented Unfamiliarity l Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms Discussion l Past reactions to unreliable alarm systems – Fluctuations in physiological responses (Breznitz, 1983) – Degraded performance (Getty et al. , 1995) – Complete Lack of Trust (Bliss, 1993) – Complete Trust – Probability Matching – Participants’ Response Rates Mirror the Perceived Reliability of the Alarm System. – These Patterns Take Time to Appear (Bliss et al. , 1996). – Question: What if Researchers Apply human trust facilitators to human-alarm relationships? WHAT ARE THOSE VARIABLES? Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms Discussion l Documented Ways to Improve Alarm Responsiveness – Maximize alarm reliability (Bliss, 1993) – Advertise high alarm reliability rates (Bliss et al. , 1995) – Add Redundant Sources of Alarm Information (Bliss et al. , 1996) – Augment alarm stimuli and response options (Bliss, 1997). l Etiquette Related Possibilities – Give alarm systems “human” qualities (include verbiage, etc. ) – Make alarm stimuli emotional (Sorkin et al. ’s “likelihood alarm displays”; altering Edworthy’s parameters) – Vary teammate trustworthiness Bliss, Sidone, & Mason, 2002