Command Visualisation NATO Workshop on Visualisation of Massive

Command Visualisation NATO Workshop on Visualisation of Massive Military Multimedia Datasets, DREV Justin G. Hollands Human-Computer Interaction Group, DCIEM

OVERVIEW • Problem and Potential Solution • Visualisation Philosophy--Human Centered – Consistent with NATO IST-05 Model – Empirical Basis • Need for Multiple Views • Command Visualisation Testbed Platform – Five Research Domains Defence and Civil Institute of Environmental Medicine (DCIEM)

INFORMATION VISUALISATION • Problem: – Military systems do not allow users to understand what their various data and information sources mean • Users cannot use engines to extract information from data • Important information may be ignored due to deficient extraction techniques • Strategic and tactical actions, simulation and training, are less efficient than they should be Defence and Civil Institute of Environmental Medicine (DCIEM)

POTENTIAL SOLUTION – Provide information visualisation tools to aid in the appropriate comprehension of information – Military benefits: · Improved information accessibility, filtration, extraction, and understanding. · Support for joint activities through better interoperation and communication - Challenger Example · Erosion of O-Rings as detailed by Thiokol Defence and Civil Institute of Environmental Medicine (DCIEM)

Defence and Civil Institute of Environmental Medicine (DCIEM)

Source: E. Tufte (1997). Visual Explanations Conclusion: The way in which information is presented affects the way it is interpreted. Defence and Civil Institute of Environmental Medicine (DCIEM)

VISUALISATION TOOLS for C 2 • CPo. F (DARPA) – Large-scale – Multimedia: Maps, Charts, Graphs, Pictures, CNN – 3 D Electronic Sand-Table – Multi-display – $$$ Defence and Civil Institute of Environmental Medicine (DCIEM)

CPo. F Graphical representation of cross-correlation of radio activity provides insight Same data in tabular form as good as useless Defence and Civil Institute of Environmental Medicine (DCIEM)

MULTIPLE VIEW AND TASK DEPENDENCY • Data have to be displayed just the “right way” – How can we increase P(Hit)? – Make available a wide variety of views • In line with graphical perception literature – Performance with a given graphical format is task-dependent • Two design implications are evident: – 1) Successful C 2 system must display data in multiple formats; – 2) System must make transition from one format to the next as straightforward as possible Defence and Civil Institute of Environmental Medicine (DCIEM)

VISUALISATION PHILOSOPHY • “Information Visualisation is the human's capacity to utilize effectively the output from the computer to understand the data. ” – NATO IST-05 • Specific Research Goal: – Produce a testbed platform based on the principles and recommendations contained in the HAT report – Platform will be designed based on performance data from human observers NATO IST-05 Reference Model Defence and Civil Institute of Environmental Medicine (DCIEM)

COMMAND VISUALISATION TESTBED • TIF (DND Technology Investment Fund) Proposal – Goal: • To create an effective command visualisation platform based on empirical performance data collected from human observers, rather than upon a programmer’s “best guess” – Approach: • 5 Stages: 1) Identification/Selection of Operational Tasks 2) Testbed Platform Setup 3) Experiments to Address Specific Research Issues (Five Domains) 4) Iterative Platform Design and Testing 5) Delivery of Results Defence and Civil Institute of Environmental Medicine (DCIEM)

Selection of Operational Tasks – Cognitive Task Analysis (CTA) will be performed on task group and/or task force commanders – Anticipated that the visualisation system would be particularly useful for: • • • battle planning and simulation; establishing the threat axis; assessing damage, weapons and supplies; checking weapon ranges correlation of sensor data across time (data fusion) – CTA will determine what kind of information should be displayed to commander at what time (in what context) Defence and Civil Institute of Environmental Medicine (DCIEM)

What are we Measuring? – Error (1 - Accuracy) • Error Magnitude, Number of Errors • Signal Detection Measures--sensitivity and bias – Response Time – Subjective Measures • Preferences • Situation Awareness, Workload – In tasks that are as similar to real operational tasks as possible (CTA) using military personnel as participants Defence and Civil Institute of Environmental Medicine (DCIEM)

FIVE RESEARCH DOMAINS – Frame of reference and visual momentum • Easing the transition between egocentric and exocentric views – Perceptual bias and reference points • Utility of reference points for reducing bias – Modeling mental operations • Minimizing operations necessary for task – Preattentive processing • Utility of “pop-out” in visual search – Mapping data to perceptual continua • What’s good for what Defence and Civil Institute of Environmental Medicine (DCIEM)

FRAME OF REFERENCE AND VISUAL MOMENTUM – Easing the transition between general and specific views • Egocentric and exocentric views • Hierarchical: generalization vs. specialization • Drill up vs. drill down • Visual momentum techniques: – Continuously available global view – Gradual transitions – Brushing Defence and Civil Institute of Environmental Medicine (DCIEM)

Roth et al. (1997) Visage. Defence and Civil Institute of Environmental Medicine (DCIEM)

PERCEPTUAL BIAS – Utility of reference points for reducing bias, and thereby error Source: Hollands & Dyre (2000) Defence and Civil Institute of Environmental Medicine (DCIEM)

MODELING MENTAL OPERATIONS • Minimizing operations necessary for task • Two factors affect graph reading performance: – 1. Number of operations necessary given a particular task-graph combination – 2. Effectiveness of the perceptual features used as input for the operations • Model yet to be extended to more realistic visualisation situation B A Ratio estimation(height[A], height[A+B]); A B Sum = Summation(height[A], height[B]); Ratio estimation (height[A], Sum); Defence and Civil Institute of Environmental Medicine (DCIEM)

Defence and Civil Institute of Environmental Medicine (DCIEM)

PREATTENTIVE PROCESSING • Utility of “pop-out” in visual search • Visual properties that are processed preattentively can be used to highlight important image characteristics – colour, shape, orientation, length, size, curvature, depth • Preattentive dimensions can be used to segment, create textured areas Defence and Civil Institute of Environmental Medicine (DCIEM)

MAPPING DATA TO PERCEPTUAL CONTINUA • Bertin--appropriate use of spatial dimensions versus, colour, shading, texture, orientation etc. • Conceptual dimension (e. g. , number of helicopters, state of alert) may be qualitative, ordered, quantitative • Appropriate in the sense of mapping conceptual dimension to perceptual dimension • Bertin’s ranking will be empirically assessed Defence and Civil Institute of Environmental Medicine (DCIEM)

SUMMARY • Problem and Potential Solution • Need for Multiple Views • Visualisation Philosophy--Human Centered – Consistent with NATO IST-05 Model – Empirical Basis • Command Visualisation Platform – Five Research Domains Defence and Civil Institute of Environmental Medicine (DCIEM)

Justin G. Hollands DCIEM 1133 Sheppard Ave. W. , P. O. Box 2000 Toronto, Ontario M 3 M 3 B 9 Canada (416) 635 -2000 x 3082 justin. hollands@dciem. dnd. ca QUESTIONS/COMMENTS? Defence and Civil Institute of Environmental Medicine (DCIEM)