EXPERIMENTAL DESIGNS FOR UTILITY TESTING OF VISUALIZATION TECHNIQUES

4 PHASE PLAN Phase 1 Literature Review Phase 2 Develop Heuristics Phase 3 Pilot

Phase 1 – Literature Review Identify potential design principles Scientific Evidence Cognition Perception Sociology

Phase 2 Develop Heuristics heuristics/rules of thumbs (empirically supported) to better inform visualisation design

Phase 3 Pilot Study to Fill the Gaps Likely that not all Heuristics are

Phase 4 Applications Evaluation (1) Experimental Design Analysis Limitations Conclusions

Phase 4 Applications Evaluation (2) Experiment Design Appropriate Sample Size vs power and sample

Phase 4 Applications Evaluation (3) Limitations How to measure analyst tinkering with visualization –

Phase 4 Applications Evaluation (4) Conclusions Propose visualisation standards/guidelines/best practices for designers/developers/acquisition teams Internal

Slides: 10

Download presentation

EXPERIMENTAL DESIGNS FOR UTILITY TESTING OF VISUALIZATION TECHNIQUES • • • Cristin Hall Chris Horeczy Annette Kaster Michael Kleiber Marcus Lem Sonya Mc. Mullen

4 PHASE PLAN Phase 1 Literature Review Phase 2 Develop Heuristics Phase 3 Pilot Study Phase 4 Field Test Applications

Phase 1 – Literature Review Identify potential design principles Scientific Evidence Cognition Perception Sociology and Anthropology? Other Disciplines Group members with access to literature in other Languages? Funds/grad students

Phase 2 Develop Heuristics heuristics/rules of thumbs (empirically supported) to better inform visualisation design Example: Capacity to discriminate between colours/shades, shapes, size differences… Develop a rating scale of effectiveness to use as metric in experimentation Identify Gaps

Phase 3 Pilot Study to Fill the Gaps Likely that not all Heuristics are appropriately quantified and supported by solid evidence. Studies/experimentation to complete the dataset taken forward to the Test of Applications Iterative Process that may result in the modification of the Heuristics

Phase 4 Applications Evaluation (1) Experimental Design Analysis Limitations Conclusions

Phase 4 Applications Evaluation (2) Experiment Design Appropriate Sample Size vs power and sample size reported in lit What are the tasks? Who is our population(s) of interest? Analysis Field Testing Iterative process

Phase 4 Applications Evaluation (3) Limitations How to measure analyst tinkering with visualization – creating wrong impression May need spatial statistics to measure true properties of data vs analyst‘s manipulation of visualization or intrinsic properties of the visualization tool

Phase 4 Applications Evaluation (4) Conclusions Propose visualisation standards/guidelines/best practices for designers/developers/acquisition teams Internal and External Validity Logic, statistical power, etc Real world scenarios: social psychology influences – stress, group dynamics, authority. . . NETWORK VISUALIZATION STANDARDS/GUIDELINES 1. Best practice 2. Benign/inert practice 3. Prohibitive practices

Discussion? !