Too much information Visualization agents that see the

Too much information Visualization agents that see the “diamonds on the dust” Karen Gundy-Burlet (NASA Ames) Johann Schumann (RIACS) Tim Menzies (WVU) 2/2/07

May your wishes never come true Olde world: never enough information to make decisions l New world: analysts overloaded with data by simulations from l Supercomputers – Cheap LINUX clusters – l Situation is going to get worse – More and more network CPU available SETI project l Grid computing l l How to handle all that data?

Cognitive Overload l Consider this 5 -D plot X, y, z – Visual cues for rotational momentum. – Colors used for density – And additional dimensions shown bottom right – l Consider the problem of exploring all these dimensions Jump on in – Move those sliders around – Have you missed anything? – Does your exploration of this space suffer from “greedy search? ” (miss a global optimum? ) –

What happens in higher dimensional space? l E. g. Constellation re-entry simulation – Already in 24 -dimensions l One for every setting to the input specs Produces wide range of behavior l Visualizations need agents l To cut through the maze of interactions – Return the core ideas –

Does clustering help? l Constellation re-entry: 24 dimensions, – l We can cluster them according to domain knowledge – l (24^2 - 24/2 = 276 sets of interactions E. g. distance to target But we don’t know the minimum we need to do to access the different clusters

Does treatment learning help? l Treatments = least set of factors that most change the outcome – E. g. if sick, your doctor can instantly, schedule surgery Or, take your temperature l and ask if other people where you work have the flu. l – l One of these actions leads to more cost-effective solutions. Out of 276 settings, treatment learner It found two that most selected for finding the targets – But it missing the middle region – l treatment learning knows about rectangles, not stripes

So, what to do l l l Different methods offered solutions to part of the problem Clustering : defined the space Treatment learning: focused the analysis Visualization: displayed, suggested new insights (e. g. add a new slanted dimension). Iterate: Cluster focus display, apply new insights – Repeat –

Benefits l l l More data, understood faster More exploration of models Less effort in examining results A structured methodology for inspecting data An audit trial for why certain decisions were made Clustering pointed us to “this”; – treatment learning revealed “that”, – so we explored “the other thing” –

Extensions l After doing the above on NASA data, we see ways to do the above – l In real-time So – – – Not simulate, then study But study why you simulate. Data miners (clustering, treatment learners) can be queried not just for a learned theory l – Make better use of our simulators l l But for hints on what would be the next best sim to run Not just more data, but more focused data Can be done before or after launch. – – – Fly an agent on the rocket Offers back to ground control the most crucial issues to look at. An in-flight V&V agent in a box.

Deliverables l 6 months – Scripts, integrated into XYZ simulation (say, TRICK, or whatever) l l 9 months – l As above, plus dimensionality synthesis tools (e. g. PCA to find that diagonal stripe). 12 months – Tool (version 0. 1): An n-dimensional display with added information from the treatment learning l l Automatic clustering, treatment learning, automatic generation of a small number of graphs showing most important regions Here, there be dragons 24 months – – Tool (version 1. 0). The above tool, augmented l l with a real-time visualization agent that surfs the space the user Pointing out features you may have missed