Busting the Myths About What it Takes to

Busting the Myths About What it Takes to Become a GEOINT Expert so You Can Become One Too DGI 2019 Dr. Todd S. Bacastow, Penn State University In cooperation with Todd M. Bacastow, Radiant Solutions 28 -30 January 2019 5/20/2021 1

Bottom Line Up Front How to become a geospatial intelligence expert in the future: 1. Focus learning on critical human-machine tasks. 2. Teach in the context of how we work. 3. Learn through reflection on doing. 5/20/2021 2

Agenda Part I: Myths About Being a Geospatial Intelligence Expert Analyst Part II: Becoming a Geospatial Intelligence Expert in the Future 5/20/2021 3

Terms of Reference geospatial intelligence: Has no universally accepted definition but is characteristically work that is competitive, multidisciplinary, and completed by humans and machines. human–machine team: A team in which the functions of a human operator (or a group of operators) and a machine are integrated. reductionism: The practice of analyzing and describing a complex phenomenon in simpler or more fundamental terms. expert: The distinguished journeyman, highly regarded by peers, whose judgments are uncommonly accurate and reliable, whose performance shows consummate skill and economy of effort, and who can deal effectively with certain types of rare or “tough” cases. The expert has special skills or knowledge derived from extensive experience with subdomains. 5/20/2021 4

Terms of Reference (Cont) myth: A widely held but false belief or idea. 5/20/2021 5

Myths About Being a Geospatial Intelligence Expert Analyst 5/20/2021 6

Myth #1 Expectation: The expert geospatial intelligence analyst generates multiple options and compares them to pick the best option. Best Practice: Use Analysis of Competing Hypothesis (ACH). Reality: Experienced analysts seldom methodically compare options (Klein, 1989; Hoffman, 1992). 5/20/2021 7

Myth #2 Expectation: The expert geospatial analyst depends on explicit knowledge and trained procedures. Best Practice: Analytical procedures help an analyst to apply domain knowledge. Reality: Experts rely heavily on tacit (informal and uncodified) knowledge and experience (Klein and Hoffman, 1993). 5/20/2021 8

Myth #3 Expectation: Analysis starts with a goal. Best Practice: “Agreed upon goals within the set scope, time, quality and standards. ” Reality: Many analytic projects involve wicked problems with ill-defined goals (Hoffman, 2007). 5/20/2021 9

Myth #4 Expectation: Making sense involves building up from data, to information, to knowledge, then to wisdom. Best Practice: Analysis follows the DIKW pyramid. Reality: Experts use their experience (wisdom and knowledge) to highlight the relevant cues (data and information). (Skriver et al. , 2004; Schraagen et al. , 2008). 5/20/2021 10

Myth #5 Expectation: The expert geospatial intelligence analyst should have no preconceived ideas (biases). Best Practice: Avoid all biases; have no preconceived ideas. Reality: Biases (mental models) are key to detecting contradictions and anomalies, and noticing connections (Klein, 2013). 5/20/2021 11

Summary: Truths About How A Geospatial Intelligence Expert Works 1. 2. 3. 4. Seldom use a methodically approach. Replies on tacit knowledge. Frequently works with ill defined problems, contexts, and goals. Use a mental model to define what counts as data in the first place. 5. Relies on a repertoire if mental models. 5/20/2021 12

So What? • Many of our best practices are severe simplifications of complex cognitive activities. • These best practices might get the apprentice to the journeyman level but, our “reductionist views” only put the learner on a path to becoming an expert. • Being an expert requires extended learning, experience in complex domains, and practice at tough cases. Question: Are these myths influencing how we view and prepare experts in the future human-machine team? 5/20/2021 13

Becoming a Geospatial Intelligence Expert in the Future 5/20/2021 14

5/20/2021 15

5/20/2021 16

http: //www. kasparov. com/garry-kasparov-every-profession-will-feel-the-pressure-of-ai-closing-bell-cnbc-may-1 st-2017/ 5/20/2021 17

“Human strategic guidance combined with the tactical acuity of a computer was overwhelming. ” - Garry Kasparov Rasskin-Gutman, D. (2009; 2012; ). Chess metaphors: Artificial intelligence and the human mind. Cambridge, Mass: MIT Press. UK Ministry of Defence: Development, Concepts, and Doctrine Centre: ‘Human-Machine Teaming’. Joint Concept Note 1/18, 21 May 2018. https: //www. gov. uk/government/publications/human-machine-teaming-jcn 118, p. 40. 5/20/2021 Soft skills ……… 18

“Machines don’t fight wars. People do, and they use their minds. ” - John Boyd “So realising this potential will depend on understanding the relative strengths of humans and machines, and how they best function in combination to outperform an opponent. ” - UK MOD UK Ministry of Defence: Development, Concepts, and Doctrine Centre: ‘Human-Machine Teaming’. Joint Concept Note 1/18, 21 May 2018. https: //www. gov. uk/government/publications/human-machine-teaming-jcn 118, p. 4. 5/20/2021 19

Human and Machine Activity Adapt Predict Iterate Transact Machine-only activity Judge Create Empathize Lead Human-only activity After Daugherty, Figure 1. 1 Daugherty, P. R. , & Wilson, H. J. (2018). Human + machine: Reimagining work in the age of AI. Boston, Massachusetts: Harvard Business Review Press. 5/20/2021 20

Daugherty’s Missing Middle Adapt Predict Iterate Machine-only activity Transact Embody Interact Amplify Sustain Explain Human and machine activities Train Judge Create Empathize Lead Human-only activity After Daugherty, Figure 1. 1 Daugherty, P. R. , & Wilson, H. J. (2018). Human + machine: Reimagining work in the age of AI. Boston, Massachusetts: Harvard Business Review Press. 5/20/2021 21

Reimagining the High-level Competencies of the Future Geospatial Intelligence Expert • “Training” AI systems. • Explaining AI results. • Sustaining the human-machine team by insuring data quality and applying critical thought. • Amplifying anomalies by applying AI to Big Data. • Interacting with an AI system to focus on high value activities. • Embodying machines in human processes to augment the analyst’s work. 5/20/2021 22

3 Ways to Change How We Develop Geospatial Intelligence Expertise 1. Focus on the “missing middle” of human-machine activities. 2. Teach taskwork in the context of trusted human-machine teamwork. 3. Apply experimental learning, in other words, "learning through reflection on doing". 5/20/2021 23

Question: Are myths influencing how we view and prepare experts in the future human-machine team? Yes, the myths are: o Misdirecting our attention from what should be learned. o Obfuscating how to learn as a human-machine team. 5/20/2021 24

References • • • • 5/20/2021 Chapanis, A. (1991), "To Communicate the Human Factors Message, You Have to Know What the Message Is and How to Communicate It, " Human Factors Society Bulletin , Volume 34, Number 11, November 1991, 1 -4. Coquillette, D. R. (1994). Professionalism: The deep theory. North Carolina Law Review, 72(5), 1271. Daugherty, P. R. , & Wilson, H. J. (2018). Human + machine: Reimagining work in the age of AI. Boston, Massachusetts: Harvard Business Review Press. Fletcher, J. D. (2009). The value of expertise and expert performance: A review of evidence from the military. In K. A. Ericson (Ed. ), Development of professional expertise (pp. 449 -469). Cambridge: Cambridge University Press. Gould, Peter; White, Rodney (1993). Mental Maps. New York: Rutledge. Klein, G. , Moon, B. , & Hoffman, R. R. (2006 a). Making sense of sensemaking 1: Alternative perspectives. IEEE Intelligent Systems, 21(4), 70 -73. doi: 10. 1109/MIS. 2006. 75 Klein, G. , J. K. Phillips, and D. A. Peluso (2007) A data-frame theory of sensemaking. Expertise Out of Context: Proceedings of the Sixth International Conference on Naturalistic Decision Making. Klein, G. , Moon, B. , & Hoffman, R. R. (2006 b). Making sense of sensemaking 2: A macrocognitive model. IEEE intelligent systems, 21(5), 88 -92. Hernandez-Serrano, J. and Jonassen, D. H. (2003). The effects of case libraries on problem solving. Journal of Computer Assisted Learning, 19, 103 -114. Hoffman, R. R. , Shadbolt, N. R. , Burton, A. M. , & Klein, G. (1995). Eliciting knowledge from experts: A methodological analysis. Organizational behavior and human decision processes, 62(2), 129 -158. Hoffman, R. , Feltovich, P. , Fiore, S. , Klein, G. , Missildine, W. , Di. Bello, L. , & FLORIDA INSTITUTE FOR HUMAN AND MACHINE COGNITION INC PENSACOLA FL. (2010). Accelerated proficiency and facilitated retention: Recommendations based on an integration of research and findings from a working meeting Hoffman, R. R. , Feltovich, P. J. , Fiore, S. M. , Klein, G. , & Ziebell, D. (2009). Accelerated learning (? ). IEEE Intelligent Systems, 24(2), 18 -22. doi: 10. 1109/MIS. 2009. 21 Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16, 235 -266. 25

• • • 5/20/2021 Hmelo-Silver, C. E. & Barrows, H. S. (2006). Goals and strategies of a problem-based learning facilitator. Interdisciplinary Journal of Problem-based Learning, 1. 21 -39. Hmelo-Silver, C. E. , Duncan, X. & Chinn, X. (2007). Scaffolding and Achievement in Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42, 99– 107. Moore, D. T. , & Center for Strategic Intelligence Research (U. S. ). (2011). Sensemaking: A structure of an intelligence revolution. Washington, DC: Center for Strategic Intelligence Research, National Defense Intelligence College. NGA. (2006). National System for Geospatial Intelligence: Geospatial Intelligence (GEOINT) Basic Doctrine. Office of Geospatial-Intelligence Management. Orr, J. E. (1996). Talking about machines: An ethnography of a modern job. New York: Cornell University Press. Rasskin-Gutman, D. (2009; 2012; ). Chess metaphors: Artificial intelligence and the human mind. Cambridge, Mass: MIT Press. Shadrick, S. B. , & Lussier, J. W. (2009). Training complex cognitive skills: A theme-based approach to the development of battlefield skills. In K. A. Ericson (Ed. ), Development of professional expertise (pp. 286 -311). Cambridge: Cambridge University Press. Sieck, W. R. , Klein, G. , Peluso, D. A. , Smith, J. L. , Harris-Thompson, D. , Gade, P. A. , & Klein Associates Inc Fairborn OH. (2007). FOCUS: A model of sensemaking Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science 12, 257– 285. Sweller, J. (2006). The worked example effect and human cognition. Learning and Instruction 16, 165– 169. Sweller, J. , Van Merrienboer, J. , & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review 10, 251– 296. UK Ministry of Defence: Development, Concepts, and Doctrine Centre: ‘Human-Machine Teaming’. Joint Concept Note 1/18, 21 May 2018. https: //www. gov. uk/government/publications/human-machine-teaming-jcn-118 26