Evaluation 2011 Values and Valuing Multisession 329 Valuing

Evaluation 2011: Values and Valuing Multisession 329 Valuing Knowledge Flow in the Evaluation Process Karen Widmer, M. S. Claremont Graduate University November 3, 2011 Karen. Widmer@cgu. edu 707 -815 -2286 123 E. 8 th Street Claremont, CA 91711

Widmer’s Ode to Knowledge § The flow of knowledge is the chief necessary condition for evaluation capacity/building § It mediates or moderates all evaluative activities and learning § It determines how well an organization can selfassess and how well it can use what it finds when it looks.

LEARNING ORGANIZATIONS Valuing Knowledge Flow in the Evaluation Process Knowledge platform Mediates Moderates Evaluative capacity Organizational performance My model suggests: By building the platform on which it stands, evaluation capacity grows and delivers a return on performance. A learning organization does this well.

Knowledge Basics Follow the flow = better evaluation Polanyi (1966) “Tacit and explicit knowledge” Argyris and Shon ( 1978) “Single- and double-loop learning” Nonaka (1994) “Knowledge creation” Becerra-Fernandez (2010) “Knowledge roles” Davenport & Prusek (1988) “Data Knowledge” Information Knowledge Evaluation Performance

Tacit and Explicit Knowledge Polanyi 1. Tacit: “Indwells” - understanding that resides in the mind Cognitions Technical know-how Relationships ~ and trust 2. Explicit: “articulated” – written or recorded content Apply Ways to elicit tacit knowledge on-the-job Collaborative problem-solving, exit interview, focus group, network, job rotation, facilitate, mentor, apprentice, rounds, coach, cross -train, “master”, lessons learned, case-based reasoning Did observed outcomes match the evaluand’s program theory?

Argyris & Shon Single- and double-loop learning Single-Loop Learning – low-level evaluation Underlying assumptions What we do Results Double-Loop Learning - continuously create new knowledge by reconstructing the framework Apply Do you structure reframing into your evaluations?

Nonaka Converting the forms of knowledge Tacit knowledge Explicit knowledge Apply Explicit knowledge Socialization Externalization Internalization Combination Are participants in synch and ready to action-ize the newly internalized?

Becerra. Fernandez Four ways to use tacit Discover o Socializationa o Combinationb Capture o Internalizationa o Externalizationb Share o Socializationa o Exchangeb a Apply explicit flow! Apply o Directiona, b o Routinea =tacit knowledge; b = explicit knowledge Awareness of the purposes that are served by knowledge aids evaluation design and choice of measures

Davenport & Prusek Distinguishing Data Information Knowledge Discrete objective facts Data with a message Actionable insights Y-values § § § 4 3 2 1 0 0 Apply 2 Categorized Condensed Corrected Calculated Contextualized § § § Experience Ground truths Complexity rules Involves judgment Values and beliefs * 4 What do you need to measure (data, information, or knowledge)? And to what end?

2 nd Generation Concepts More recent streams of knowledge flow Oliver (2009) “Knowledge transfer” Ashley (2009) “Innovation diffusion” Davison (2009) “Knowledge translation” Hawe (2009) “The complexity lens” Apply What other disciplines evaluate by following the knowledge flow? Hint Information systems, business management, instructional design, public policy, healthcare…other?

Widmer’s Ode to Knowledge § The flow of knowledge is the chief necessary condition for evaluation capacity/building § It mediates or moderates all evaluative activities and learning § It determines how well an organization can selfassess and how well it can use what it finds when it looks.

For more on knowledge and evaluation? Join Us! (11: 40 AM, today) Session 329: The Value of Knowledge Management in Evaluation: A Research Perspective • Knowledge Translation • Knowledge Management/Knowledge Exchange Multi-paper Session 329; Laguna A Thursday, Nov 3; 11: 40 AM to 12: 25 PM Sponsored by the Research on Evaluation TIG

Reference s Ashley, S. R. (2009). Innovation diffusion: Implications for evaluation. In J. Ottoson & P. Hawe (Eds. ), New Directions for Evaluation, 124, 35 -46. DOI: 10. 1002/ev. 310 Argyris, C. , & Schön, D. (1978) Organizational learning: A theory of action perspective. Reading, MA: Addison Wesley. Becerra-Fernandez, I. , & Sabherwal, R. (2010). Knowledge management: Systems and processes. Armonk, N. Y. : M. E. Sharpe, Inc. Davenport, T. H. & Prusak, L. (1998). Working knowledge. How organizations manage what they know. Boston, MA: Harvard Business School Press. Davison, C. M. (2009). Knowledge translation: Implications for evaluation. In J. Ottoson & P. Hawe (Eds. ), New Directions in Evaluation, 124, 75 -88. DOI: 10. 1002/ev. 310 De. Groff, A. & Cargo, M. (2009). Policy implementation: Implications for evaluation. In J. Ottoson & P. Hawe (Eds. ), New Directions in Evaluation, 124, 47 -59. DOI: 10. 1002/ev. 310

Reference s Hawe, P. , Bond, L. , Butler, H. (2009). Knowledge theories can inform evaluation practice: What can a complexity lens add? In J. Ottoson & P. Hawe (Eds. ), New Directions in Evaluation, 124, 89 -100. DOI: 10. 1002/ev. 310 Nonaka, I. (1994). A dynamic theory of organizational knowledge creation. Organization Science, 5 (1), 14 -37. Oliver, M. L. (2009). The transfer process: Implications for evaluation. In J. Ottoson & P. Hawe (Eds. ), New Directions in Evaluation, 124, 61 -74. DOI: 10. 1002/ev. 310 Polanyi, M. (1966). The tacit dimension. London, England: Routledge and Kegan Paul. Preskill, H. & Boyle, S. (2008). A multidisciplinary model of evaluation capacity building. American Journal of Evaluation, 29 (4), 443 -459. Online. Retrieved September 11, 2008, from http: aje. sagepub. com; doi: 10. 1177/1098214008324182

Evaluation 2011: Values and Valuing November 3, 2011 Multisession 329 Contact Information Valuing Knowledge Flow in the Evaluation Process Karen Widmer, M. S. Claremont Graduate University 123 E. 8 th Street Claremont, CA 91711 karenwidmer@earthlink. net (707) 815 -2286