The Logic Model and Systems Thinking Can They

The Logic Model and Systems Thinking: Can They Co-Exist? American Evaluation Association Conference November 2009 Orlando, FL Robert F. Richard, Ph. D. Seven-oaks@cox. net

In theory there’s no difference between theory and practice. In practice there is. Yogi Berra

What are we here to think about? Logic Model background Systems Thinking background How do they mix/blend/co-mingle?

Logic Model

Program Development Program Performance SEEC Evaluate impacts Practices KASA Reactions Participation Activities Resources Assess needs/opportunities and program design SEEC Participation Activities Evaluate processes Bennett’s Model

Assess needs Define Extension’s role and nature of commitment Assess resources and develop program Initiate networks and coalitions Provide educational experiences Consequences Louisiana 4 -H Evaluability Assessment Program Logic Model Richard (1992)

Logical Linkages: Series of If-Then Relationships If Then If Program invests time & money Inputs Resource inventory can be developed Families will know what is available Outputs Then If Families will access services Then Families will have needs met Outcomes

Basically a description of what has to happen to get what we desire. A tool that describes theory of change underlying an intervention.

Program reaches out to pregnant teens Pregnant teens enter and attend the program (participation) Teens learn prenatal nutrition and self care (increased knowledge) Teens develop commitment to take care of themselves and their babies (attitude change) Teens adopt healthy behaviors; no smoking, no drinking; attend prenatal clinic, eat properly (behavior change) Teens have healthy babies (desired outcome)

Other Systems Welfare Prenatal clinic and hospital outreach Prenatal program Legal Nutrition programs Transportation Child protection Media messages School system Context factors Politics Economic incentives Social norms Culture Music Church Pregnant teen’s knowledge, attitudes and behavior Other community based programs Youth culture

A Logic Model will tell you that the after school program is an activity and improved reading scores is an outcome. It might tell you that attendance at the after school program is an intermediate outcome. 11

But it won’t tell you that: “students need to attend after-school programs at least 3 days per week for a minimum of 60 days, and the curricula must focus on love of reading and literacy, IN ORDER FOR test scores to rise” 12

Systems Thinking

Systems • Parts are interdependent such that a change in one part changes all parts • The whole is greater than the sum of the parts • Focus on interconnected relationships • Systems are made up of sub-systems and function within a larger system

Systems concepts in evaluation • Offers more effective ways of dealing with complexity and complex situations • Develops new ways of understanding situations • Pays attention to coalitions • Pays attention to properties that emerge unexpectedly • Helps identify leverage points • Recognizes the evolutionary nature of programs

It is unwise to focus on one view or definition of a system without examining its relationship with another system Systems concepts can be particularly useful to evaluators in situations where rigorous rethinking, reframing and unpacking complex realities and assumptions

Simple Following a recipe v Recipes produce standard products v Certainty of same results every time Complicated Complex Going to the moon v. Formulas are critical and necessary Raising a child v. No instructions; difficult to predict results v. High level of expertise in many specialized fields + coordination v. Interdependent and non-linear v. High degree of certainty of outcome v. Multitude of external factors

Simple Cause Effect Complicated Parts Results Parts Complex

Far from Complex Agreement Environmental issues Global warming Poverty Close to Complicated Simple ANR prescriptive Close to Certainty Far from The Stacey Matrix

In systems work, richness implies that the whole can only be understood as a product of its parts plus the dynamic relationship between those parts Does our use of the Logic Model sometime ignore the dynamic relationship between parts?

…logic models are far more than templates for preparing plans of work. Without such understanding…Extension educators tend to focus on “filling out the forms” rather than using logic models as a framework for articulating program theory. Consequently the linkages between inputs, outputs and outcomes tends to be weak or unconfirmed. Rennekamp & Arnold (2009) Journal of Extension

Alice Boucher Project Inputs Agent time Materials School space Outputs School Garden Body Walk 4 -H club Outcomes Opportunities for parents to work with children Barriers Little experience with gardening Absentee parents/grandparents as main provider Limited resources More engaged parents Assumptions Parents desire to be with their children Community is interested in gardening School is supportive of effort

Alice Boucher Project More engaged parents School Garden Subject matter, Teamwork, Cooperation Involve Master Gardeners? What is their knowledge of youth and adult education? When can parents be involved? Evenings, weekends? How much can be produced? How is it distributed? Do parents know how to prepare? Body Walk 4 -H Club Improved nutrition How are lessons provided before, during and after? How are parents involved? In lessons, in actual event? When – evenings, weekends? Improved social skills How much repetition is necessary? Are nutritious foods available locally? Parental involvement? Leader/co-leader? Club structure to support sense of belonging, group behavior Project enrollment? How many meetings attended before change in behavior can be expected?

More engaged parents School Garden Subject matter, Teamwork, Cooperation Body Walk Improved nutrition 4 -H Club Improved social skills

More engaged parents How long (years) a project is this? Do we need to measure something for each program component? School Garden Subject matter, Teamwork, Cooperation How are parenting skills purposefully introduced for each program component? What will the school consider a success? What will each component consider a success? Body Walk Improved nutrition Is there agreement as to what better parenting skills look like? What do we really want to measure? 4 -H Club Improved social skills

In preparing for battle I have always found that plans are useless, but planning is indispensible. Dwight D. Eisenhower

So what do we have here? I think…

Logic models graphically illustrate program components, and creating one helps stakeholders clearly identify outcomes, inputs and activities Systems Thinking help us to create a theory of change to explain HOW and WHY the desired change is expected to come about 28

Logic Models require identifying program components, so you can see at a glance if outcomes are out of sync with inputs and activities, but they don’t show WHY activities are expected to produce outcomes Systems Thinking require justifications at each step – you have to articulate the hypothesis about why something will cause something else

…boundaries in human systems are massively entangled. An individual participates in multiple natural systems: family, church, community etc. Emergent patterns within each of those boundaries shapes the individual who, in turn, shapes patterns as they emerge in other contexts.

Other Systems Welfare Prenatal clinic and hospital outreach Prenatal program Legal Nutrition programs Transportation Child protection Media messages School system Context factors Politics Economic incentives Social norms Culture Music Church Pregnant teen’s knowledge, attitudes and behavior Other community based programs Youth culture

A common understanding of complexity is that it results from the number of elements and/or interactions in a system being beyond the capacity of the observer to readily understand. …even a small number of elements and/or interactions may be complex when they are in a dynamic and difficult to predict process of change.

Alice Boucher Project More engaged parents School Garden Subject matter, Teamwork, Cooperation Involve Master Gardeners? What is their knowledge of youth and adult education? When can parents be involved? Evenings, weekends? How much can be produced? How is it distributed? Do parents know how to prepare? Body Walk 4 -H Club Improved nutrition How are lessons provided before, during and after? How are parents involved? In lessons, in actual event? When – evenings, weekends? Improved social skills How much repetition is necessary? Are nutritious foods available locally? Parental involvement? Leader/co-leader? Club structure to support sense of belonging, group behavior Project enrollment? How many meetings attended before change in behavior can be expected?

…building an understanding that elements of a system interact through feedback loops, that causes often are not linked directly in time with effects, and that delays can cause unanticipated behavior are extremely valuable for both the program designers and evaluators.

Emerging complexity is characterized by disruptive change whose challenges include: 1. The solution to the problem is unknown 2. The problem statement itself is still unfolding 3. Who the key stakeholders are is not clear

It’s obvious by now: Systems Thinking, aka Theories of Change or Program Theory take a lot of work and time. On the other hand, Logic Models may sometimes be too simplistic to be helpful…. . 36

Do you agree/disagree? Are we asking the right questions? Do we have the correct boundaries?

Just because something doesn't do what you planned it to do doesn't mean it's useless. Thomas A. Edison

Eoyang, G. H. (2007). Human Systems Dynamics: Complexity-Based Approach to a Complex Evaluation. In B. Williams & I. Iman (eds). Systems Concepts in Evaluation: An Expert Anthology (pp. 123 -139). Point Reye, CA: Edge. Press of Inverness. Frechtling, J. A. (2007). Logic Modeling Methods in Program Evaluation San Francisco, CA: Jossey-Bass Patton, M. (1997). Utilization Focused Evaluation (3 rd ed. ). Thousand Oaks, CA: Sage Publications. Scharmer, O. (2007). Theory of U. Cambridge, MA: The Society for Organizational Learning, Inc. Westley, F. , Zimmerman, B. & Patton, M. (2006). Getting To Maybe. Random House, Canada. Wholey, J. , Hatry, H. , Newcomer, K. (2004). Handbook of Practical Program Evaluation. San Francisco, CA: Jossey-Bass

Chaos, Confusion, Disorder…. My work here is done

Robert F. Richard, Ph. D. Thinking Guide P. O. Box 916 Scott, LA 70583 Seven Oaks Consulting Helping you think Phone: 337 -230 -9944 E-mail: seven-oaks@cox. net