A Program Evaluation Strategy The Causal Pathway Evaluation

  • Slides: 35
Download presentation
A Program Evaluation Strategy The Causal Pathway

A Program Evaluation Strategy The Causal Pathway

Evaluation Strategy • Our Evaluation Model starts in the initation phase of a program,

Evaluation Strategy • Our Evaluation Model starts in the initation phase of a program, not upon its conclusion. • We start by developing a “Causal Pathway” or roadmap, which identifies the steps necessary to achieve the desired goal. Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University

Causal Pathway Framework A roadmap of a project that shows: Center for Population and

Causal Pathway Framework A roadmap of a project that shows: Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University

Causal Pathway Framework Impact Change in the health, social, economic status of the population

Causal Pathway Framework Impact Change in the health, social, economic status of the population Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University

Causal Pathway Framework Effect: Change in the knowledge, attitudes, skills, behavior of the population

Causal Pathway Framework Effect: Change in the knowledge, attitudes, skills, behavior of the population of interest Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University Impact

Causal Pathway Framework Outputs Effect Outputs Products and services that must be in place

Causal Pathway Framework Outputs Effect Outputs Products and services that must be in place for the effects and impact to occur. Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University Impact

Causal Pathway Framework Activities Outputs Effect Activities Technical and support tasks required to produce

Causal Pathway Framework Activities Outputs Effect Activities Technical and support tasks required to produce the outputs Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University Impact

Causal Pathway Framework Inputs Activities Outputs Effect Inputs Resources needed to support the activities

Causal Pathway Framework Inputs Activities Outputs Effect Inputs Resources needed to support the activities Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University Impact

Evaluation Strategy • This Evaluation model starts by helping to design a program which

Evaluation Strategy • This Evaluation model starts by helping to design a program which has the highest probability of achieving success • We use this model to plan the program starting with the desired Goal or Impact, and planning all intermediate steps. Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University

Causal Pathway Framework Inputs Activities Outputs Effect Design Direction Center for Population and Family

Causal Pathway Framework Inputs Activities Outputs Effect Design Direction Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University Impact

Evaluation Strategy • Given that we now have a program with built-in logical steps

Evaluation Strategy • Given that we now have a program with built-in logical steps toward success, we can then use this Causal Pathway tool to evaluate the progress and success of the program. Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University

Evaluation Strategy • Whereas in the planning phase, we worked from the desired Impact

Evaluation Strategy • Whereas in the planning phase, we worked from the desired Impact back to necessary Inputs, in the Evaluation Phase we monitor the completion of each step from Inputs to Activities to Outputs to Effects to Impact. Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University

Causal Pathway Framework Inputs Activities Outputs Effect Monitoring and Evaluation Direction Center for Population

Causal Pathway Framework Inputs Activities Outputs Effect Monitoring and Evaluation Direction Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University Impact

Evaluation Strategy • One of the benefits of using this same tool in Evaluation

Evaluation Strategy • One of the benefits of using this same tool in Evaluation is that even if there has not been enough time to realize a change in Impact, we can still determine how far along the logical pathway our program has advanced. Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University

Causal Pathway Framework Say for instance that we observe no change in Impact: Inputs

Causal Pathway Framework Say for instance that we observe no change in Impact: Inputs Activities Outputs Effect Impact No Impact Monitoring and Evaluation Direction Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University

Evaluation Strategy • We can still determine how far along the logical pathway our

Evaluation Strategy • We can still determine how far along the logical pathway our program has advanced as an indication of success. • We can also use this process to Diagnose where things are not working Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University

Causal Pathway Framework Inputs Activities Outputs Effect Impact No Impact Were inputs available? Were

Causal Pathway Framework Inputs Activities Outputs Effect Impact No Impact Were inputs available? Were activities performed? Were outputs produced? Were effects observed? Center for Population and Family Health Joseph L. Mailman School of Public Health of Columbia University

BDI LOGIC MODEL

BDI LOGIC MODEL

“If you don’t know where you want to go, how do you know if

“If you don’t know where you want to go, how do you know if you’ve gotten there? ” THE RABBIT, Alice in Wonderland

BDI Logic Model Intervention Determinants Behavior Health Goal

BDI Logic Model Intervention Determinants Behavior Health Goal

What is resiliency?

What is resiliency?

Risk & Protective Factors

Risk & Protective Factors

Risk Factors: conditions that increase the likelihood of outcomes that comprise health, quality of

Risk Factors: conditions that increase the likelihood of outcomes that comprise health, quality of life, or life itself

Risk Factors Occur in 5 major life domains: family community individual peer groups school

Risk Factors Occur in 5 major life domains: family community individual peer groups school

The more risk factors, the greater the risk

The more risk factors, the greater the risk

Common risk factors predict diverse behavioral problems: • • • substance abuse delinquency violent

Common risk factors predict diverse behavioral problems: • • • substance abuse delinquency violent behavior risky sex teen pregnancy school behavior problems dropping out

If the impact of any given risk factor can be reduced, it is likely

If the impact of any given risk factor can be reduced, it is likely to reduce several risk behaviors

Risk Behavior: behavior that can compromise adolescent psychosocial development Examples?

Risk Behavior: behavior that can compromise adolescent psychosocial development Examples?

Risk Taking Behavior: behaviors that entail a conscious awareness of the risk/danger in a

Risk Taking Behavior: behaviors that entail a conscious awareness of the risk/danger in a deliberate thrill-seeking attempt Examples?

Protective Factors: factors that buffer youth against the adverse effects of risk factors to

Protective Factors: factors that buffer youth against the adverse effects of risk factors to which they are exposed and diminish the likelihood of negative health and social outcomes

Protective Factors occur in many domains: • individual (e. g. resilient temperament, social &

Protective Factors occur in many domains: • individual (e. g. resilient temperament, social & cognitive skills) • connections to family, friends, school, other adults • opportunities for involvement (e. g. church, youth clubs in communities)

Many adolescent risk behaviors serve a purpose and are goaldirected

Many adolescent risk behaviors serve a purpose and are goaldirected

Drinking, smoking grass, smoking cigarettes • • • feel relaxed feel socially accepted feel

Drinking, smoking grass, smoking cigarettes • • • feel relaxed feel socially accepted feel independent feel grown up feel excitement bucking the system

Magnitude of Risk? • • intensity of involvement # of risk behaviors age of

Magnitude of Risk? • • intensity of involvement # of risk behaviors age of onset presence of protective factors

Implications for Prevention • comprehensive, multiple domains & multiple behaviors • target risk and

Implications for Prevention • comprehensive, multiple domains & multiple behaviors • target risk and protective factors • address larger social context in which behavior embedded

Intrinsic pathway Extrinsic pathway Common pathway thombosis ThomboembolusIntrinsic pathway Extrinsic pathway Common pathway thombosis Thomboembolus
Causal Loop Diagrams Causal loop diagrams can beCausal Loop Diagrams Causal loop diagrams can be
Causal Diagrams and the Identification of Causal EffectsCausal Diagrams and the Identification of Causal Effects
Causal Directed Acyclic Graphs DAG Causal Diagrams 2013Causal Directed Acyclic Graphs DAG Causal Diagrams 2013
Using Causal Conjunctions Causal conjunctions are used toUsing Causal Conjunctions Causal conjunctions are used to
Causal Inference Causal Graph D Drug C CureCausal Inference Causal Graph D Drug C Cure
Causal Networks Denny Borsboom This is a causalCausal Networks Denny Borsboom This is a causal
Using Causal Conjunctions Causal conjunctions are used toUsing Causal Conjunctions Causal conjunctions are used to
THEORY CONSTRUCTION 8 Causal Models Dina Abdelhafez CausalTHEORY CONSTRUCTION 8 Causal Models Dina Abdelhafez Causal
Causal Diagrams and the Identification of Causal EffectsCausal Diagrams and the Identification of Causal Effects
Experimental Design making causal inferences Causal and EffectExperimental Design making causal inferences Causal and Effect
SUSSERS CAUSAL CRITERIA Nigel Paneth CAUSAL CRITERIA COMPAREDSUSSERS CAUSAL CRITERIA Nigel Paneth CAUSAL CRITERIA COMPARED
CHAPTER 12 CAUSAL ARGUMENTS KINDS OF CAUSAL ARGUMENTSCHAPTER 12 CAUSAL ARGUMENTS KINDS OF CAUSAL ARGUMENTS
aka Cause and Effect Essays CAUSAL ESSAYS Causalaka Cause and Effect Essays CAUSAL ESSAYS Causal
Causal Inference Chapter 1 Preliminaries Statistical and CausalCausal Inference Chapter 1 Preliminaries Statistical and Causal
International Strategy Multidomestic Strategy Global Strategy Transnational StrategyInternational Strategy Multidomestic Strategy Global Strategy Transnational Strategy
Content Strategy Content Strategy Content Strategy Content StrategyContent Strategy Content Strategy Content Strategy Content Strategy
Ys strategy Strategy 1 Strategy 2 Strategy 1Ys strategy Strategy 1 Strategy 2 Strategy 1
Strategy Strategy Phases Strategy Plan l Strategy DocumentStrategy Strategy Phases Strategy Plan l Strategy Document
Pathway Program GPA Explanation Pathway Grading System LearnPathway Program GPA Explanation Pathway Grading System Learn
Causal Pathway of Disease or Disability n nCausal Pathway of Disease or Disability n n
Causal Pathway of Disease or Disability n nCausal Pathway of Disease or Disability n n
Program Evaluation Lecture Overview Program evaluation and programProgram Evaluation Lecture Overview Program evaluation and program
1 Offshoring Strategy Framework Strategy Evaluation Efficiency Evaluation1 Offshoring Strategy Framework Strategy Evaluation Efficiency Evaluation