Evaluation 201 Skills You Need Are Not Taught

  • Slides: 19
Download presentation
Evaluation 201: Skills You Need & Are Not Taught From Bumps & Bruises, to

Evaluation 201: Skills You Need & Are Not Taught From Bumps & Bruises, to Wisdom Martha Ann Carey, Ph. D, RN Kells Consulting Media, PA Carey & Engle Molly Engle, Ph. D Oregon State University Corvalis, OR AEA 2010 copyright 2010

What is Needed For Success • Methodological coherence – Purpose, Process, Product • Communication

What is Needed For Success • Methodological coherence – Purpose, Process, Product • Communication – Formal & informal • Management – beyond logic models Carey & Engle AEA 2010 copyright 2010

Interdependent & Interlaced Carey & Engle AEA 2010 copyright 2010

Interdependent & Interlaced Carey & Engle AEA 2010 copyright 2010

Interdependent & Interlaced Carey & Engle AEA 2010 copyright 2010

Interdependent & Interlaced Carey & Engle AEA 2010 copyright 2010

Methodological Coherence • Purpose – Goals, objectives, aims – Project & program • Process

Methodological Coherence • Purpose – Goals, objectives, aims – Project & program • Process – Sampling, analysis, reporting • Product – Knowledge, improved outcome Carey & Engle AEA 2010 copyright 2010

Multi-Level Programs • Adds complexity challenges – Not a fractal – Overall program as

Multi-Level Programs • Adds complexity challenges – Not a fractal – Overall program as complex, adaptive system <Teague> • Process - form, storm, norm, perform • “Theory” & program - espoused vs actual Carey & Engle AEA 2010 copyright 2010

Communication • Basic and most challenging – Intuitively appears simple – Good will =

Communication • Basic and most challenging – Intuitively appears simple – Good will = not enough • Ongoing process • “Listening” & “culture” • Cultures – demography, disciplines - Bridging gaps Carey & Engle AEA 2010 copyright 2010

Formal & Informal Communication • Formal - regular & established – Documents, email, etc.

Formal & Informal Communication • Formal - regular & established – Documents, email, etc. – Listserv, required to be read • Informal – recognize and “utilize“ • “Meeting before the meeting” • Cultures – demography, disciplines Carey & Engle AEA 2010 copyright 2010

Management • Monitoring & feedback – what, who, how • Communication = integral •

Management • Monitoring & feedback – what, who, how • Communication = integral • Cultures - demography, disciplines • CBPR – advisory board Carey & Engle AEA 2010 copyright 2010

Structure • “No” flexibility • Very flexible – may be difficult to manage •

Structure • “No” flexibility • Very flexible – may be difficult to manage • Adapt to changes • Cultures - demography, disciplines Carey & Engle AEA 2010 copyright 2010

Roles & Responsibilities • Established – SOW, TOR – regulations, policies • Agreements -

Roles & Responsibilities • Established – SOW, TOR – regulations, policies • Agreements - developed – sense vs consensus – “parking lot” – in writing & with a date – review & revise Carey & Engle AEA 2010 copyright 2010

R & R, continued • How to know rules • Data access & publications

R & R, continued • How to know rules • Data access & publications • Cultures – ethnic, disciplines Carey & Engle AEA 2010 copyright 2010

Resistance & Conflict Management • Personal & professionally meaningful goals – Perceived as being

Resistance & Conflict Management • Personal & professionally meaningful goals – Perceived as being listened to, & achievable goals (Shapiro, AJPH, October 2010) • Multiple objectives, conflicting objectives – “manage expectations” • Resources for both • Continuity • Cultures – ethnic, disciplines Carey & Engle AEA 2010 copyright 2010

Power • Professional, personal, situational • Ascribed characteristics, achieved • Who has power &

Power • Professional, personal, situational • Ascribed characteristics, achieved • Who has power & how do you get it? • Who controls the budget? • Cultures – ethnic, disciplines Carey & Engle AEA 2010 copyright 2010

Eliciting & Sustaining Participation • Who decides & assesses • Reward structure • What

Eliciting & Sustaining Participation • Who decides & assesses • Reward structure • What impact & how measured • Cultures – ethnic, disciplines Carey & Engle AEA 2010 copyright 2010

Ethics • Ongoing concern • “Human Subjects” protection • Cultures – ethnic, disciplines •

Ethics • Ongoing concern • “Human Subjects” protection • Cultures – ethnic, disciplines • Whose “voice? ” Carey & Engle AEA 2010 copyright 2010

Other Topics - ? For Future Sessions • • Software Evidence based issues Systems

Other Topics - ? For Future Sessions • • Software Evidence based issues Systems concepts OD & management theories Research- practice interface, or gap Data management Reports & publications Carey & Engle AEA 2010 copyright 2010

Carey & Engle AEA 2010 copyright 2010

Carey & Engle AEA 2010 copyright 2010

Carey & Engle AEA 2010 copyright 2010 Dingle, Ireland 2010

Carey & Engle AEA 2010 copyright 2010 Dingle, Ireland 2010