Influencing Evaluation Policy and Practice A Progress Report

Influencing Evaluation Policy and Practice: A Progress Report from AEA’s Evaluation Policy Task Force Patrick Grasso Chair, AEA Evaluation Policy Task Force (EPTF) George Grob Advisor to the EPTF November 2011 1

AEA Evaluation Policy Task Force Purpose To influence evaluation policies that are critically important to the practice of evaluation 2

AEA Evaluation Policy Task Force Members Eleanor Chelimsky Catherine Dawes Patrick Grasso (Chair) Jennifer Greene Susan Kistler AEA Executive Director) Mel Mark Stephanie Shipman George Grob (Advisor) 3

AEA Evaluation Policy Task Force Consultative Campaign Federal Government Office of Management and Budget The Administration Priority Legislation Federal Regulations Administrative Guidances 4

AEA Evaluation Policy Task Force Public Presence Initiative AEA Member Engagement Outside Meetings AEA Newsletter Articles Media Coverage 5

Accomplishments July 2007 – June 2010 6

Policy Successes OMB Impact Evaluation Policy Evaluation Policies in the Federal 2011 and 2012 Budget Guidances U. S. Global HIV/AIDS Appropriations Guidance Evaluation Requirement of the Global Health Initiative Requirements for oversight of Health Care Reform GAO Review of International Food Assistance Programs Using Criteria Developed from the Roadmap 7

Recent Activities and Accomplishments July 2010 – Present 8

Policy Successes USAID Evaluation Policy Integration of evaluation and program planning Definitions and Distinctions of evaluiton types Requirements for evolutions of major programs and untested interventions Acknowledgement of need for both quantitative and quantitative methods 3% set aside of major program funds for evaluation GPRA Modernization Act Carries over existing policies requiring evaluation Schedules. Evaluating success in meeting program goals Requires identification of skills and competencies for goal setting and evaluation Requires Federal website to publish goals and assessments 9

Policy Successes State Department Evaluation Policy Evaluation framework covering all programs, projects, and activities in bureaus and missions Evaluation requirements Evaluation plans Evaluator Independence Evaluation Documentation GAO Report on International School Feeding Program Uses AEA Evaluation Roadmap as the standard for evaluation of this program HHS Inspector General authorized to hire 35 new evaluators to oversee implementation of Health Care Reform Legislation. (IG responsibility for oversight resulted from AEA advocacy 10

AEA Advice to Policy Makers To Government Accountability Office (GAO) on Auditing Standards To HHS Health Resources and Services Administration on evaluating evidence of the effectiveness for maternal, infant, and early childhood home visiting programs To Congressional staff developing evaluation legislation Senator Mark Udall Senators Thomas Carper and James M. Inhofe To NIH Evaluation Key Committee on Clinical and Translational Science Awards Inspection and Evaluation Committee of the Council of Inspector General on Integrity and Efficiency 11

Outreach to Public AEA letter to editor of New York Times in defense of evaluator independence 12

Engagement of AEA Members Invitation to comment or emerging policy related to the protection of human research subjects Monthly AEA “Policy Watch” Newsletter articles 13

Evaluation Policy Infrastructure Advocacy statements summarizing key evaluation policy approved by AEA Board Final approval and publication of AEA Evaluation Roadmap with AEA member and Board input Requirements and Procedures for involving AEA members and Board of Directors in formulating evaluation policy proposals 14

Evaluation Policy Infrastructure Procedures for future update of the AEA Evaluation Roadmap Formation of AEA Evaluation Policy TIG AEA Membership in Consortium of Social Science Associations (COSSA) 15

The Future 16

Priorities Targets of Opportunity Protection of human research subjects Clinical and translational research awards Other legislative and regulatory policy opportunities as they emerge New EPTF Members Formal assessment of the EPTF by the AEA Board More opportunities for AEA members to influence evaluation policy Strengthen connection with evaluation organizations 17

AEA Members’ Policy Watch Staying In Touch Email: evaluationpolicy@eval. org Website: http: //www. eval. org/EPTF. asp Discussion Group: http: //www. eval. org/EPTF. signup. di scussion. asp 18