Process of exante evaluation for the 2014 2020

Process of ex-ante evaluation for the 2014 -2020 programming period Tomasz Kot Head of National Evaluation Unit, Ministry of Regional Development, Poland Budapest, 26 th September, 2013 Ministerstwo Rozwoju Regionalnego - listopad 2020 1

Ex-ante evaluation arrangements in Poland: CPR requirements (art. 48) and Guidance Document on Ex-ante Evaluation as a reference points to ex-ante evaluation arrangements in Poland Main rules developed in close cooperation with all Operational Programme Evaluation Units (3 seminars organized) Based on CPR requirements, Guidance Document and consultation results National Evaluation Unit prepared the document: „Recommendations for ex-ante evaluation of the operational programmes 2014 -2020 in Poland” The document is coherent with CPR requirements and EC Guidance Document and includes: § Scope of the study (main evaluation questions with commentary) § Organizational issues § Methodological recommendations § Terms of Reference template Ministerstwo Rozwoju Regionalnego - listopad 2020 2

Main principles of the ex-ante evaluation in Poland: • close cooperation between evaluator and programming institutions • advisory services during OP negotiations • recommendations implemented as they arise (interactive process, workshop methods) • Steering Groups (participation of stakeholders) Participatory model of ex-ante evaluation: Intervention logic analysis as a methodological approach (Theory Based Evaluation): • analysis of cause and effect relations … • … based on evidence – intervention logic (cause and effect relations) verified on the basis of available knowledge (socio-economic theory, evaluation results etc. ) Strong coordination of the ex-ante evaluation process: • National Evaluation Unit as a coordinator • regular monitoring meetings of Ex-ante evaluation Steering Group (with all OP Evaluation Units) Ministerstwo Rozwoju Regionalnego - listopad 2020 3

State of play: Regional Operational Programmes: National Operational Programmes: • For all 16 ROP’s (ERDF+ESF) exante evaluation process in progress • Most of final reports expected in 2013 • for all of national OP’s (6) ex-ante evaluation process in progress (5 ERDF, 1 ESF) • in case of 2 OP’s evaluation process is finished (ESF programme: Knowledge – Education – Development OP, Technical Ministerstwo Rozwoju Regionalnego - listopad 2020 Asisstance OP) Partnership Agreement: • ex-ante evaluation process in progress - first draft of the final evaluation report 4

Partnership Agreement ex-ante evaluation: PA ex-ante evaluation divided into two main processes: • expert analysis of the PA intervention logic (cause and effect relations; empirical and theory based rationale of the intervention) • participatory evaluation – series of 30 thematic workshops with the main stakeholders and programing institutions + plenty of meetings (discussion on the PA intervention logic ) Two types of ex-ante evaluation products: • recommendations on the intervention logic (based on the expert analysis and workshop discussion) • recommendations on the document logic (based mainly on the workshop discussions) Sample recommendations already implemented: PA ex-ante evaluation rated by stakeholders as a useful instrument for programming process • Redefining the intervention logic in all thematic areas: • clear defining the main expected goals of the PA intervention (as a reference for OP’s) • improving complementarity of the planned interventions • defining the most important conditions for intervention effectiveness • Standardizing thematic priorities description: • socioeconomic rationale for all interventions/priorities • homogeneous architecture of results for all priorities logical narrative with reference to diagnosis • clear demarcation of the priorities Ministerstwo Rozwoju Regionalnego - listopad 2020 5

Key conclusions and challenges: Participatory model of ex-ante evaluation as an effective solution for „changing documents problem” (main shortage of 2007 -2013 ex-ante evaluation) Very positive reception of participatory ex-ante evaluation in the case of Partnership Agreement (useful instrument for programming institutions) Main challenges: not sufficient evaluation market for such complex evaluation process in short time period (tendency for oligopoly in the evaluation services market) changing the ex-ante evaluation requirements (e. g. administrative burdens assessment) especially in the context of participatory model of evaluation when TOR should be defined at very early stage Ministerstwo Rozwoju Regionalnego - listopad 2020 6

Thank you for your attention tomasz. kot@mrr. gov. pl Ministerstwo Rozwoju Regionalnego - listopad 2020 7