Can participatory evaluation approach trigger a greater demand

Can participatory evaluation approach trigger a greater demand for results? The evaluation of European Union‘s Cohesion policy in the 2014 -2020 Period Kruno Karlovcec, Slovenia Ministry for economic development and technology

Outline of the presentation • History of European Cohesion policy • History of Evaluation of European Cohesion policy • Experience in 2007 -2013 programming period • Participatory evaluation approach in 2014+ • Case study – Slovenia

History of European Cohesion policy • Purpose • The aim is to diminish the regional disparities among the European regions • Social and Economic cohesion • Solidarity • Tackling unemployment and regions lagging behind • Redistributive policy – restribution of wealth among countries and regions

• In 1980 s, Spain, Portugal and Greece joined European Union – less developed European countries • Regional disparities in EU increased • In the late 80 s, the shift from annual project selections to multi-annual programmes • New doctrine and principles:

• Concetration – on a limited number of projects focusing on least developed regions • Multi-annual programming based on analysis, strategic planning and evaluation • Principle of partnership with relevant stakeholders

Evaluation of European Cohesion policy • First ex-ante evaluation reports for the programming period 1994 -1999; • The evaluation of programmes intensified in 2000 -2006. • Mid-term evaluation in 2003 – extensive exercise • Evaluation Network – example of coordination and sharing of lessons learnt between member states

• Evaluation Network of Directorate General for Regional policy and Directorate General of Employment • The Evaluation Network comprises representatives of Member States who are responsible for evaluation of cohesion policy. • It meets two or three times each year and is chaired by the Evaluation Unit of DG

• In 2007 -2013 programming period – a more strategic approach of evaluation • Evaluation plans – ex-ante, on-going (formative) and ex-post (summative) evaluation • Very flexible approach to evaluation, serious issue is the time-lag (long process of wining the contract) and thus „outdated“ recommendations

Ownership Stakeholders Empowerment participation learning negotiation flexibility Participatory evaluation approach Competencies Standards

Key points 1. Stakeholder identification process is an ex-ante process of determining individuals/organisation affected by the intervention. 2. Active involvement of stakeholders in the evaluation process is crucial as it increases the demand for results and volumes up the ownership and empowerment. 3. EU is adopting a needs-driven approach which facilitates the participatory evaluation approach

• Stakeholders are organisations/individuals who are affected by intervention during its lifetime or in subsequent years (Rist, R and Morra Imas, L. , 2009). • It is important to include as stakeholders those who would not typically be asked to participate in evaluation. • “Stakeholder analysis” is a method for stakeholder identification management. • Who are the stakeholders in European Cohesion policy? • Direct (legislature – the institutional framework of European Cohesion Policy) Ø Policy-makers, members of (EU) Parliament Ø EU Citizen are affected by the EU Cohesion policy

• Usually, evaluations are conducted by top-down approach, • That’s how the ECP evaluations are predominantely elaborated; • Participatory evaluation approach started as a response to traditional top-down approach; • Utilisation of evaluation findings and recommendations – is it more effective in top-down or in PE?

Participatory evaluation (1) • Evaluation should and can be used to empower the citizens to analyse and solve their own problems and address the needs; • Participatory evaluation approach is more appropriate formative (mid-term, ongoing, process) evaluation than summative (ex -post, final);

Participatory Evaluation (2) What are the pros? What are the cons? • Useful information for the program managers, • Enhances utilisation of evaluation findings, • Ownership of the evaluation report, • Mutual responsibility • Flexibility and loosened rigidity of the evaluation process, • Empowerment and active participation of (EU) citizen • • • Time- and resource-consuming, The conflicts among stakeholders easily emerge, One of the important issues is unequal powers of stakeholders Too much flexibility can distort the evaluation process, Too much time, money and other resources spent on stakeholder analysis which hinders evaluation process

How EC encourages the use of PEA in Evaluation • Reference in DG REGIO’s Draft Guidance on monitoring & evaluation, page 7, paragraph 2 • The principle of partnership embedded in the legislature, also in the 2014 -2020 (Article 5) • Multi-level governance – from local to national/European • Technical Assistance – resources, trainings, networking, • Still challenges: ownership and empowerment of the relevant stakeholders

Active participation and principle of partnership in the 2014+ • Social and economic cohesion – two basic pillars of ECP; • A new pillar was added – teritorrial cohesion, • Place-based programming enhances more effective needdriven implementation framework, • Place-based programming and implementation enhances the active participation of EU citizen in monitoring and delivery of results, • Partnership Agreement, Performance Framework – one of the new instruments to enhance better performance and evaluation utilisation…

• Vocabulary 2007 -2013: Efficiency, Effectiveness, Compliance… • Vocabulary 2014 -2020: Results, Change, Adressing Needs, Performance Management … and Territorial Cohesion • Which is added to Social and Economic Cohesion

SLOVENIA • Activities in the field of evaluation began in 2005 (2004 -2006 period) • Inter-ministerial steering group was established in 2008 (2007 -2013 period), • Negotiation process on the evaluation scope was extensive but evaluations were contracted out to external evaluators; • Evaluations used mainly descriptive and normative evaluation questions, designs & approaches, • No causal (cause & effect) question and design was ever used

SLOVENIA (2) • 2014 -2020 programming period: • 12 Regional development programmes 2014 -2020; • September 2013 – Our Directorate joined Community of Practise on Results-based management, • But does the Slovenia‘s cultural setting support the resultsbased management? • Geert Hostefe‘s National Culture Dimensions (http: //geerthofstede. com/national-culture. html)

SLOVENIA (3) • Power distance This dimension expresses the degree to which the less powerful members of a society accept and expect that power is distributed unequally. The fundamental issue here is how a society handles inequalities among people. People in societies exhibiting a large degree of power distance accept a hierarchical order in which everybody has a place and which needs no further justification… Slovenia‘s score is 71 out of 100

SLOVENIA (4) • Individualism vs. Collectivism The high side of this dimension, called Individualism, can be defined as a preference for a loosely-knit social framework in which individuals are expected to take care of themselves and their immediate families only. Its opposite, Collectivism, represents a preference for a tightly-knit framework in society in which individuals can expect their relatives or members of a particular in-group to look after them in exchange for unquestioning loyalty. Slovenia‘s score is 27 out of 100 – very collectivistic country

SLOVENIA (5) • Masculinity vs. Feminity The masculinity side of this dimension represents a preference in society for achievement, heroism, assertiveness and material reward for success. Society at large is more competitive. Its opposite, femininity, stands for a preference for cooperation, modesty, caring for the weak and quality of life. Society at large is more consensus-oriented. Slovenia‘s score is 19 out of 100 – „very feminine“ country

SLOVENIA (6) • Uncertainty avoidance The dimension Uncertainty Avoidance has to do with the way that a society deals with the fact that the future can never be known: should we try to control the future or just let it happen? Slovenia scored 88 out of 100 – high tendency to avoid the uncertainty

SLOVENIA – what‘s next • My idea is – demand for results must come from stakeholders and citizen • A survey sent to regional/local authorities on how the demand for results is already felt • Findings of survey in December 2013

Conclusions • Stakeholder analysis – identify relevant stakeholders, not all of them. • Identifying the needs is crucial and should drive • Participative evaluation approach enhances empowerment and ownership of relevant stakeholders – delivery of results is more effective.

Literature: • 1. Rist, Ray C. and Morra Imas, L. G. : The Road to Results, The World Bank, 2009; • 2. Zukoski, A. and Luluquisen, M. : Participatory Evaluation; What is it? Why do it? What are the challenges? , Communitybased Public Health Policy & Practice, 2002 • http: //geert-hofstede. com/ • COP RBM: http: //www. coprbm. eu/