Difficulties in implementing the assessment Russian experience Moscow

Difficulties in implementing the assessment: Russian experience Moscow June 7, 2019

Problem area: quality of program design 2

Program design A reasoned explanation of how and why the programs actions will lead to the achievement of the program's objective Program purpose Actions 3

From a speech of the chairman of Russian Federation Audit Chamber in the State Duma (2014) "The objectives of most programs are non-specific, contain vague language that allows for arbitrary interpretation, which makes it impossible to verify their implementation. . . Typical situations where the achievement of the program goal is not achieved by any of the goals or the task is not aimed at achieving any of the goals. " http: //www. ng. ru/economics/2014 -06 -11/4_programms. html 4

Problem area: mixture of genres 5

Evaluation role in program management Decision-making Assessment Inspection Research Identification and prevention of violations Obtaining new knowledge © Company "Process Consulting" , 2018 6

Phases of the program's lifecycle Evidencebased approach to program management What needs to be validated ("to prove") ? Initiation The correct understanding of a situation, relevance of the chosen task Planning Why actions within the program will result in the planned result Execution Understanding of the events, correctness of management decisions Program execution ? ? ? Achievement of the goal, reasons of deviations 7

The head of the Center for assessing the effectiveness of the Federal target programs summed up the results of check of the embodiment of the program for construction of schools in regions (2016) Member of the General Council of United Russia, the head of the Center for assessing the effectiveness of the Federal target programs , First Deputy Chairman of the State Duma Education Committee Vladimir Bourmatov during the performance from fraction at the morning plenary session of chamber of parliament paid attention to the revealed violations and various corruptogenic factors at program implementation for construction of new schools in regions. https: //er. ru/news/142066/ 8

Problem area: domination of one approach (indicator assessment) 9

Expert assessment Making a judgments based on knowledge which is available for experts 10

Indicator assessment Making a judgment based on the values of a limited number of measures (indicators) 11

Empirical assessment Judgment based on systematically collected empirical* data * (from Greek “empeiria” - experience) - establishing and generalizing social facts through direct or indirect registration of fait accomplished characteristic of studied social phenomena, objects and processes. 12

Expert Types of program evaluation Indicator Empirical 13

Methodology of assessment of Program implementation efficiency Assessment of Program implementation efficiency is made by the following criteria: Indicators The list of the main target indicators of Program implementation efficiency specified in Section IV of the Program Criteria for evaluation of Program implementation efficiency depending on the number of the reached target indicators, percentage of their total number inefficient insufficiently effective highly effective to 30 from 31 to 50 from 51 to 70 over 71 http: //www. duma. yar. ru/service/projects/docs/prilog 13. html 14

Assessing the effectiveness of the federal target program "Improving road traffic safety in 2013 -2020» Number of the persons who died in the road accidents 2014 Calculated value 2015 Number of the children who died in the road accidents 2016 2014 Actual Calculated Actual value value 2015 2016 Calculated Actual value value 14 15 16 17 18 19 20 21 22 23 24 25 27025 26963 26814 23114 26500 20308 872 878 872 737 830 710 1223 1142 1215 1010 1203 812 30 34 29 30 156 140 155 119 154 100 5 4 5 5 5 8 64 60 63 56 62 69 0 0 0 2 428 391 425 302 420 244 13 9 13 6 12 8 215 181 214 195 212 150 4 5 4 10 4 5 189 170 188 152 186 100 5 5 5 1 http: //www. fcp-pbdd. ru/achievements/efficiency/ 15

16