EVALUATION IN METHODOLOGICAL RECOMMENDATIONS FOR THE DEVELOPMENT OF

EVALUATION IN METHODOLOGICAL RECOMMENDATIONS FOR THE DEVELOPMENT OF FEDERAL AND REGIONAL PROGRAMS AND PUTTING THEM INTO PRACTICE Aleksander Borovykh Strategy Director 1 Charitable Fund «Downside Up»

METHODOLOGICAL GUIDANCE ON DEVELOPMENT AND IMPLEMENTATION OF THE GOVERNMENT PROGRAMS IN THE RUSSIAN FEDERATION DECREE № 582 OF THE MINISTRY OF ECONOMIC DEVELOPMENT AS OF SEPTEMBER, 16 2016: “ON APPROVAL OF GUIDELINES FOR THE DEVELOPMENT AND IMPLEMENTATION OF GOVERNMENT PROGRAMS OF THE RUSSIAN FEDERATION” Government programs development is implemented based on the following principles: «setting up measurable implementation objectives for the government programs (final outcomes, i. e. status of the socioeconomic development (status change), characterized by quantitative and/or qualitative indicators…, which reflects the benefits from the government program (subprograms) implementation and immediate results, i. e. indicators of volume and quality of the event focused on final result of the government program implementation (subprogram)…» 2

DEFINITION OF THE EXPECTED OUTCOMES OF THE GOVERNEMNT PROGRAM Expected outcomes of the government program implementation are indicated as qualitative and quantitative characteristics of the basic expected (planned) final outcomes (changes reflecting the effect caused by the government program implementation) including description of specific completed events (facts, phenomena), allowing unambiguously assess the results of the government program implementation, as well as the values of indicators for the last year of the program implementation and their dynamics.

INDICATORS OF THE GOVERNMENT PROGRAMS The amount of the indicators is set up based on the principles of necessity and sufficiency to achieve the goals and objectives of the government program. At the level of the government program, the indicators focused on achieving only the final outcomes of its implementation should be reflected. Established indicators of the government program's subprogram may characterize both immediate and final outcomes of its implementation, at the same time, their amount cannot exceed more than twice the amount of the main activities implemented within the subprogram framework.

INDICATORS OF THE GOVERNMENT PROGRAMS The indicator system of the government program (subprogram) should allow to clearly assess the progress in achieving all the goals and objectives of the government program (subprogram).

INDICATORS SHOULD MEET THE FOLLOWING REQUIREMENTS: Adequacy Accuracy Objectiveness Comparability Unambiguity Cost effectiveness Credibility Timeliness and coherence

GOVERNMENT PROGRAM (SUBPROGRAMS) INDICATORS HAVE TO INCLUDE: indicators, characterizing quantitatively the process of the program implementation, achievement of its main objectives and goals; indicators, which reflect the quality of the most mass and socially significant public services provided by the federal executive bodies, as well as the main indicators of the state task in terms of quality and volume of the services provided.

Concept of early intervention development in the Russian Federation for the period till 2020 Decree of the Government of the Russian Federation as of August, 31 2016, No 1839 -p

CONCEPT OF EARLY INTERVENTION DEVELOPMENT Priority areas of activity on addressing the indicated objectives are: ensuring continuous professional development of specialists providing early intervention services; ensuring early intervention service quality management and creation of performance evaluation criteria of the services provided; organization of monitoring of the provision of the early assistance to children of the target group and their families.

CONCEPT OF EARLY INTERVENTION DEVELOPMENT Creation of the legislative basis for early intervention presupposes: Development of official statistical reporting indicators characterizing the level of development of early intervention in the Russian Federation. Creation of conditions for the timely commencement of early intervention includes: development of key indicators for the quality of early intervention and statistical indicators of its efficiency

CONCEPT OF EARLY INTERVENTION DEVELOPMENT Enabling environment for quality management of early intervention services and determining the criteria for performance evaluation of these services includes: management of the program of early intervention ensuring its sustainability, development, high quality, methodological and organizational integrity;

CONCEPT OF EARLY INTERVENTION DEVELOPMENT Developing early intervention in the regions, taking into account this Concept, provides for: introduction of the monitoring and performance evaluation systems, allowing to manage the quality of early intervention services; development of the criteria for the effectiveness of early intervention services and criteria for the completion of an individual program of early intervention, exit from the program of early intervention.

“Guidelines for creation of programs and a plan for the development of early intervention in the constituent entity of the Russian Federation” (approved by the Ministry of Labor of Russia, April 2018). http: //legalacts. ru/doc/metodicheskie-rekomendatsii-po-sozdaniiuprogramm-i-plana-razvitija-rannei/#100102

1. 3. Principles of creation of the system of early intervention in the region (constituent entity) of the Russian Federation: 1. 3. 6. Existence of a quality management system for the provision of early intervention services, including presence of clients’ feedback mechanisms and independent quality assessment of the early intervention services. 1. 11. Monitoring and evaluation of the effectiveness of the implementation of the Early Intervention Program in the region of the Russian Federation. 1. 1 Mechanisms and instruments of the monitoring and evaluation of the implementation efficiency of the Early Intervention Program in the region of the Russian Federation. 1. 11. 2 Implementation effectiveness indicators of the Early Intervention Program in the region of the Russian Federation (Annex 1).

CREATION OF PROGRAMS AND DEVELOPMENT PLANS FOR EARLY INTERVENTION IN THE REGION OF RUSSIAN FEDERATION 1. 11. 3. Annual implementation report on the realization of the Early Intervention Program in the region (constituent entity) of the Russian Federation, presented at the session of the Coordination body of the supreme executive authority of the region of the Russian Federation. 2. Implementation plan of the Early Intervention Program development in the region of the RF. 2. 2. Early Intervention Program development activities. 2. 2. 6. Design and introduction of the independent quality evaluation mechanisms of the early intervention services and beneficiary’s feedback.

Annex 1 RECOMMENDED SOURCES OF INFORMATION FOR DETERMINING INDICATORS OF THE EARLY INTERVENTION PROGRAM PEFORMANCE IN THE CONSTITUENT ENTITY OF THE RUSSIAN FEDERATION

DEFINING INDICATORS FOR ASSESSING EFFECTIVENESS OF THE EARLY INTERVENTION PROGRAM 1. Data on the existence of the children early detection system and its description. 2. Results of monitoring the need for early intervention services. 3. Percentage of children for whom an individual program of early intervention is designed, receiving early help in natural situations by their parents.

DEFINING INDICATORS FOR ASSESSING EFFECTIVENESS OF THE EARLY INTERVENTION PROGRAM 4. Percentage of children obtaining services in the framework of the individual program of early intervention, by whom the decrease of the degree of disability was registered during the current year. 5. Outcomes of the early intervention services assessment received by the adults, raising a child (biological or foster parents, caregivers, other relatives). 6. Challenges of the Early Intervention Program functioning in the region of the Russian Federation.

EXAMPLES OF THE GOALS, OBJECTIVES AND INDICATORS FROM PRACTICE OF THE GOVERNMENT PROGRAMS

CONCEPT GOAL OF THE EARLYI NTERVENTION The purpose of this Concept is to develop the basic principles and guidelines for creating conditions for the early intervention services provision on an interdepartmental basis, ensuring early detection of health issues and disabilities, optimal development and adaptation of children, integration of a family and a child into society, prevention or reduction of severity of disability, strengthening physical and mental health,

OBJECTIVES OF THE EARLY INTERVENTION CONCEPT (EXAMPLES) development of the legal and methodological framework guidelines and regulations for early intervention, taking into account best domestic and foreign practices; transition from private models of early intervention organizing in certain regions to creating a unified system of early intervention, taking into account regional aspects; Creation of the environment for early intervention programs in the regions of the Russian Federation.

PRIORITY AREAS OF ACTIVITY TO ADDRESS THESE OBJECTIVES ARE (EXAMPLES): creation of the legal framework for early intervention services; methodological support for the creation and operation of early intervention programs; improvement of mechanisms for the timely detection of the children in need of early intervention, and the definition of the criteria for their inclusion in the program of early intervention; ensuring the territorial and financial accessibility of early intervention services for children of the target group and their families; ensuring that children in the target group and their families have access to the full range of early intervention services required;

MODEL METHODOLOGY FOR ASSESSING THE QUALITY AND EFFECTIVENESS OF EARLY INTERVENTION SERVICES FOR CHILDREN AND THEIR FAMILIES Indicator 1 – availability of a coordination system for development of early intervention in the region (constituent entity) of the Russian Federation. Indicator values and evaluation criteria: availability of downloadable current open documents (posted on the official web-resources of the executive authorities of the constituent entity of the Russian Federation, a specialized Internet resource to support the development of early assistance to children and their families in the Russian Federation) on creation, composition and operations of the interdepartmental Coordinating Body for the development of the Early Intervention Program in the region (constituent entity) of the Russian Federation , …….

ASSESSMENT OF EFFECTIVENESS OF EARLY INTERVENTION SERVICES INCLUDES USING THE FOLLOWING INDICATORS: Indicator 12. Percentage of children who demonstrate a decrease in the severity of activity limitations by target categories in accordance with the “International Classification of Functioning, Disability and Health” and (or) an increase in involvement in natural life situations in the process of implementing the IEIP (Individual Early Intervention Program) in the total number of children receiving early intervention services under the IEIP.

ASSESSMENT OF EFFECTIVENESS OF EARLY INTERVENTION SERVICES INCLUDES THE USE OF THE FOLLOWING INDICATORS: Indicator 13. Percentage of families that noted the positive impact of the IEIP on any aspect of family functioning (e. g. , the understanding of the child's characteristics by family members, their ability to contribute to a child's development, the quality of their relationships and interactions with the child and in the family as a whole, the adaptation of the family, the expansion of positive social interaction between the family and society) in the total number of families involved in the implementation of the IEIP. Indicator 14. Proportion of children enrolled in educational institutions upon completion of the IEIP in the total number of children who completed the early intervention program in the current year.

EXAMPLES OF INDICATORS FOR MONITORING THE EFFECTIVENESS OF ACTIVITIES CARRIED OUT IN THE FRAMEWORK OF THE DECADE OF CHILDHOOD Improvement of the well-being of families with children Share of low income households with children under the age of 16 (18) in the total number of the low income households Modern childhood infrastructure Percentage of children aged 3 -6 years who are placed on the waiting list for a place in a preschool educational organization (as a percentage of the total number of children of the corresponding age who do not attend preschool educational organization)

EXAMPLES OF INDICATORS FOR MONITORING EFFECTIVENESS OF ACTIVITIES CARRIED OUT IN THE FRAMEWORK OF THE DECADE OF CHILDHOOD Ensuring children's safety Number of children subjected to violence (abuse) in organizations for orphans and children deprived of parental care during the year. Comprehensive education for children Number of competitions and other intellectual and (or) creative competitions, events aimed at the development of intellectual and creative abilities, abilities to engage in physical education and sports, interest in scientific (research) activities, etc.

EXAMPLES OF INDICATORS FOR MONITORING EFFECTIVENESS OF ACTIVITIES CARRIED OUT IN THE FRAMEWORK OF THE DECADE OF CHILDHOOD Safe information space for children Number of convictions for distribution of extremist materials online. Child and his right to family Proportion of orphans and children without parental care in foster care in the total number of orphans and children without parental care.

EXAMPLE FROM PRACTICE OF «DOWNSIDE UP» CF Vision of the Downside Up Staff, founders of Downside Up, families, people with DS, professionals, donors Monitoring & evaluation Programmes and projects of the fund Families, people with DS, professionals Experts, officials, NGOs, schools, etc. Surveys of the particular needs of the beneficiaries Strategy of the Charitable Fund «Downside Up» Case study and external data analysis Impact on the improvement of the life quality of people with DS Findings and recommendations regarding efficiency and social impact of the programmes People with DS and their families support program and special projects Findings on the needs of the families, experts and people with DS Mission and strategic goals Raw and secondary data

EXAMPLE FROM PRACTICE OF «DOWNSIDE UP» CF

EXAMPLE FROM PRACTICE OF «DOWNSIDE UP» CF 31

Thank you! Charitable Fund «Downside Up» +7 (499) 367 -10 -00 www. downsideup. org