LESSONS FROM EVALUATIONS UNDP SUPPORT TO DIGITALIZATION IN

LESSONS FROM EVALUATIONS: UNDP SUPPORT TO DIGITALIZATION IN CRISIS COUNTRIES 1

Reflections - IEO’s series of knowledge products! § Look into past evaluations and capture lessons learned from UNDP’s work across its programmes. § Provide evaluative evidence for improved decisionmaking and better development results. § The first edition of these papers highlights lessons from evaluations of UNDP’s work in crisis settings. 2

Methodology § Rapid evidence assessment § Independent country-level, thematic evaluations and high-quality decentralized evaluations commissioned by the country offices considered § Emphasis on identifying relevant lessons for UNDP § The analysis seeks to offer practical and timely insights to support UNDP’s decision-makers for effective crisis response Context – accelerated reliance globally on digital technology due to COVID-19 – fast-tracked UNDP Digital Strategy (2019), including to address the digital divide 3

At a glance – Lessons learned 1 4 Noteworthy digital initiatives in crisis countries remain undocumented and unassessed. Digital transformation is about sourcing, connecting and developing talent – more than it is about technology. 7 2 Supporting connectivity and open access is good. Supporting ‘meaningful connectivity’ is better. 5 Successful digitalization takes time and must carefully consider local readiness and culture. Mind the gap. The ‘digital divide’ is real and growing. 8 3 6 Collaborating with the private sector is a ‘must’, provided ethical dilemmas can be resolved. Digitalization benefits from risk taking and collaborating with local digital disrupters. Acting on data insights is key, especially for decisions around scale-up and replication. 4

Lesson 1. Noteworthy digital initiatives in crisis countries remain undocumented and unassessed. § Digital solutions found across focus areas, mainly development (e-governance, digital health, etc. ) § Crisis: biometric equipment for voter registration; digital finance to transfer salaries; early warning systems and climate monitoring § Crisis: Mainly digitization (efficiency gains), not digitalization (quality improvement) Examples: Angola, Afghanistan Cuba, Equatorial Sierra Leone Guinea, Guinea-Bissau and Central African Republic Zimbabwe Barbados Liberia § Pilots: drones (Mali, Syria), blockchain (Lebanon), etc. – typically not fully evaluated 5

Lesson 2. Supporting connectivity and open access is good. Supporting ‘meaningful connectivity’ is better. Examples: § Critical infrastructure is needed for the roll-out, scaling and sustainability of digitalization efforts, e. g. mobile phone network, Internet § Key roles for both government and private sector Examples: Angola, Kazakhstan Cuba, Equatorial Tanzania Guinea, Guinea Zambia Bissau and Malawi Zimbabwe Ethiopia Bhutan § Connectivity is meaningful when digital literacy and skills are widespread, an enabling environment for innovation exists, and digital solutions are inclusive and equitable 6

Lesson 3. Collaborating with the private sector is a ‘must’, provided ethical dilemmas can be resolved. § The private sector usually owns networks and data so plays a pivotal role regarding the coverage, reliability and quality of digital connectivity § Private sector collaboration/leveraging are a key success factors for the roll-out and scaling-up of national digitalization efforts Examples: Angola, Sierra Leone Cuba, Equatorial Bangladesh Guinea, Guinea Pacific Island Countries Bissau and Zimbabwe Kyrgyzstan § Ethical dilemmas must be tackled, going beyond privacy and data protection, to include human dignity and autonomy; discrimination and unequal balance of power 7

Lesson 4. Digital transformation is about sourcing, connecting and developing talent – more than it is about technology. Examples: § Government ability to adapt to an even more digital future depends on developing the next generation of skills § Digital transformation initiatives which invest in capabilities to effectively use, manage, and maintain associated technology are more likely to succeed Moldova Examples: Angola, CIS region Cuba, Equatorial Guinea, Guinea Papua New Guinea Bissau and Jordan Zimbabwe Haiti Nepal § UNDP’s ability to provide a bridge between local challenges and global expertise is a key asset (sourcing of external/inhouse experts) – gives UNDP an ‘in’ to influence the design of digital solutions that are inclusive and based on international best practice 8

Lesson 5. Successful digitalization takes time and must carefully consider local readiness and culture. § Trade-off between speed and quality: more time, effort and capacities needed at design stage to include a digital component into a programme – often efficiency gains in the longer-terms § Digital solutions need to be introduced in line with country capacities and readiness, and respecting local cultures; more successful if in partnership with local actors, and as part of larger initiatives Examples: Angola, Sierra Leone Cuba, Equatorial Guinea Bissau Guinea, Guinea Bangladesh Bissau and Argentina Zimbabwe Jordan Uganda § “Flying in” and testing technologies not yet used in highly developed countries tends to be problematic 9

Lesson 6. Digitalization benefits from risk taking and collaborating with local digital disrupters. § UNDP can support governments in risk-taking, e. g. exposing themselves to citizen scrutiny, and beginning to see failure as a strategy for achieving success in the long run § Importance of tolerating quick pilots for learning purposes, even if prone to failure (“failing fast”) Examples: Angola, Kazakhstan Cuba, Equatorial Serbia Guinea, Guinea Honduras Bissau and Bosnia and Herzegovina Zimbabwe Afghanistan Rwanda India § Work with local digital disrupters to demonstrate the potential of homegrown innovation and the power of youth to shape their own digital future 10

Lesson 7. Mind the gap. The ‘digital divide’ is real and growing. § Access to digital solutions remains uneven: typically reserved for younger, more affluent persons living in cities, without disabilities, and male (anglophone helps) § Digitalization can make indirect contributions to the empowerment of women and girls or exclude them further Examples: Barbados Examples: Angola, Uganda Cuba, Equatorial Egypt Guinea, Guinea Bissau and Afghanistan Zimbabwe Papua New Guinea India Bahrain § To close the digital divide, or at least not exacerbate it, successful programmes o explicitly included the most vulnerable as part of design o produced materials in accessible formats o used inclusive language, including in terms of gender 11

Lesson 8. Acting on data insights is key, especially for decisions around scale-up and replication. § Scale-up and replication hinge on documentation and assessment, incl. measurement of unintended effects § Good M&E practices include: o Measuring intermediate results, e. g. of rapid prototyping, along the results chain/theory of change o Systematically identifying challenges and adverse effects o Differentiating between results achieved by UNDP and results of larger (e. g. country-led) efforts § Supporting scale-up and replication decisions (of often very recent or ongoing digital initiatives) through evaluation needs new tools Examples: CIS region Kyrgyzstan Turkey Serbia Bangladesh 12

Thank You @undp_evaluation @ieoundp http: //web. undp. org/evaluation/reflection s