EU external action at your fingertips OPSYS Webinar

EU external action at your fingertips OPSYS Webinar to present the feedback from the 2 nd test phase with Implementing Partners April 17 th, 2018

Welcome to the OPSYS webinar to launch the test phase Who is running the webinar ? Michal Krejza Véronique Lena Lucile Petitpierre Guido Bilanceri 2

EU external action at your fingertips OPSYS Introduction to the webinar Michal Krejza, Head of Unit, Results and Business Processes, DEVCO

Agenda of the webinar Agenda: Feedback from the 2 nd test phase with Implementing Partners Introduction to the Webinar Michal Krejza, Head of Unit, Results and Business Processes, DEVCO Feedback from the test phase: key elements Véronique Lena, Team Leader, Change management What’s next in Results and Monitoring: releases and functionalities Lucile Petitpierre, Business Manager Results and Monitoring Next steps and conclusion Michal Krejza, Head of Unit, Results and Business Processes, DEVCO 4

Introduction to the webinar Objectives of the first test phase with Implementing Partners: • Introduce OPSYS to external stakeholders • Demonstrate the interactions between the IP and the OM: OM provides access to IP, IP submits the logframe to the OM, the OM approves the logframe • Show to which extent the logframe will be central for project monitoring and results reporting 5

Feedback: number of participants Users EC Staff 59 Implementing Partners 57 DEVCO NEAR FPI 38 13 7 Plus 12 phone interviews with Implementing Partners 6

EU external action at your fingertips OPSYS Feedback from the test phase: key elements Véronique Lena, Change Management, Results and Business Processes, DEVCO

Feedback: number of participants Users DEVCO NEAR FPI EC Staff 59 Implementing Partners 57 38 13 7 European Union Delegation Headquarters NGO Consulting company Other (UN, GIZ, universities, etc. ) 39 19 Policy/ Thematic unit Geographic unit Finance and Contracts Other horizontal unit Cooperation/ Operation Finance and Contracts Other Started 35 2 2 7 5 1 5 At HQ In the field Home-based Other Plus 12 phone interviews with Implementing Partners 28 19 10 31 17 4 5 In a EU country In a NEAR country In a DEVCO country Other 27 4 16 10 8

Feedback: could you do it? Test Case – Operational Manager 1. Access OPSYS and provide access to the Lead IP 4. Approve logframe Yes, but… No 76% 17% 6% 68% 32% Yes, but… No 2. Access OPSYS (Results Portal) 63% 35% 2% 3. Access, edit and submit logframe for approval 43% 40% 18% 5. Add current values to the logframe 52% 24% 8. Receive notification of removal 44% 19% 37% Test Case – Implementing Partner 0% 6. View current values added by the Lead IP 71% 21% 7% 7. Revoke access to the Lead IP 93% 7% 0% 9. Access and edit logframe 55% 45% 0% 10. Add values to the logframe 60% 40% 0% 9

Feedback: main difficulties & answers (IP) General • I don’t have access!: many IPs could not start the test because OM had not provided them access. We will make sure to make it clearer next time! • • I’m lost in the system! How do you rate the navigation in OPSYS/Results Portal? Very clear and user friendly 56% Average 32% Confusing and not user friendly 12% My connection is slow and it took me ages to do the testing: one of the so-called Non functional requirements for OPSYS is to be able to work with slow connections. • I was using Internet Explorer and it was slow and bugging: unfortunately, the performance in IE is still unsatisfactory at this moment. In the future, OPSYS will be able to work on all browsers. How fast did OPSYS/Results portal run Very fast 4% Fast 40% Average 36% Slow 4% Very slow 16% Which browser did you use? Mozilla Firefox Google Chrome Internet Explorer Safari 9% 14% 6% 0% Edge 8% 10

Feedback: main difficulties & answers (All) Logframe encoding • I could not edit results and indicators: it will be possible to edit all fields, including name and indicator’s type. • I could not understand the order of my results and indicators: the logframe is ordered by level (Overall Objective - Impact, Specific Objective - Outcome, Output) and within each group by date of creation. In the next releases, it will be possible to manually re-order results and indicators and to number them. • I need other types of disaggregation!: more options for disaggregation will be available. • I couldn’t copy or duplicate indicators: the indicators you create will be stored in the system and you will be able to re-use them. • I tried to submit but I had an «Invalid logframe» message: bug! We are sorry about this: the system will clearly list all the issues in your logframe 11

Feedback: main difficulties & answers (All) List of suggested indicators • What I am supposed to do with this list? : the objective is that users will use more and more the suggested indicators (SMART, well formulated, linked to DAC codes) instead of creating new ones of low quality. • My sector’s indicators were not there / I could not find any relevant indicator: for this test phase, we uploaded only four sector lists for DEVCO and corporate indicators for DEVCO, NEAR and FPI. In the future, all sectors will be covered and the lists will be regularly updated by the relevant units. • What about SDGs? : SDGs will also be in the list of suggested indicators. • What about national results framework indicators? : this is one of potential lists that can be included, to be discussed. 12

Feedback: list of suggested indicators Implementing Partners Strongly agree Somewhat Neither agree Somewhat agree or disagree Strongly disagree These core indicators are appropriate for my sector 9% 31% 38% 16% 6% These core indicators are clear and well formulated 16% 41% 31% 9% 3% I will surely use these core indicators when I will create the logframe of my future project 16% 39% 23% 19% 3% It was easy to find and select the core indicators relevant for me 4% 25% 29% 14% 29% They are similar to the indicators I have used in my projects 6% 19% 31% 25% 13

Feedback: main difficulties & answers (All) Business process • I don’t understand the different approval steps: The logframe approval is done in two steps: 1. STRUCTURE + BASELINE AND TARGET: first the OM has to approve the logframe structure (results and indicator). In the future, the first step will be carried during the contracting phase with minor adjustments during implementation. 2. CURRENT VALUES: after the approval, the IP can add current (actual) values to an indicator. The indicator’s page is now available. • But I had already approved the logframe! Why is it «draft» again? : every time you modify the structure (i. e. results and indicators) of the logframe, its status changes back to draft. In this case, you won’t be able to add current (actual) values until the logframe is approved again. 14

Feedback: elements to improve the logframe (IP) Could you understand all the elements of your logframe? Yes, everything was clear It took me a while to understand where to look I could not understand how the information was displayed 62% 31% 8% Was the logframe correctly imported? Yes, all information was correct Partly, a part of the information was missing or incorrect No, the largest part of the information was missing or incorrect I could not understand where to find the information How often do communicate on the progress of your intervention? Only with the progress report During a key event of the intervention (PSC, field mission, HQ mission, etc) On formal demand Very often, mostly informally (phone calls, emails, etc) *multiple choice How do you monitor the progress of your intervention? Current value versus Final target 44% Current value versus Intermediary target 4% Current value versus previous value 8% Current value versus baseline 20% Other 24% 44% 56% 40% 48% 44% 16% 20% All these elements will help us to improve the logframe and the related processes: submission mechanisms, data presentation, etc. Do you have intermediary targets? Always 12% Often 20% Rarely 44% Never 24% 15

Feedback: elements from IPs Who in your organization is in charge of reporting (including current values) on the logframe? The team leader and/or the experts in the field 69% The backstopping team of the consulting company / NGO central office 12% Other 19% In the testing, there was no mechanism to submit values. Your feedback will help us develop the review values mechanism Do you include the logframe in your progress report? Yes 64% No 24% Other 12% How do you plan and monitor your activities? In the logframe 8% In a separate workplan 60% Both 32% For the testing, there was a simplified mechanism to provide access and manage IP. Cascade of access rights from lead contractors to other contractors (ex. consortium or team leader) will be analysed How often would report on your indicators? Whenever the data is available (i. e. as soon as you achieve a results) Only when submitting your progress report Other 23% 54% 23% Your feedback will help develop the monitoring functionalities, including the submission of progress reports and the management of activities 16

Feedback: Most voted option for Logframe view (All) In the next releases, there will be more options for logframe view. This is the option that was most voted for logframe view. We will take into consideration all your comments also for the other options! 17

Feedback: Most voted option for Logframe export (All) In the next releases, it will be possible to export the logframe. This is the option that was most voted for logframe export. We will take into consideration all your comments also for the other options! 18

Feedback: help and support • General positive feedback on the e-learning videos, more videos will be produced. • Integrated guidance: the system will include integrated guidance both methodological ( «what’s a baseline? » ) and technical ( «where do I have to click? » ) • Languages: video and instructions will be produced also in French for the releases • Integrated chat between OM and IP is under analysis 19

Get the latest news Please do not hesitate to contact us at the address : Europe. Aid-OPSYSUSER-SUPPORT-COMMUNICATION@ec. europa. eu If you want to be kept updated, feel free to join us on capacity 4 Dev group: https: //europa. eu/capacity 4 dev/implemeting-partners---opsys 20

EU external action at your fingertips OPSYS What’s next in Results and Monitoring: releases and functionalities Lucile Petitpierre, BM Track 1, Results and Business Processes, DEVCO

Next steps for Results and Monitoring OMs access their interventions OMs create and update, print Logframes OMs can report on indicators values (disaggregated by sex when relevant) Traffic lights are generated OMs grant access to Implementing partners (IPs) OMs can review Logframes and values encoded by IPs A notification mechanism and task manager is in place Existing Logframes are imported into the system A user guidance and support is in place OM/IP report on a periodic basis (by quarter, by year) Additional disaggregation can be generated for each indicator Feedback from pilot phase is incorporated Quality Manager manage a list of core indicators Quality Managers perform ad-hoc reviews of encoded Logframes Indicators are aggregable Budget support specificities are covered (variable tranche) Users can upload documents IPs submit their narrative reports Results dashboarding are made available Logframes for Basic Act and Indicative programming can be created Other modalities specificities are taken into account Narrative reports and Logframes are linked to contracts and invoices Additional monitoring features are available (risk, calendar, etc. ) Link with ROM and EVAL module is implemented Link with EAMR is implemented Release 1 Intermediary release Release 2 Other Releases

Next steps for Results and Monitoring OMs access their interventions OMs create and update, print Logframes, OMs can report on indicators values (disaggregated by sex when relevant) Traffic lights are generated OMs grant access to Implementing partners (IPs) OMs can review Logframes and values encoded by IPs A notification mechanisms and task manager is in place Existing Logframes are imported into the system A user guidance and support is in place OM/IP report on a periodic basis (by quarter, by year) Additional disaggregation can be generated for each indicator Feedback from pilot phase is incorporated Quality Manager manage a list of core indicators Quality Managers perform ad-hoc reviews of encoded Logframes Indicators are aggregable Budget support specificities are covered (variable tranche) Users can upload documents IPs submit their narrative reports Results dashboarding are made available Logframes for Basic Act and Indicative programming can be created Other modalities specificities are taken into account Narrative reports and Logframes are linked to contracts and invoices Additional monitoring features are available (risk, calendar, etc. ) Link with ROM and EVAL module is implemented Link with EAMR is implemented Release 1 Intermediary release Release 2 Other Releases

Next steps for Results and Monitoring OMs access their interventions OMs create and update, print Logframes, OMs can report on indicators values (disaggregated by sex when relevant) Traffic lights are generated OMs grant access to Implementing partners (IPs) OMs can review Logframes and values encoded by IPs A notification mechanisms and task manager is in place Existing Logframes are imported into the system A user guidance and support is in place OM/IP report on a periodic basis (by quarter, by year) Additional disaggregation can be generated for each indicator Feedback from pilot phase is incorporated Quality Manager manage a list of core indicators Quality Managers perform ad-hoc reviews of encoded Logframes Indicators are aggregable Budget support specificities are covered (variable tranche) Users can upload documents IPs submit their narrative reports Results dashboarding are made available Logframes for Basic Act and Indicative programming can be created Other modalities specificities are taken into account Narrative reports and Logframes are linked to contracts and invoices Additional monitoring features are available (risk, calendar, etc. ) Link with ROM and EVAL module is implemented Link with EAMR is implemented Release 1 Intermediary release Release 2 Other Releases

Next steps for Results and Monitoring OMs access their interventions OMs create and update, print Logframes, OMs can report on indicators values (disaggregated by sex when relevant) Traffic lights are generated OMs grant access to Implementing partners (IPs) OMs can review Logframes and values encoded by IPs A notification mechanisms and task manager is in place Existing Logframes are imported into the system A user guidance and support is in place OM/IP report on a periodic basis (by quarter, by year) Additional disaggregation can be generated for each indicator Feedback from pilot phase is incorporated Quality Manager manage a list of core indicators Quality Managers perform ad-hoc reviews of encoded Logframes Indicators are aggregable Budget support specificities are covered (variable tranche) Users can upload documents IPs submit their narrative reports Results dashboarding are made available Logframes for Basic Act and Indicative programming can be created Specificities of other modalities are taken into account Narrative reports and Logframes are linked to contracts and invoices Additional monitoring features are available (risk, calendar, etc. ) Link with ROM and EVAL module is implemented Link with EAMR is implemented Release 1 Intermediary release Release 2 Other Releases

EU external action at your fingertips OPSYS Next steps and conclusion Michal Krejza, Head of Unit, Results and Business Processes, DEVCO