Horizon 2020 SME Instrument Experts Briefing Part 2

CONTENT 1. The Evaluation Process; 2. Your role as independent experts: 3. The Individual Evaluation Report; 4. Tips & tricks to complete the IER.

1. The Evaluation Process

Overview of the Evaluation Process for the SME Instrument Receipt of proposals Individual evaluation Eligibility check Remote evaluation Allocation of proposals to evaluators Individual Evaluation Reports (explanatory comments) Automated Consensus Report (arithmetic median/ No negotiation) Standardised Evaluation Summary Report Finalisation Final ranked list Informing the applicants

Admissibility and Eligibility Checks ü Admissibility - proposals must be: ü Readable, Accessible and Printable; ü Complete (all requested forms); ü Eligibility: ü SME status, country; ü Limited number of pages (10 – Phase 1; 30 - Phase 2) Excess pages are watermarked - ignore them; ü Only one application per company allowed for all phases (no concurrent submission or implementation). If you spot an issue relating to eligibility, please inform EASME

Evaluation time line ü Allocation of all proposals right after the cut-off date; ü Accept/Decline task within 24 hrs- otherwise the task will be reallocated to another expert; ü 7 calendar days to complete the evaluations; ü Inform us as early as possible if you face difficulties to meet this deadline; ü Completing the evaluation reports before the deadline is appreciated.

Payment of experts New fee since January 2017 ü Fee = 90 € (or 2 working units) per Phase 1 proposal and 135€ (or 3 working units)/Phase 2 proposal; ü 1 working unit is worth 45€ (0, 1*450€) - 1 working day represents 10 working units (450€) ü Payment is done separately for Phase 1 & 2 and starts at the end of the evaluation process; ü You will receive an email informing you that the payment procedure is starting; ü Please submit your request for payment within the given deadline; ü If you miss claiming your reimbursement within the designated period, we will be unable to do ad-hoc payments and there will be a long delay in payment;

2. Your role as independent expert

Guiding principles ü Independence: evaluating in your personal capacity; ü Impartiality: treating all proposals equally; ü Objectivity: evaluating each proposal as submitted; ü Accuracy: evaluating against the official evaluation criteria; ü Consistency: applying same standards of judgment.

Confidentiality As we are dealing with close-to-market activities, confidentiality is crucial and is the basis for a trusted evaluation. ü Do not discuss evaluation matters; ü Do not contact partners in the consortium, sub-contractors or any third parties; ü Maintain the confidentiality of documents: destroy or delete all confidential documents upon completing your work.

Conflict of interest It is the responsibility of the evaluator to determine whether a conflict of interest exists. Before accepting any evaluation: • Check if you are in a situation of conflict of interest; • If yes, decline the task assigned and justify the reason; • Inform the SME Instrument team. Please note that EASME has the final decision on whether a conflict of interest exists.

Definition of conflict of interest (1/2) Check Article 2 – Annex 1 - Code of conduct of the experts' contract. A conflict of interest arises if an expert: a) was involved in the preparation of the proposal; b) stands to benefit directly or indirectly if the proposal is accepted; c) has a close family or personal relationship with any person representing an applicant legal entity; d) is a director, trustee or partner or is in any way involved in the management of an applicant legal entity;

Definition of conflict of interest (2/2) e) is employed or contracted by one of the applicant legal entities or any named subcontractors; f) is a member of an Advisory Group set up by the Commission to advise on the preparation of EU or Euratom Horizon 2020 Work Programmes, or Work Programmes in an area related to the call for proposals in question; g) is a National Contact Point, or is directly working for the Enterprise Europe Network; h) is a member of a Programme Committee.

In the following situations, EASME will decide whether a COI exists ü Were employed by an applicant or sub-contractor in the last 3 years; ü Were involved in a grant agreement, the membership of management structures or a business collaboration with an applicant in the last 3 years; ü Are in any other situation that casts doubt on your impartiality or that could reasonably appear to do so

Am I in a conflict of interest situation? I have been involved as a consultant/advisor/service provider/applicant preparing a proposal. Yes, if you are evaluating proposals for the SME Instrument whatever the topic. Please note that you may be required to suspend your evaluator activities during the ongoing evaluation.

Am I in a conflict of interest situation? I have been asked to give a presentation on the programme. No, there is no conflict of interest if you speak in general about the Programme. Yes, if you mention the contents/details of a proposal you have evaluated. Can I be an evaluator and a coach at the same time? No, in this combination of roles there is a potential conflict of interest.

3. Individual Evaluation Report

Proposal scoring ü Each evaluation sub-criterion is scored out of 10 points (one decimal may be used); ü Each evaluation sub-criterion question has the same weight, except overall perception that weights 25% of the total score of that criterion; ü The individual scores (from 0 to 10) given to each sub-criterion are automatically converted to a scale from 0 to 5 to calculate each of the three Criterion scores per evaluator. ü The total maximum score for a proposal is 15 points. ü The scale used to obtain the qualitative assessment is the following: ü ü ü Scores from 0 to 2. 99 – generate "Insufficient" Scores from 3 to 4. 99 – generate "Insufficient to Fair" Scores from 5 to 6. 99 – generate "Fair to Good" Scores from 7 to 8. 99 – generate "Good to Very Good" Scores from 9 to 10. 0 – generate "Very Good to Excellent"

The final score ü Each proposal is assessed by 4 evaluators; ü The median score across all 4 evaluators for each criterion ü Overall score is the sum of the 3 criteria median scores.

How is an IER structured? Please note that the formulation of several sub-criteria was modified and the order has changed compared to 2016 PHASE 1 PHASE 2 Operational capacity Impact Excellence Quality and efficiency of implementation Subcontracting Scope of the proposal Threshold: 13 Threshold: 12

Impact PHASE 1 PHASE 2 Threshold: 4 Sub-criteria: 9 Main aspects to tackle in the evaluation: • • • Compliance with the relevant Work Programme; The demand/market of the innovation proposed; The targeted users or user groups; The market conditions; The impact on the growth of the applying company; The commercialisation plan; The European dimension; The IPR filing status and ownership, licensing; The Regulatory and standard requirements. To determine the ranking, the score for the criterion ‘impact’ will be given a weight of 1. 5.

Excellence PHASE 1 PHASE 2 Threshold: 4 Threshold: 3 Sub-criteria: 7 Sub-criteria: 8 Main aspects to tackle in the evaluation: • • • The new market opportunities for EU/global challenges; The current stage of development of the innovation (for Phase 2 TRL 6 or above except SMEInst-05 and phasing out PHC-12); The comparison with known commercial solutions; The objectives and the approach/activities to be developed; The commercial viability of the innovation proposed; The risks and opportunities of the market introduction.

Implementation PHASE 1 PHASE 2 Threshold: 4 Threshold: 3 Sub-criteria: 4 Sub-criteria: 5 Main aspects to tackle in the evaluation: • • The resources to develop the activities; The technical/scientific knowledge/management experience; The time frame and the implementation description; The work packages and major deliverables and milestones. For Phase 2 only: subcontracting

Technological Readiness Level TRL – NEW Phase 2 only The TRL described in the proposal has to be assessed by replying to the following question: Is the project proposed containing activities above TRL 8? TRL 8 corresponds to 'system complete and qualified' (not yet proven in operational environment). The answer is set to 'No' by default in the Individual Evaluation Report Form (IER). If your assessment reveals a TRL>8, switch the radio button to 'Yes'.

Technological Readiness Level TRL – NEW Phase 2 only Please note that it is NOT an evaluation criterion. We ask you to note the level of TRL but being above TRL 8 will not disqualify the proposal. This assessment of TRL is necessary in the framework of the Seal of Excellence*. Potential national funding authorities are informed if TRL 9 activities (already commercialised) are foreseen to avoid that their related costs are considered eligible for funding through other public resources. * The Seal of Excellence is a quality label granted by the EC to proposals submitted under Horizon 2020, which succeeded an independent highly competitive evaluation at EU level but could not be funded due to insufficient call budget. The Seal allows regions, Member States or any other funding sources to easily identify these high quality proposals and possibly support them.

The 'out of scope' option ü Only to be used when a proposal is very clearly submitted in a wrong topic. ü If a proposal is partially relevant to the topic, it should be considered within scope. ü If a proposal is considered not innovative, not disruptive, not well explained, incomplete, etc. , the proposal is still within scope but this opinion should be expressed by means of a lowered score in the relevant (sub) criteria.

The 'out of scope' option As long as there is a link between a proposal and the challenges described in the relevant Work Program, it is IN scope. Examples: • A proposal aiming at developing a technology for conversion of pressurized gas to energy is at TRL 3 instead of TRL 6. The proposal is IN scope! • A proposal concerning an innovative textile for the fashion industry is submitted under the Transport topic. The proposal is OUT of scope!

Operational capacity - Phase 2 only If you believe that an applicant does NOT have the operational capacity to carry out the proposed work, you should choose NO, justify the reason and score the Quality and efficiency of implementation below the threshold (<3).

Operational capacity - Phase 2 only Examples: • The company does not have technical & financial resources to manage the project/subcontractors. NO! • The company will subcontract many tasks but has the resources to manage the core activities. YES! Suggestions: • Check the company's website • Check the team involved in the project

Subcontracting - Phase 2 only • Regulated under Art 13 of the H 2020 SME Instrument Phase 2 Model Grant Agreement (see link - page 551); • Subcontracting is NOT restricted to a limited part of the action; • It is in the SME Instrument spirit that the applying SME has the capacity to carry out the activity; • Compliance with best value-for-money is assessed during the evaluation (not for topic SMEInst-05 -2016 -2017 and phasing out PHC 12).

Subcontracting - Phase 2 only • A table in annex 4 -5 to detail each subcontractor and task subcontracted.

Subcontracting - Phase 2 only For each subcontracted task, there are only two options: • Yes • No or lack of explanation If "no or lack of explanation", experts need to justify and reflect this in the assessment in the Quality & efficiency of implementation criterion (score below the threshold: <3). By default, the task is set to 'yes' even when the proposal does not foresee any subcontracting. If there are no subcontractors in the proposal you shall not change this "yes" set by default.

Health topics ü SMEInst-06 -2017 - Accelerating market introduction of ICT solutions for Health, Well-Being and Ageing Well (same rules as other topics) ü SMEInst-05 -2016 -2017 a) Supporting innovative SMEs in the healthcare biotechnology sector (No phase 1 applications at all in 2017) b) Clinical research for the validation of biomarkers and/or diagnostic medical devices (only at the first cut-off date in 2017 and for phase 2 applications - phasing out of the PHC 12 -2014 -2015 topic introduced in the Work Programme 2014 -2015) Specific Rules for SMEInst-05 • • • TRL indication does not apply Contribution up to € 5 million Funding rate at 100% General model Grant Agreement applies No evaluation of best value for money of subcontracting

4. Tips and tricks to complete the IER

How to write a quality IER? ü Maintain a very high quality standard – do not be too positive; ü Exercise critical judgement - assess the credibility or plausibility of very positive growth figures for instance ü Reflect shortcomings in a lower score for the relevant criterion; ü Provide explanation of shortcomings but do not give recommendations;

ü Remember if you score above threshold, the proposal has a high chance to get funding so think as an investor, would you put money in this proposal ü No grant negotiation phase! − You evaluate each proposal as submitted, not on its potential if certain changes were to be made − Maintain a very high quality standard. Any proposal which does not meet this should not receive above-threshold scores − If you identify shortcomings (other than minor ones and obvious clerical errors), you must reflect those in a lower score for the relevant criterion − You should explain the shortcomings, but do not make recommendations − Proposals with significant shortcomings should not receive above-threshold scores − Any proposal with scores above thresholds and for which there is sufficient budget will be selected as submitted

How to write a quality IER? Yes No Dedicated and thorough comments for each sub-criterion Comments repeating the subcriterion description and/or identical comments for all sub-criteria and/or blank text boxes Consistency between scores and comments Good score but negative comments Coherence between operational capacity/subcontracting and the score of the 'Implementation' criterion Selecting NO for operational capacity/subcontracting and scoring 'Implementation' above threshold (3)

During each cut-off Due to the time constraints your availability and accountability is crucial! • • • Be accurate when you specify your availability; Accept/decline the tasks assigned within 24 hours; Let us know if you can take more evaluations; Stick to the deadlines; Inform us asap if you cannot cope with the workload; First cut-off Phase 2 on the 18 of January 2017 !

Further sources of information ü Horizon 2020 Participant Portal http: //ec. europa. eu/research/participants/portal/ ü EASME website https: //ec. europa. eu/easme/en/horizons-2020 -sme-instrument ü FAQ: http: //ec. europa. eu/easme/sites/easme-site/files/faq 4 expertssme_instrument_january 2017. pdf ü Functional mail box for all your questions and comments EASME-EXPERTS@ec. europa. eu ü Project Officer responsible for each topic; firstname. familyname@ec. europa. eu