Sigried Caspar European Commission DG Employment Social Affairs

Sigried Caspar European Commission, DG Employment & Social Affairs Moderator

Workshop A 7: how to evaluate your communication activities 2014 -2020 December 10, 2013 – Brussels Céline Mas Occurrence est certifiée ISO 9001 depuis 2004

"Half the money I spend on advertising is wasted; the trouble is I don’t know which half. " John Wanamaker Inventor of mass retailing in the United States Occurrence - Workshop A 7 - 10/12/2013

Key issue > Communications are under pressure: - How much does it cost? How much does it contribute? € > View it as an investment and not as an expense What does it contribute? > Provide the resources to prove your effectiveness: 5% to evaluation. > "How much does overlooked inefficiency cost? “ : if you can not assess it, you can not improve it! Occurrence - Workshop A 7 - 10/12/2013 4

The missing link in a virtuous circle Communication plan Most of the time, evaluation is occasional or partial. A goals and resources « contract » with institution or company management It rarely shapes dialogue between the communications team and the other decision makers. Assessment Reporting on the achievement of the targets, or on the progress and effectiveness of the implementation Occurrence - Workshop A 7 - 10/12/2013 Actions A goals and resources « contract » with company management

Behaving like any other function > Communications must not exclude itself from Quality and Operating Excellence systems > Communications is a job and a skill; it must include ongoing improvement procedures The Deming Wheel Quality management system Occurrence - Workshop A 7 - 10/12/2013 6 i ing o g t On men ve o r mp

The 4 main benefits of an evaluation 1. PROMOTING 2. MANAGING • Circulating results and performances to other departments/teams • Allocating resources in accordance with performance indicators • Achieving investment choices that are based on targets, and not on expenditures on resources • Identifying the most effective initiatives for achieving your various objectives 4. SHARING 3. SAVING TIME • Prioritizing/Sorting initiatives by order of effectiveness • • Gathering all the activity and effectiveness data Concentrating your efforts and budget on the most effective initiatives • Highlighting best practices Occurrence - Workshop A 7 - 10/12/2013 • AND SPECIFIC TO THE PUBLIC SECTOR Giving evidence of a sound use of public money • Reinforce citizen’s trust 7

2 main categories of performance (KPI) indicators Activity including Resources (What? ) Effectivenes s including the Audiences (For what purpose? ) e. g. Number of initiatives/tools, type of initiatives/tools, assessment of the content issued (Press releases, and internal communications), Ressources: Who? How much? How long? e. g. Memorization, Understanding, Buy-in, Incentive, Transformation, Satisfaction, Improving the brand's image, and satisfying internal customers Audience: How many people attended? How many Likes ? How many readers? … > A third, highly practical approach is possible: assessing the satisfaction of (internal) customers > Defining performance thresholds for each indicator Occurrence - Workshop A 7 - 10/12/2013 8

Communications Dashboard communications plan Process : global picture Target audience Communications objectives Opinions/perceptions to share Segmentation 1. Define 2. Count 3. Contact details Do you agree with the following assertion ? Surveys, research - "ABC respects environment” media analysis, - … Ye observations Indicators - - "ABC respects environment” 60% Current Targeted score: score : 60 % 90 % Occurrence - Workshop A 7 - 10/12/2013 x 1 [File qualification] x N [Sample survey] Gap between results and objectives !! ! OK s Yes : No : Don’t know : èManagement • • • Allocate Understand Adjust Maintain Etc…

Advice > Efficiency means producing the desired effects on the desired target audiences > Therefore, you need to define the target audiences that you want to reach with which effect, prior to the initiative, and ideally to define the performance threshold > As a starting point: KISS (Keep It Simple, Stupid) - Keep things simple at the beginning > BUT assess them regularly > And don't change the assessment system for each evaluation > Do not restrict the evaluation to the activity, in order not to limit communications to initiatives and tools > Design SMART (Specific, Measurable, Achievable, Realistic, and Timed) tools and goals for each initiative > Share the results and the decisions they help to take in order to enhance the value added of evaluation Occurrence - Workshop A 7 - 10/12/2013 10

Thank you > Contact : Céline Mas > Partner & Research Director Occurrence > celine. mas@occurrence. fr Occurrence - Workshop A 7 - 10/12/2013

Brussels, 9 -10 December

Workshop B 7: How to evaluate communication activities 2014 -2020 UK Government perspective December 9, 2013 – Brussels Paul Njoku Cabinet Office, UK

Agenda A • Why evaluate (context)? • Barriers to evaluation B • How to go about it - PROOF guiding principles • 4 stage evaluation process - The Big IDIA • Main performance categories - KPI indicators C • Top tips Brussels, 9 -10 December

A 1 Why evaluate? 16

A 2 • Context Austere times Ø Ø • Media landscape & consumption patters Ø Ø • Need to make every € count View as an investment not an expense Evidence of what works and what does not Optimise use of scarce resources The role of communications Ø How it supports achievement of policy outcomes Ø Business planning & activity prioritisation Brussels, 9 -10 December

A 3 Strategic alignment DEPARTMENTAL OBJECTIVE To address specific issue Policy development, policy delivery, reputation management COMMUNICATION OBJECTIVE Role that communication will play in achieving departmental objective Overall communication strategy - how communication will achieve its objective Sub-objective Communication plan Specific activities, channels, target audiences

A 4 Barriers – stopping it happening Lack of SMART or unrealistic policy objectives / targets Culture & entrenched behaviours Insufficient time / resource / budget Difficulty accessing the right data / tools Gaps in evaluation standards & capability 19

B 1 How to go about it Five key principles P Pragmatic – best available within budget, not best ever R Realistic – prove what you can, acknowledge what you can’t O Open – record and share as much as possible O Objective – be honest & constructive about results, to inform future learning F Fully integrated – integral part of planning & delivery, not an add-on Brussels, 9 -10 December

B 2 1 Evaluation stages –The Big IDIA Identify The scope of your project 2 Develop Your evaluation plan 3 Implement Gather data to measure performance 4 Analyse & report Performance against plan Task 1: Define what you need to evaluate by asking: • What activity am I evaluating? • What do I know & what factors could affect the outcome? • What is my evaluation expected to achieve? Output: Summary of your proposed evaluation approach Task 2: Define how you’ll measure success: • Set SMART objectives & defining your target audience • Map out how activity will work • Set performance metrics (KPIs) & agreeing targets Output: Draft evaluation plan Task 3: Identify and gather evaluation data: • Make most of existing data • Gather additional data (research, feedback & proxies) • Review data gaps (more budget ? ) manage expectations Output: Completed evaluation plan Task 4: Assess the success of your activity: • Analyse effectiveness & provide insights for future • Demonstrating efficiency and value for money • Demonstrating role of communications in supporting the achievement policy objective (outcome) Brussels, 9 -10 of. December Output: Final evaluation report

B 3 Key performance indicator categories Activity Effectiveness Result Brussels, 9 -10 December

Top tips 1. Strategic alignment – Ensure activity objectives are SMART and supports policy delivery. 2. Business impact – Aim to measure true business impact (outcome) rather than for example, the perceived quality of specific channels. 3. PROOF the big IDIA – Try to adopt the suggested guiding principles and follow the big IDIA stages. 4. Continuous improvement – Ensure results drive appropriate actions and any learnings inform future activities. 5. Best practice – Be objective, share results and make evaluation an integral part of your communications planning process. Brussels, 9 -10 December

Thank you! Contact: Paul Njoku Email : paul. njoku@cabinet-office. gsi. gov. uk Web link to guide: https: //gcn. civilservice. gov. uk/guidance/evaluation/ 24

Appendix Brussels, 9 -10 December

How reputation is created Direct Experience Products Investments Customer Service Employment What your department Says/Does Branding Marketing Public Relations Social Responsibility What Others Say MEDIA (Traditional, Social) Topic Experts, Leaders, Friends/Family Perceptions & expectations Supportive Behaviour Results

Reputation drivers & dimensions Reputation Strength A measure of the emotional connection. Reputation Dimensions The seven dimensions specify at a more operational level, which aspects are most important for stakeholders’ perceptions and expectations – i. e. what’s driving a company’s reputation Reputation Attributes The attributes and dimensions have different meanings and importance for different stakeholders. Beneath the 7 dimensions, 32 attributes underpin the individual dimension themes. Different stakeholder groups typically have unique attributes that are found more important than others (reputation drivers). Supportive Behaviour Reputation has a positive/negative impact on support. An increase in reputation = an increase in support. Support (such as buying products and services, saying something positive, giving the benefit of doubt in times of crisis (etc. ) leads to increased business results

Brussels, 9 -10 December