EVENT EVALUATION Powerpoint Figures from the book Getz

EVENT EVALUATION: Powerpoint Figures from the book (Getz, 2018) • • • • Figure 3. 1 CIPP Applied to a Charity Event Figure 3. 2 Evaluation Complexity Model Figure 3. 3 A Systems Model Figure 4. 1: Major Considerations in the Evaluation Process Figure 5. 4 Logic Model Illustrated Figure 6. 1 Modified Balanced Scorecard Concept for Event Management and Event Tourism Figure 6. 2 Generic Strategy Map Structure Figure 7. 1 Overcoming Tunnel Vision Figure 7. 2 The Event Compass - Concept Figure 7. 3 Sample Event Compass Radar Graph Figure 8. 1 A Framework for Evaluating the Event Organisation Figure 8. 12 Root-Cause and Impact Forecasting Combined Figure 11. 5 Importance-Performance Illustration Figure 12. 4 A Hierarchical Model For Evaluating Training Effectiveness © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

• GOALS CIPP Applied to a Charity Event • Raise money for charity • Foster community development • Attract tourists CONTEXT EVALUATION INPUT EVALUATION • PLANS • Stakeholder input • Resources available • Market intelligence • Past experience • The concept; theme CORE VALUES • OUTCOMES PRODUCT EVALUATION PROCESS EVALUATION • ACTIONS • Produce a 1 -day food event • Generate sponsorship revenue • Market to special© Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, interest food Goodfellow Publishers Ltd, Oxford, UK tourists • X money raised • Satisfied customers • Enhanced community selfreliance

Evaluation Complexity Model Evaluating Long-term, Cumulative Impacts & Sustainability Determining The Worth Of Events, Portfolios, Policies POLITICAL COMPLEXITY Proving Cause & Effect Full Cost / Benefit Evaluation Comprehensive Impact Assessment Forecasting Impacts Evaluating Organisational Effectiveness and Efficiency Evaluating Goal Attainment; Summative evaluation Evaluating Experience & Quality From Multiple Stakeholder Perspectives Evaluating Performance (HR) Discrepancy Identification and Correction; Process Evaluation Problem Solving & Supporting Decisions With Evidence; Formative Evaluation Permanent Data Collection & Monitoring © Donald Getz 2018 Event Evaluation: THEORETICAL Theory and Methods for event management and tourism, & TECHNICAL COMPLEXITY Goodfellow Publishers Ltd, Oxford, UK

A Systems Model ENVIRONMENT COMMUNITY CONTEXT Internal functions INPUTS Internal functions TRANSFORMING PROCESSES OUTPUTS -producing an event -managing event portfolios INTERNAL EVALUATIONS EXTERNAL EVALUATION BY STAKEHOLDERS © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Major Considerations in the Evaluation Process PARADIGMS & THEORIES -Positivism -Interpretive -Critical/Emancipatory EVALUATION CONTEXT; WHY EVALUATE? e. g. , formal vs informal -Internal vs external -One-time vs periodic -Decision support or determining worth WHAT TO EVALUATE METHODS AND MEASURES USES OF EVALUATION -Formative, -Process, -Summative -Qualitative and Quantitative -Indicators and KPIs -The TOOLBOX -Uses and Misuses -Ensuring relevance and utility © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Logic Model Illustrated for goal-attainment evaluation: single event & event portfolios Single events do not usually pursue enduring change Process for Single Events INPUTS -mandate -resources -venues -information PURPOSE & GOALS PLANNED ACTIONS (the event as a transforming process) EXPECTED OUTPUTS -e. g. attendance, tourists, money raised, satisfaction DESIRED IMPACTS; CHANGE Special Concerns for Event-Tourism Portfolios Portfolio Strategy; Long-term Investment -Overall ROI -Bidding -Growth in -Economic and -Sustainability (one-time events) tourism yield community growth -Integration with -Owning/producing -Leveraging -Competitive © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, other policy fields -Supporting events effects advantages Goodfellow Publishers Ltd, Oxford, UK

Modified Balanced Scorecard Concept for Event Management and Event Tourism ORGANISATIONAL CULTURE -enhancing capacity to learn, adapt, grow -evaluation system -continuous improvement CUSTOMER & STAKEHOLDER ORIENTATION -satisfying guests/customers -satisfying regulators, suppliers, facilitators, partners and residents MANDATE, VISION, STRATEGY, GOALS, KPIs, EVALUATIONS SUSTAINABILITY -financial/economic -social/cultural -environmental INTERNAL OR TRANSFORMING PROCESSES -all management functions -event production -managed event portfolios © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Generic Strategy Map Structure DESTINATION STATEMENT: Mandate, Vision, Core Values OUTCOMES/IMPACTS with Key Performance Indicators VALUE to be created for residents and other stakeholders: Economic, Ecological, Built Environment, Social, Cultural ACTION PLAN: The transforming processes that create value, each linked to desired outcomes ENABLERS: Organisational Culture (Learning and Growth); Internal Processes; Customers/Stakeholders; Finances/Stewardship/Sustainability © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Overcoming Tunnel Vision Did we attain our goals? Systematic analysis of inputs, processes, outputs Use theory or logic models Identify unintended and undesired consequences What EXTERNALITIES can be attributed to our event/actions? Engage stakeholders on: -Pollution -Costs borne by others? -Cumulative impacts © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

The Event Compass - Concept © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Sample Event Compass Radar Graph ORGANISATION MARKETIN G RISK SOCIOCULTURAL OUTCOME S 120 100 80 60 40 20 0 PLANNING DESIGN ENVIRONMENT AL OUTOMES ECONOMIC OUTCOME © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, S Goodfellow Publishers Ltd, Oxford, UK

A Framework for Evaluating The Event Organisation OWNERSHIP MANDATE GOVERNANCE BUSINESS MODEL ADMINISTRATION Conformity To Standards Adequacy Of Support Services INFORMATION & KNOWLEDGE HUMAN RESOURCES PLANNING Project, Strategic, Business, Marketing, Site, Operations DESIGN EVALUATING THE EVENT ORGANISATION PROFESSIONALISM ETHICS FINANCES CONTROL RISK AND SECURITY MARKETING COMMUNICATIONS SUSTAINABILITY GREEN OPERATIONS SOCIAL RESPONSIBILITY © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Root-Cause and Impact Forecasting Combined PEOPLE PLACE PRODUCT SOCIAL IMPACTS PROGRAMMING ECONOMIC IMPACTS ENVIRONMENTAL IMPACTS Training CAUSES? Lack of Sponsors PRICE PACKAGING; PROMOTION; PARTNERSHIPS COMMUNICATIONS DISTRIBUTION PROBLEM FINANCIAL (LOSSES? ) CONSEQUENCES? REPUTATION (DAMAGED? ) PARTNERS (COMMITMENT? ) © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Importance-Performance Illustration CONCENTRATE HERE KEEP UP THE GOOD WORK Alternative to grand means Grand Means of Scores LOW PRIORITY POSSIBLE OVERKILL Increasing Importance © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Increasing Performance Goodfellow Publishers Ltd, Oxford, UK

A HIERARCHICAL MODEL FOR EVALUATING TRAINING EFFECTIVENESS GOAL: MEASURE POSITIVE RESULTS (ROI; ROO; ROE) GOAL: TRANSFER LEARNING TO THE EVENT; NEW BEHAVIOURS GOAL: LEARN SKILLS & KNOWLEDGE; CHANGE ATTITUDES GOAL: POSITIVE REACTIONS TO THE TRAINING EXPERIENCE Source: Adapted from Kirkpatrick (2007) and tourism, © Donald Getz 2018 Event Evaluation: Theory andand Methods for event management Goodfellow Publishers Ltd, Oxford, UK