OECD Space Forum Evaluation Impact Assessment Claire Jolly

OECD Space Forum: Evaluation / Impact Assessment Claire Jolly Head, OECD Space Forum Science and Technology Policy Division Directorate for Science, Technology and Innovation Assessing the Impact of Satellite EO on Society and Policy Organised by NICT, Keio Univ. , Hosei Univ. , JAXA 9 November 2015 - Hitotsubashi Hall, Tokyo Japan

The Organisation for Economic Co-operation and Development (OECD) - Fast facts The OECD is an international economic organisation. • A global forum in which governments representatives work together to address the economic, social and environmental challenges of interdependence and globalisation • A source of economic statistics - provider of comparative data, analysis and forecasts to underpin multilateral cooperation (www. oecd. org) Ø No vested interest in the space sector. . . But a recognised expertise in the “economics” of space programmes and activities 2

The Organisation for Economic Co-operation and Development (OECD) - Fast facts • Established: 1961 • Headquarters: Paris • OECD Centres: Washington, Mexico City, Tokyo, Berlin • Members: 34 • Secretary-General: Angel Gurría • Secretariat staff: 2 500 • Nearly 300 expert committees and working groups with participation of +100 countries (all sectors of the global economy covered) 3

The OECD Space Forum Background: IFP space project (2002 -05) led to strong demand to set up an international platform hosted in OECD, complementary to other structures (ITU, UNCOPUOS…). Missions: Forum established to better identify statistically the sector, and investigate its economic dimensions as an infrastructure for the larger economy. • • Development of comparable statistics and indicators, exchange lessons learned and best practices (industry associations, NSOs…) Assess the contribution of space technologies to addressing major challenges the world faces (e. g. environment) including socioeconomic impacts 4

Participation in the OECD Space Forum q The Space Forum activities are managed by the OECD Secretariat, with analytical inputs from the Forum’s Steering Group. q The Steering Group is composed of 10 space–related administrations (often a department dedicated to strategy) and/or ministries with oversight of their country’s space activities, from OECD countries: – Canadian Space Agency (Canada) – CNES (France) – DLR (Germany) – ASI (Italy) – NASA (US) – Korean space agency (Korea) – Norwegian Space Centre (Norway) – Swiss Space Office (Switzerland) – UK Space Agency (United Kingdom) – And the European Space Agency (ESA) In addition close co-operation with industry associations, the private sector, other agencies and ministries… 5

The Space Forum contributes to different OECD missions Stats and indicators Impacts Innovation Policy Country Reviews X X The Impact of Public Support for Business R&D and innovation X X Innovation, Inclusiveness and Development X X Higher Education Institutions in the Knowledge Triangle X X Selected Science Technoloy and Innovation (STI) Themes 2015 -16 As well as cooperation with different directorates and divisions (Environment, Trade, Agriculture, Fisheries…) 6

Achievements so far Ø Highlighting the economic significance of space activities Ø Bringing together experts and building a community of practitioners Ø Making progress on key methodological issues in indicators and statistics Ø Making progress on evaluation / impact assessments of space investments 7

It starts with the numbers OECD Measures, Analyses and Compares • Comparable data across time and countries • International methods • Thorough research 8

Examining socio-economic impacts OECD International Futures Programme 9

Selected economic impacts in different sectors Source: OECD (2014), The Space Economy at a Glance, OECD Publishing, Paris 10

Value of monitoring sea routes with satellites Source: OECD (2014), The Space Economy at a Glance, OECD Publishing, Paris

Some pointers on evaluation and impact assessments (1/2) q Growing demand (justification for spending, more complex programmes, internationalisation of funding mechanisms and expected spillovers) q Evaluation culture and practices uneven across countries, and in some case across national administrations q Fuzziness of basic concepts (difficulty to transpose) q Mismatch between manuals/guidebooks vs. studies’ daily realities (e. g. availability of primary data) q Ignorance about general evaluation guidelines q Need to be aware of unreasonable expectations from “customers”, with ensuing unreasonable methodological approaches to meet these expectations. 12

Some pointers on evaluation and impact assessments (2/2) The best impact assessments on space applications: • The earlier the better when launching assessments, to create a “history” of case studies and time series • Higher quality, when technology / scientific impacts are also provided (not only focus on economic & social) • Survey method or peer review, rather than black box method with too many assumptions • Open tender process for assessments (competition pays off) 13

The way forward…. On assessments of space investments: more effort is needed in building the knowledge base internationally to provide: § Know-how and valid experiences to practitioners (avoiding reinventing the wheel) § Evidence-based information to decision-makers and citizens on benefits (and limitations) of space applications OECD Space Forum contributions to the field: § Bringing together communities of practices (e. g. dedicated expert workshops, IAF Space Economy Technical Committee, SEB group with GEO, …) § Delivering outputs to facilitate international discussions and transfers of know-how (Handbook on measuring the space economy, forthcoming OECD working papers…) 14

“Data to Decisions: Valuing the Societal Benefit of Geospatial Information”. A workshop organized by the Socioeconomic benefits community of practice, in collaboration with OECD, NASA and USGS March 10 -11, 2016, OECD, Paris, France Workshop Objectives: Define foundation case studies for demonstration and comparison of methodologies for assessment of geospatial information and a path forward for research. Identify best practice methodologies for the processes from data to decision. Societal benefit focus areas: v. Disaster Mitigation and Response – focusing on natural disasters, and extreme events v. Ecosystems based Management – highlighting the land-water-energy nexus v. For details, see www. socioeconomicbenefits. org

Thank you for your attention. 16