AEACES Think Tank Operating in Pasteurs Quadrant Use

Overview of the Think Tank • Evolution of the concept: Use Inspired Basic Research

The “Linear” Model of R&D Leading to Innovation Funding Societal Basic Research Condensed Matter

Complex “Ecosystem” in which Innovation Flourishes Funding Societal Basic Research Capital Markets Applied Research

Theoretical Constructs • Kline’s Chain-Linked Model (1985) – The central process of innovation is

Stokes’s Model Consideration of Use Quest for Understanding Research Inspired by No Yes No

Defining Use Inspired Basic Research • Basic research is defined as systematic study directed

Defining Use Inspired Basic Research Environment Government/Regulators/Industry • Basic research is defined as systematic

Examples • United States – Most of National Institutes of Health – Parts of

Fundamental Question Is UIBR meaningful concept worthy of evaluating differently than other kinds of

Evaluation Challenges - 1 • Programs tend to be unclear as to whether they

Evaluation Challenges - 2 • More stakeholders, all of whom have conflicting priorities –

Evaluation Challenges - 3 • Complex pathways to success – Great variability across projects;

UIBR Follows Complex Pathways in the Innovation Spectrum

Environment Requirements Identify societal/market needs; define system and system requirements Products and Outcomes Technology

Measures for UIBR - Paper productivity - Partnerships - Stakeholder - Paper/journal engagement quality

Evaluation Challenges - 4 • Project data (especially process-related data) is distributed across multiple

Challenges in Evaluating the New Zealand New Economy Research Fund (NERF) Program

What is NERF? • NERF was established by the Ministry of Research, Science and

Evaluation Methodology • We began “traditionally” – Used an adapted linear model (Branscomb et

Challenge 1: Program Unclear if UIBR • NERF’s Ministerial notice stated four broad goals:

Challenge 2: Stakeholders with Conflicting Priorities • Identified stakeholders – – – Researchers Research

Challenge 3: Complex Pathways to Success • NERF’s Ministerial notice was written in broad

Challenge 4: Project data was distributed across multiple sectors • Did not encounter this

Discussion Questions 1. 2. 3. 4. 5. Is the concept of UIBR relevant today?

Concluding Thought “Some bargains are Faustian, and some horses are Trojan. Dance carefully with

Slides: 32

Download presentation

AEA/CES Think Tank Operating in Pasteur’s Quadrant: Use Inspired Basic Research Bhavya Lal Core Staff Member Science and Technology Policy Institute 1899 Pennsylvania Avenue NW, Washington DC 20006 Lisa Messeri Analyst, Abt Associates Inc 55 Wheeler Street, Cambridge MA 02138

Overview of the Think Tank • Evolution of the concept: Use Inspired Basic Research (UIBR) • Definition • Evaluation Challenges • Case Study • Discussion: Potential Solutions

The “Linear” Model of R&D Leading to Innovation Funding Societal Basic Research Condensed Matter Physics Applied Research Development Technology/ Application Benefit Electronics Industry

Complex “Ecosystem” in which Innovation Flourishes Funding Societal Basic Research Capital Markets Applied Research Development Govt. Regulation Technology/ Application Industry Benefit

Theoretical Constructs • Kline’s Chain-Linked Model (1985) – The central process of innovation is not science but design, and the assumption that basic research drives innovation is flawed (Kline & Rosenberg, 1986) • Leydesdorff “triple helix” (2000) – It is incorrectly assumed that universities and industry are the only players. Several other actors play important roles in innovation processes, such as government, investors, and end users • Stokes’s Pasteur Quadrant (1997) – There is an artificial distinction between basic and applied research http: //nanoandsociety. com/ourlibrary/documents/bsts-nano. pdf

Stokes’s Model Consideration of Use Quest for Understanding Research Inspired by No Yes No Pure Basic Research (Bohr) Yes Use Inspired Basic Research (Pasteur) Pure Applied Research (Edison) http: //www. cspo. org/products/conferences/bush/Stokes. pdf Research Inspired by

Overview of the Think Tank • Evolution of the concept: Use Inspired Basic Research (UIBR) • Definition • Evaluation Challenges • Case Study • Discussion: Potential Solutions

Defining Use Inspired Basic Research • Basic research is defined as systematic study directed toward fuller knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications towards processes or products in mind. NSF Definition of Basic Research • Use-Inspired basic research is simply that – basic research motivated by ultimate application!

Defining Use Inspired Basic Research Environment Government/Regulators/Industry • Basic research is defined as systematic study directed toward fuller knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications towards processes Fundamental Insights or products in mind. System Requirements NSF Definition of Basic Research • Use-Inspired basic research is simply that – basic research motivated by ultimate application! Knowledge Base Fundamental Research Develop useful insights from fundamental knowledge

Examples • United States – Most of National Institutes of Health – Parts of NSF – Engineering, Computer and Information Science and Engineering • • • Canada – National Centers of Excellence Japan – JST/ERATO Sweden – Competence Centres Australia – Cooperative Research Centres New Zealand – New Economy Research Fund

Overview of the Think Tank • Evolution of the concept: Use Inspired Basic Research (UIBR) • Definition • Why is it difficult to evaluate • Case Study • Discussion: Potential Solutions

Fundamental Question Is UIBR meaningful concept worthy of evaluating differently than other kinds of programs?

Evaluation Challenges - 1 • Programs tend to be unclear as to whether they are UIBR, and what that means – Is it enough for the research to be “inspired” by practical use? Or do users need to be engaged? – Are most programs today partially use-inspired? • Simply because that is how you can justify taxpayerfunded programs! Can one identify a UIBR program as one?

Evaluation Challenges - 2 • More stakeholders, all of whom have conflicting priorities – PIs want to focus on “publishable” research; may often be unaware of, ignore, or purport to meet program goals – Industry has more influence, and wants “widgets” – Government wants to: justify programs to taxpayers, and show links between funding and economic development How should the evaluation be designed differently as a result of this?

Evaluation Challenges - 3 • Complex pathways to success – Great variability across projects; different definitions of “success” – Process issues becomes as important to examine and model as outputs and outcomes How do you operationalize these concepts into meaningful indicators?

UIBR Follows Complex Pathways in the Innovation Spectrum

Environment Requirements Identify societal/market needs; define system and system requirements Products and Outcomes Technology Integration Systems Research System Requirements Integrating fundamental knowledge into enabling technology Develop useful insights from fundamental knowledge Technology Elements Technology Base Enabling Tech. Research Fundamental Insights Knowledge Base Fundamental Research Source: NSF ERC Program

Measures for UIBR - Paper productivity - Partnerships - Stakeholder - Paper/journal engagement quality - Awards and prizes - Creation of test beds, prototypes, instrumentation - Creation of intellectual property - Dissemination - Development of human capital (training) Cheaper/newer/ better products

Evaluation Challenges - 4 • Project data (especially process-related data) is distributed across multiple sectors, and therefore more likely to be incomplete or inaccurate – Tendency to exaggerate/de-emphasize impact – Some data is outside of the projects; often data on use is confidential All of the data collection challenges of basic research and applied research PLUS additional challenges

Overview of the Think Tank • Evolution of the concept: Use Inspired Basic Research (UIBR) • Definition • Evaluation Challenges • Case Study • Discussion: Potential Solutions

Challenges in Evaluating the New Zealand New Economy Research Fund (NERF) Program

What is NERF? • NERF was established by the Ministry of Research, Science and Technology to fund high risk, novel research which could lead to the emergence of new industries in New Zealand. Researchers self classified their activities as basic research and proof-of-concept.

Evaluation Methodology • We began “traditionally” – Used an adapted linear model (Branscomb et al) • During field work, it became apparent that the projects were taking more complex paths to satisfy the criteria of “establishing new industry” • NERF exemplified UIBR – Our methodology needed alteration

“Platform” Representation

Challenge 1: Program Unclear if UIBR • NERF’s Ministerial notice stated four broad goals: Fund investigator-initiated basic research; build critical mass; develop advanced human capital and skills; create a sustainable knowledge platform • Researchers were not focused on the Ministerial Notice • To deal with the ambiguity, our evaluation created a working definition for each goal

Challenge 2: Stakeholders with Conflicting Priorities • Identified stakeholders – – – Researchers Research organizations Government officials Venture capitalists Industry leaders • Due to an insufficient program model, were unable to link back stakeholders needs to project performance

Challenge 3: Complex Pathways to Success • NERF’s Ministerial notice was written in broad terms – Researchers freely interpreted the notice, leading to a wide range in project goals • Were unable to assign a single or a small set of ‘success metrics’ to projects – Some projects reported large numbers of publications; others focused on IP creation • Did highlight many different styles of success in case studies and qualitative examples

Challenge 4: Project data was distributed across multiple sectors • Did not encounter this anticipated challenge – Did not have systematic data from all sectors – Collected qualitative data during site visit

Overview of the Think Tank • Evolution of the concept: Use Inspired Basic Research (UIBR) • Definition • Evaluation Challenges • Case Study • Discussion

Discussion Questions 1. 2. 3. 4. 5. Is the concept of UIBR relevant today? Are other paradigms (e. g. triple helix, innovation systems) superior? Is pretty much every program these days UIBR? Are there particular management issues with UIBR that are worth discussing? Are the challenges that we encountered while evaluating UIBR unique? What challenges did we miss? How would you overcome these challenges?

Concluding Thought “Some bargains are Faustian, and some horses are Trojan. Dance carefully with the porcupine, and know in advance the price of intimacy” http: //nanoandsociety. com/ourlibrary/documents/bsts-nano. pdf