Program Evaluation and Metrics for SBA Regional Innovation

Program Evaluation and Metrics for SBA Regional Innovation Clusters Presented by: Berna Demiralp, Ph. D. Optimal Solutions Group, LLC M Squared Research Park | University of Maryland 5825 University Research Court, Suite 2800 College Park, MD 20740 301 -306 -1170 | info@Optimal. Solutions. Group. com

Introduction • SBA’s Regional Innovation Clusters Initiative provides funding to ten clusters with the goal of Østrengthening clusters and enhancing their ability to assist small businesses; Øincreasing opportunities for small business participation and growth within the clusters; Østimulating regional economic development and growth.

What are clusters? • “Clusters are geographic concentrations of interconnected companies, specialized suppliers, service providers, firms in related industries, and associated institutions (e. g. , universities, standard agencies, and trade associations) in particular fields that compete but also cooperate” (Porter, 1998).

SBA Regional Innovation Clusters • Clusters selected for the SBA’s Regional Innovation Clusters Initiative represent a broad spectrum of geographic areas and industries.

Advanced Power and Energy Cluster (MN, ND, SD, WI): power generation, storage, and alternative Illinois Smart Grid Regional Innovation Cluster (IL): Smart Grid innovation improving electric transmission infrastructure. Northeast Electrochemical Energy Storage Cluster (CT, NY, MA, ME, VT, NH, RI): developing, manufacturing, and deploying fuel cell and hydrogen technologies Agricultural Innovation Cluster (Monterey – Santa Cruz – San Benito, CA): agricultural innovation and leading-edge agriculture research. San Diego Advanced Defense Technologies Cluster (San Diego, CA): autonomous systems, cyber security, and other technologies Upper Michigan Green Aviation Coalition (MI): expanding green aviation industry. Carolina Nuclear Cluster (NC, SC): improving nuclear energy supply chain. Enterprise for Innovative Geospatial Solutions (MS, LA): geospatial businesses Huntsville Advanced Defense Technology Cluster Initiative (Huntsville, AL): aerospace technology Nor. Tech Flex. Matters (OH): advanced energy and electronic devices printed on flexible substrates.

SBA’s Regional Innovation Clusters • Small businesses-oriented activities/events: ØBusiness counseling; ØWorkshops, training, and mentoring; ØMatchmaking and networking activities; ØAssistance in obtaining financing; ØCommercialization and technology transfer; ØExport services; ØIntellectual property and patent services; ØMarketing services; and, ØContract counseling and other legal services.

Evaluation Components • Implementation evaluation ØAre clusters reaching the targeted small businesses in their regions? ØAre the cluster activities / events meeting the needs of the targeted small businesses? • Impact evaluation ØAre the small businesses participating in cluster activities / events attaining their desired goals?

Program Parameters 1. Variety of cluster configurations and business models; 2. Varied small business recruitment strategies; 3. Varied inclusion and exclusion criteria for selecting small firm; and, 4. Clusters were free to choose the types and frequency of services/activities.

Data Collection • Both quantitative and qualitative data: ØCluster Administrator Survey ØSmall Business Survey ØLarge Organization Survey ØInterviews with Cluster Administrators ØInterviews with selected small businesses ØCluster Quarterly Reports ØQuality Indicators Survey (to be completed by both the cluster administrator and the evaluator)

Data Collection • Secondary data sources: ØDun & Bradstreet database ØQuarterly Census of Employment and Wages (BLS) ØADP Small Business Report ØLongitudinal Business database

Program Metrics and Outcomes • Proximal: Ø Ø Ø Ø Ø Access to capital; New alliances formed; Exports; Development of new products and services; Patents; Access to cleared secure facilities; Acquiring a contract or subcontract; Participation in industry chain; and Commercialization of new technology.

Program Metrics and Outcomes • Distal: ØNumber of businesses created; ØRevenue; ØNumber of employees; and, ØTotal compensation.

Metrics and Methodology • How do the performance outcomes of small businesses participating in cluster activities/events change during the first year? ØNon-experimental: üPerformance metrics (levels) üGrowth models

Metrics and Methodology • How do the changes in outcomes experienced by small businesses participating in cluster activities/events compare to the well-known benchmarks? ØCompare the changes in participating firms’ performance metrics (revenue, number of employees and total compensation) to similar small businesses in the same industries and geographic areas using secondary data sources.

Metrics and Methodology • What impact did cluster participation have on small businesses’ performance metrics (revenue, number of employees, total compensation) during the first year? ØQuasi-experimental: üDifference-in-Difference models üMatched pair models

Metrics and Methodology • What impact does cluster participation have on regional economies? ØUtilize input-output models for each metropolitan area. ØInput-output models take into account interindustry interactions and transactions between producers, intermediaries, and consumers. ØAlternative input-output models: ü RIMS II (Census Bureau) ü IMPLAN ü REMI

Metrics and Methodology • Input-output models estimate the impact on the regional economy: Ødirect effect (e. g. 10 new jobs created in firm A); Øindirect effect (i. e. increase in output and employment in industries that supply firm A); and Øinduced effect (i. e. the effect of increased household income and spending).

Metrics and Methodology • What is the direct effect that will be analyzed in the input-output model? ØThe change in participating firms’ performance outcomes (revenue, number of employees, total compensation) over and above the change in the outcomes experienced by the comparison group in the quasi-experimental models.

Conclusion • Producing more cluster activities/events do not necessarily yield innovation and key outcomes; • Investing in clusters do not necessarily yield key outcomes in the short-run (“not shovel ready”); • Literature suggests that clusters can produce sustainable improvements in innovation and key outcomes; and, • However, there are few rigorous evaluations of innovative clusters that provide generalizable findings.