Testing Solutions Validating Product Market Fit Edmund Pendleton

Testing Solutions Validating Product. Market Fit Edmund Pendleton Lead Instructor, NSF I-Corps & SBIR Bootcamp Director, DC I-Corps Node

The Prequel

My startup set out to change an entire industry

10 years $20 M later

The Sequel

…a different beach…

Innovation and Startup Coach

2, 000+ Teams

10, 000+ Hours

Science and Technology Focus

Long time to market and high risk

“Transforming the world’s largest seed fund into …the world’s largest Accelerator. ”

Not just resources ($) but direction

Our goal is to improve your odds.

Shift the Curve

That is WHY we are here.

Concept Review

Innovation about Customer Needs

Two Extremes of Innovation

Conceptual Innovation

Pay for market forecasts Input from sales and marketing Technical Brainstorm Google Market Trends “Guess” at potential market share Who else is doing what we are doing? What are we good at doing? Build, Show, & Hope Product Strategy What can we do “better” Formulate Product Strategy Formulate Market Strategy

Experimental Innovation

“I haven’t failed.

“I haven’t failed. I’ve just found 10, 000 ways that won’t work. ” – Thomas Edison

You can learn Experimental Innovation

Lean Experimental Innovation

Use a systematic approach…

…to identify unmet needs.

Scientific Method Observe Formulate phenomena hypothesis Test hypothesis via rigorous experiments Modify hypothesis Establish theory based on repeated validation of results

Hypothesis Testing, Iteration, and Pivoting

Essence of Lean Approach

“Can you identify and validate a problem or need in the market that enough people care about? ”

Problem-Solution Fit Product-Market Fit Business-Model Fit

“Can you build and deliver a product/service that satisfies the customer problem or need? ”

Problem-Solution Fit Product-Market Fit Business-Model Fit

Is this a business worth doing…

Problem-Solution Fit Product-Market Fit Business-Model Fit

Needs First Solutions Second!

Initial Focus

Test the Need First

Get out of the building!

Talk (listen) to customers…

Customer Discovery

What are you trying to discover?

Identify the Customer Understand “Jobs-to-be. Done” Quantify Metrics for Success Segment based on Needs Customer Discovery Process Steps Design Value Proposition

#1. Are you going after top-of-mind need?

#2. Who is the person(s) you intend to help?

#3. What does that person seek to do better?

#4. How does that person define better?

Opportunities for innovation arise from better understanding the customer job.

Identify Customer Segment and Value Proposition(s) prior to building rest of Business Model

Hypotheses

How do we turn these into facts…

Identify the Customer Understand “Jobs-to-be. Done” Quantify Metrics for Success Segment based on Needs Design Value Proposition

Customer Discovery Methods

Primary Research (Continue) Customer Interviews (Continue) Customer Observation (Continue) Problem MVP (Continue) Solution MVP (Now) Customer Surveys (Later) Focus Groups (Later)

Primary Research (Continue) Customer Interviews (Continue) Customer Observation (Continue) Problem MVP (Continue) Solution MVP (Now) Customer Surveys (Later) Focus Groups (Later)

Test the Solution Second

What in the world is an MVP?

A minimum viable product (MVP) is a development technique in which a new product or website is developed with sufficient features to satisfy early adopters. The final, complete set of features is only designed and developed after considering feedback from the product's initial users

A minimum viable product (MVP) is a development technique in which a new product or website is developed with sufficient features to satisfy early adopters. The final, complete set of features is only designed and developed after considering feedback from the product's initial users

A minimum viable product (MVP) is an artifact or representation of a value proposition. It is designed to test the validity of one or more critical assumptions or so called hypotheses surrounding a business model or value proposition. Alex Osterwalder

A minimum viable product (MVP) is an artifact or representation of a value proposition. It is designed to test the validity of one or more critical assumptions or so called hypotheses surrounding a business model or value proposition. Alex Osterwalder

A minimum viable product (MVP) is a concise summary of the smallest possible group of features that will work as a stand-alone product while still solving at least the “core” problem and demonstrating the product’s value. Steve Blank

A minimum viable product (MVP) is the simplest thing that you can show to customers to get the most learning at that point in time. Early on in a startup, an MVP could simply be a Power. Point slide, wireframe, clay model, sample data set, etc. Each time you build an MVP you also define what you are trying to test or measure. Later, as more is learned, the MVP’s go from low-fidelity to higher fidelity, but the goal continues to be to maximize learning not to build a beta/fully featured prototype of the product. Steve Blank

A minimum viable product (MVP) is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort. Eric Ries

A minimum viable product (MVP) is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort. Eric Ries

A minimum viable product (MVP) is the smallest thing you can build that delivers customer value (and as a bonus captures some of that value back). Ash Maurya

A minimum viable product (MVP) is the smallest thing you can build that delivers customer value (and as a bonus captures some of that value back). Ash Maurya

A minimum viable product (MVP) Frank Robinson We define MVP as that unique product that maximizes return on risk for both the vendor and the customer… is a mindset of the management and development-team. It says, think big for the long term but small for the short term. Think big enough that the first product is a sound launching pad for it and its next generation and the roadmap that follows, but not so small that you leave room for a competitor to get the jump on you. Too large or too small a product are big problems. The MVP is the difficult-to-determine sweet spot between them. Teams flounder tactically in trying to determine the MVP.

A minimum viable product (MVP) We define MVP as that unique product that maximizes return on risk for both the vendor and the customer… Clayton Christensen

A minimum viable product (MVP) We define MVP as that unique product that maximizes return on risk for both the vendor and the customer… Clayton Christensen

A minimum viable product (MVP) is the most pared down version of a product that can still be released and has three key characteristics: • enough value that people are willing to use it or buy it initially • enough future benefit to retain early adopters • a feedback loop to guide future development.

A minimum viable product (MVP) is that version of a new product a team uses to collect the maximum amount of validated learning about customers with the least effort. " The definition's use of the words maximum and minimum means it is decidedly not formulaic.

“The power of the MVP approach is matched only by the amount of confusion it causes…”

WTH MVP OMG

Problem / Need MVPs

Good Starting Point

What is current process (solution)?

Where are current challenges?

WORK FLOW (UPW example) Pretreatment Primary Loop Secondary Loop

Observe Current Process

Understand Competition

Both competitors already have apps out. Is Bay Alarm competitive in today's market…or do users even care about apps?

Existing Process vs Desired Job

Understand the difference between what customers want to do (the job). . . versus what they are currently doing (existing solution)

Job Map What is the customer trying to accomplish? Define Locate Prepare Pre Execute Confirm Execute Monitor Modify Post Execute Conclude

Job Steps What is the customer trying to accomplish? Define Step 1 Step 2 Step 3 … Locate Step 1 Step 2 Step 3 … Prepare Step 1 Step 2 Step 3 … Confirm Step 1 Step 2 Step 3 … Execute Step 1 Step 2 Step 3 … Monitor Step 1 Step 2 Step 3 … Modify Conclude Step 1 Step 2 Step 3 …

Success Metrics How does the customer measure success? Define *Outcome 1 *Outcome 2 *Outcome 3 … Locate *Outcome 1 *Outcome 2 *Outcome 3 … Prepare Confirm Execute Monitor Modify Conclude *Outcome 1 *Outcome 2 *Outcome 3 … *Outcome 1 *Outcome 2 *Outcome 3 …

What If… Conversations

How would process change?

After validating a true unmet need…

You will start testing the product*

Can you build and deliver a product/service that satisfies the customer problem or need…in a form the customer wants to consume it?

Can you build and deliver a product/service that satisfies the customer problem or need…in a form the customer wants to consume it?

Get out of the building!

We will use MVP as shorthand for any prototype!

Using Prototypes

Using MVPs to test Solutions

Build, Show, and Hope!

Build, Show, and Hope!

Don’t just build and show… be deliberate!

Use a systematic approach…

…to validate that you solution addresses unmet needs.

Scientific Method Observe Formulate phenomena hypothesis Test hypothesis via rigorous experiments Modify hypothesis Establish theory based on repeated validation of results

The ultimate ”test” …will people buy it.

But a “sellable” product …is often the most expensive test.

MVPs

MVPs

MVPs Not For Sale

Low Fidelity MVP

Used to test early concepts before building a “working solution”…

The purpose – to test whether you are headed in the right direction?

“Higher” Fidelity MVP

The purpose – to test whether basic functionality addresses a problem or need?

Pilot Site MVP

For some complex system solutions, pilot sites are required to prove value to customers.

Purpose of MVPs… move abstract to specific!

Example MVPs

Why is this so challenging?

“The first principle is that you must not fool yourself, and you are the easiest person to fool. ” – Richard Feynman

“The first principle is that you must not fool yourself, and you are the easiest person to fool. ” – Richard Feynman

People are Not Rational

Even scientists struggle designing experiments

“I have approximate answers and possible beliefs and different degrees of certainty about different things, but I’m not absolutely sure about anything. ” – Richard Feynman

“I have approximate answers and possible beliefs and different degrees of certainty about different things, but I’m not absolutely sure about anything. ” – Richard Feynman

“People don’t know what they want until I show it to them!”

“People don’t know what they want until I show it to them!”

You are trying to discover the ”why”

Question Answer

Intellectual Property and the MVP

Intellectual Property: any product of the human intellect that the law protects from unauthorized use by others

Why should you protect IP?

80%+ of a company’s value can reside in IP

Increase Value by… excluding competition leveraging license deals avoiding trade secrets loss

MVP Testing Risks 1. The idea becomes part of the public domain through a public disclosure. 2. The invention is appropriated through a prior filing by another.

Public Disclosure 1. The idea becomes part of the public domain through a public disclosure. – The invention is described without the benefit of confidentiality – Results in the loss of exclusivity through the inability to enforce an exclusive statutory right (patent)

First to File 2. The invention is appropriated through a prior filing by another. – The invention is patented by another’s prior filing – Results in the inability to practice ones own invention – March 16, 2013 – USA switched from “first-to-invent” to “firstinventor-to-file”

Non Disclosure Agreements (CDA) and Patent Filings Mitigate IP Risks Differently Non Disclosure agreements (CDA) • Mitigate the risk of a disclosure invalidating a later filed patent • Mitigate the risk that the receiving party will claim a patent based on the information • BUT: scope of the agreement governs what’s protected, and resulting claim is in Contract. Claims of Inventor-ship /derivation require substantial documentation. Patent Filing (Including Provisional Patent Filings) • Eliminate the risk of appropriation based on the disclosure to another • Mitigate the risk of the disclosure to another invalidating the patent • But: Ability to exclude is limited to the description provided in the patent (a significant pivot or hasty drafting increases risk of BOTH public disclosure and appropriation)

Customer Discovery: Test the Problem Can we test the problem without disclosing potential solutions? Common mistakes • • Disclosure without protection Disclosing potential solutions as a part of testing the problem (IP loss) Filing provisional patents that don’t cover the real problem or the later discovered solution Provide technology insight that enables others to file patents

Intellectual Property Plan Executed Common mistakes • • Assume provisional is a good replacement for CDA – Not enabled/or support (EU/China) – Not entitled to date – Pivot takes you outside patent scope Losing patent rights due to filing delays/invention disclosures – Disclosure in a printed publication – Offer for sale in the US if pivot on feedback makes new product not entitled to provisional date