Product Roadmap via AB Testing Product Roadmap Deconstruct

Product Roadmap via A/B Testing Product Roadmap Deconstruct Customer Product Managers or Roadmap into business Analysts Testable Chunks A/B Testing CRO Business Analyst or Developers or Expectations + Business Goals UX Designers 01 Product Market Needs + 02 03 Program Management (Dedicated CRO program Manager) Implementation via Growth Conversion Data Scientist or Analysts CRO Analysts 04 05

A/B Testing Process 01 Define Goal (conversion) Before your create a test, your 01 need to know what, exactly, you’re hoping to accomplish. 04 Crown the New Champion Once you’ve launched your test, you need to let it run for a 02 Brainstorm/Generate 05 02 long enough period of time. CONTINUOUS IMPROVEMENT Idea Once you’ve determined a goal you want to accomplish, you’ll need to generate ideas. 05 Test Until Statistically Significant By this point, you’ve set a goal, select a page and determine 03 Implement Changes You will be need to implement changes with highest potential, intuition and best practices. 03 04 what you want to test.

A/B Testing Process VARIATION A 30% Conversion rate 50% visitors see variation A VARIATION B 16% 50% visitors see variation B Conversion rate

A/B Test Results VARIATION CONVERSION RATE IMPROVEMENT CONFIDENCE CONV. /VISITORS STATUS ORIGINAL 0. 15% (+0. 10%) - - 5/5, 134 Stopped VARIATION 01 0. 25% (+0. 20%) +150. 00% 97. 01% 13/5, 152 Winner A/B Testing Line Graph Total visitors 100 90 80 Average conversion rate 70 60 50 Original 40 30 20 Variation 01 10 0 Mar-02 Mar-03 Mar-04 Mar-05 Mar-06 Mar-07 Mar-08 Mar-09 Mar-10 Mar-11 Mar-12 Mar-13 Mar-14 Mar-15 Mar-16

A/B Test Tracking OWNER LANDING PAGE (LIST NAME OF DRI FOR STATUS TEST START END PREDICTION RESULT STATISTICALLY SIGNIFICANT ? NOTES NEXT STEPS TEST HERE) How to Variation B performed 8% better Version B will perform Create John Visuals for Smith Completed LP Header swapped with subheader 5/17/17 5/20/17 Marketing better because the original sub header is Success ✓ more actionable Demo Amanda Landing Page Morgan Call Sales John Landing Page Smith Not Started Different image on B version 6/20/17 A is the title, B is Inconclusive X 7/6/17 descriptive better because it placement next than A - not statistically significant but declared B as winner. Version A will perform 6/23/17 setting up a/b on CTA 95% confidence. Declared Variation B performed 4% better perform better Different Headline. Not Started background) will Chose Variation B as winner, was statistically significant at version B the winner Version B (image with 5/27/17 than the original version and Inconclusive X matches promo copy Chose Variation B. setting up a/b testing two different types of images next Version A performed 3% better, Chose Variation A as winner, not statistically significant. Test running template-level A/B test ran for 3 weeks. next Test "A" converted 33%better 2018 Predictions Landing Page Amanda Morgan In Progress Copy bullets vs long form paragraphs 6/17/17 7/12/17 version B will perform better Success ✓ than Test "B. " We are 100% Chose Variation A as winner, certain that the changes in Test setting up a/b on CTA "A" will improve your placement next conversion rate.

Get the full version of this presentation and access our library of hundreds of corporate resources Join You Exec Plus today and take your career to new heights. Receive premium corporate resources in your inbox every week. Start today