Data Science Design Thinking a perfect blend to

Data Science + Design Thinking: a perfect blend to achieve the best user experience September 12, 2019 AI Conference SJ 2019 Michael Radwin Vice President, Intuit

Who we serve: Consumers Small businesses Self-employed

Great AI, bad design 3

Data science vs. design mindsets MODELS LABELS PRECISION/RECALL AUC K-MEANS HYPER-PERAMETER TUNING 3 -CLICK RULE GOLDEN RATIO F-SHAPED PATTERN WIREFRAME CARD SORTING HEATMAP 4

How can we come together?

Solution: Design for Delight Deep customer empathy Go broad to go narrow Rapid experiments with customers 6

Empathy Knowing customers better than they know themselves Deep customer empathy

Broad to narrow Iterating your way to the answer Go broad to go narrow

Go broad to go narrow Rapid experiments with customers Getting tangible and learning from customers quickly

Design for Delight Method 1: Follow-Me-Home Method 2: Customer Problem Statement Deep customer empathy Method 3: Brainstorming Method 4: 2 x 2 Narrowing Go broad to go narrow Rapid experiments with customers Method 5: Leap of Faith Assumptions Method 6: Paper Prototyping 10

Examples of design thinking applied to Intuit’s data science product features

Mint Overdraft Prediction

Problem: Americans pay billions of dollars in overdraft fees every year when they have insufficient funds. 50% of those fees are for transactions totaling less than $50. The average overdraft fee is $35. It’s hard for people to get ahead. 13

We built a supervised ML model to predict overdraft 14

Before Avoid overdrafting your Bank of America account by taking action After Mint customers have saved nearly $1 M in overdraft fees over the past 12 months. The overdraft prediction service sent more than 650 K alerts to help users avoid paying non-sufficient funds fees. 15

D 4 D Method: 2 x 2 Narrowing is about making decisions with intention. It’s not about reaching consensus on the easiest idea to implement or voting for your favorite.

D 4 D Method: 2 x 2 Narrowing High impact on customer benefit Narrowing is about making decisions with intention. It’s not about reaching consensus on the easiest idea to implement or voting for your favorite. A 2 x 2 forces the team to balance the tension between two distinct criteria. Ideas are then placed on the 2 x 2 matrix relative to each other, which fosters productive debate and intentional decisions about which ideas the team will pursue. Requires new capabilities Can be done today Low impact on customer benefit

D 4 D Method: 2 x 2 Narrowing High impact on customer benefit Narrowing is about making decisions with intention. It’s not about reaching consensus on the easiest idea to implement or voting for your favorite. A 2 x 2 forces the team to balance the tension between two distinct criteria. Ideas are then placed on the 2 x 2 matrix relative to each other, which fosters productive debate and intentional decisions about which ideas the team will pursue. Experiment with axis labels. You don‘t always have to pick the top-right quadrant. Or, if all your ideas are in one quadrant, try different axes. Requires new capabilities Can be done today Low impact on customer benefit

Quick. Books Self-Employed Smart. Groups 19

Problem: Business-related trips are tax-deductible for the self-employed. The Quick. Books Self-Employed mobile app automatically tracks trip miles. It’s time-consuming to review trips to determine which are personal and which are for business. 20

We used pattern mining to group business vs. personal trips Sample business trip data for a user Trip_id Start_location End_location Is_weekend_trip 12345 A B False 34394 A B False 15353 A C True 34293 A B False 46483 B A True 21632 A B False 21

Before After Customers categorize an average of 100 trips/month in significantly less time. The experience is tailored to a customer’s individual driving patterns. 22

D 4 D Method: Rapid Experimentation & Paper Prototyping Rapid prototyping with customers facilitates quick learning and iterations. A good experiment helps you get past what customers say they would do and discover what they actually do. Paper or sketch prototyping is an inexpensive way to test ideas early in the process.

D 4 D Method: Rapid Experimentation & Paper Prototyping WARM UP by greeting the customer. ASK the customer to verbalize their thoughts. SET THE SCENE where the customer will be using the prototype. OBSERVE and record the customer’s behavior. LISTEN for how they’re feeling. WATCH for surprises. LET THEM ESCAPE gracefully.

Standard Deduction vs. Itemized Deduction 25

Problem: American taxpayers waste time itemizing deductions, only to find that they should take the standard deduction. Turbo. Tax is able to quickly predict which deduction is right for the customer. Taxpayers need confidence that they are getting every penny they deserve. 26

It wasn’t enough to build an SVI deduction prediction model INPUT: PTT output GTKM output Prev year PI, Sch. A, Income , etc. Itemized (Baseline) Experience ML Model Itemized Deduction SVI Interview Alternative Experience Standard Deduction 27

Taxpayers needed us to “show our work” We estimate that nearly 90% of customers will now take the higher standard deduction when shown this screen. 40% less time spent by taxpayers! 28

D 4 D Method: Follow-Me-Home Find real customers and visit them in their natural habitat (at home or work). Observe the customer trying to complete a task. Ask them to show you. This is not an interview. Assume a posture of wonder and curiosity. Don’t judge. Be present and listen actively. Look for surprises. Insights often come from unexpected or unusual behaviors.

D 4 D Method: Follow-Me-Home Create a customer journey line during visit when you need to understand how your customer approaches a task and how they feel about it along the way, including the most delightful — and painful — points. Debrief with your team. As soon as possible after the session, get together with your team and capture the surprises and pain points you saw.

Data Science & Design mindsets MODELS LABELS PRECISION/RECALL AUC K-MEANS HYPER-PERAMETER TUNING 3 -CLICK RULE GOLDEN RATIO F-SHAPED PATTERN WIREFRAME CARD SORTING HEATMAP 31

Data Science & Design mindsets MODELS LABELS PRECISION/RECALL AUC K-MEANS HYPER-PERAMETER TUNING 3 -CLICK RULE GOLDEN RATIO F-SHAPED PATTERN WIREFRAME CARD SORTING HEATMAP CUSTOMER SUCCESS 32

Data Science + Design Thinking CUSTOMER SUCCESS Deep customer empathy Go broad to go narrow Rapid experiments with customers 33

Thank you! @michael_radwin