Data Science Design Thinking a perfect blend to
Data Science + Design Thinking: a perfect blend to achieve the best user experience September 12, 2019 AI Conference SJ 2019 Michael Radwin Vice President, Intuit
Who we serve: Consumers Small businesses Self-employed
Great AI, bad design 3
Data science vs. design mindsets MODELS LABELS PRECISION/RECALL AUC K-MEANS HYPER-PERAMETER TUNING 3 -CLICK RULE GOLDEN RATIO F-SHAPED PATTERN WIREFRAME CARD SORTING HEATMAP 4
How can we come together?
Solution: Design for Delight Deep customer empathy Go broad to go narrow Rapid experiments with customers 6
Empathy Knowing customers better than they know themselves Deep customer empathy
Broad to narrow Iterating your way to the answer Go broad to go narrow
Go broad to go narrow Rapid experiments with customers Getting tangible and learning from customers quickly
Design for Delight Method 1: Follow-Me-Home Method 2: Customer Problem Statement Deep customer empathy Method 3: Brainstorming Method 4: 2 x 2 Narrowing Go broad to go narrow Rapid experiments with customers Method 5: Leap of Faith Assumptions Method 6: Paper Prototyping 10
Examples of design thinking applied to Intuit’s data science product features
Mint Overdraft Prediction
Problem: Americans pay billions of dollars in overdraft fees every year when they have insufficient funds. 50% of those fees are for transactions totaling less than $50. The average overdraft fee is $35. It’s hard for people to get ahead. 13
We built a supervised ML model to predict overdraft 14
Before Avoid overdrafting your Bank of America account by taking action After Mint customers have saved nearly $1 M in overdraft fees over the past 12 months. The overdraft prediction service sent more than 650 K alerts to help users avoid paying non-sufficient funds fees. 15
D 4 D Method: 2 x 2 Narrowing is about making decisions with intention. It’s not about reaching consensus on the easiest idea to implement or voting for your favorite.
D 4 D Method: 2 x 2 Narrowing High impact on customer benefit Narrowing is about making decisions with intention. It’s not about reaching consensus on the easiest idea to implement or voting for your favorite. A 2 x 2 forces the team to balance the tension between two distinct criteria. Ideas are then placed on the 2 x 2 matrix relative to each other, which fosters productive debate and intentional decisions about which ideas the team will pursue. Requires new capabilities Can be done today Low impact on customer benefit
D 4 D Method: 2 x 2 Narrowing High impact on customer benefit Narrowing is about making decisions with intention. It’s not about reaching consensus on the easiest idea to implement or voting for your favorite. A 2 x 2 forces the team to balance the tension between two distinct criteria. Ideas are then placed on the 2 x 2 matrix relative to each other, which fosters productive debate and intentional decisions about which ideas the team will pursue. Experiment with axis labels. You don‘t always have to pick the top-right quadrant. Or, if all your ideas are in one quadrant, try different axes. Requires new capabilities Can be done today Low impact on customer benefit
Quick. Books Self-Employed Smart. Groups 19
Problem: Business-related trips are tax-deductible for the self-employed. The Quick. Books Self-Employed mobile app automatically tracks trip miles. It’s time-consuming to review trips to determine which are personal and which are for business. 20
We used pattern mining to group business vs. personal trips Sample business trip data for a user Trip_id Start_location End_location Is_weekend_trip 12345 A B False 34394 A B False 15353 A C True 34293 A B False 46483 B A True 21632 A B False 21
Before After Customers categorize an average of 100 trips/month in significantly less time. The experience is tailored to a customer’s individual driving patterns. 22
D 4 D Method: Rapid Experimentation & Paper Prototyping Rapid prototyping with customers facilitates quick learning and iterations. A good experiment helps you get past what customers say they would do and discover what they actually do. Paper or sketch prototyping is an inexpensive way to test ideas early in the process.
D 4 D Method: Rapid Experimentation & Paper Prototyping WARM UP by greeting the customer. ASK the customer to verbalize their thoughts. SET THE SCENE where the customer will be using the prototype. OBSERVE and record the customer’s behavior. LISTEN for how they’re feeling. WATCH for surprises. LET THEM ESCAPE gracefully.
Standard Deduction vs. Itemized Deduction 25
Problem: American taxpayers waste time itemizing deductions, only to find that they should take the standard deduction. Turbo. Tax is able to quickly predict which deduction is right for the customer. Taxpayers need confidence that they are getting every penny they deserve. 26
It wasn’t enough to build an SVI deduction prediction model INPUT: PTT output GTKM output Prev year PI, Sch. A, Income , etc. Itemized (Baseline) Experience ML Model Itemized Deduction SVI Interview Alternative Experience Standard Deduction 27
Taxpayers needed us to “show our work” We estimate that nearly 90% of customers will now take the higher standard deduction when shown this screen. 40% less time spent by taxpayers! 28
D 4 D Method: Follow-Me-Home Find real customers and visit them in their natural habitat (at home or work). Observe the customer trying to complete a task. Ask them to show you. This is not an interview. Assume a posture of wonder and curiosity. Don’t judge. Be present and listen actively. Look for surprises. Insights often come from unexpected or unusual behaviors.
D 4 D Method: Follow-Me-Home Create a customer journey line during visit when you need to understand how your customer approaches a task and how they feel about it along the way, including the most delightful — and painful — points. Debrief with your team. As soon as possible after the session, get together with your team and capture the surprises and pain points you saw.
Data Science & Design mindsets MODELS LABELS PRECISION/RECALL AUC K-MEANS HYPER-PERAMETER TUNING 3 -CLICK RULE GOLDEN RATIO F-SHAPED PATTERN WIREFRAME CARD SORTING HEATMAP 31
Data Science & Design mindsets MODELS LABELS PRECISION/RECALL AUC K-MEANS HYPER-PERAMETER TUNING 3 -CLICK RULE GOLDEN RATIO F-SHAPED PATTERN WIREFRAME CARD SORTING HEATMAP CUSTOMER SUCCESS 32
Data Science + Design Thinking CUSTOMER SUCCESS Deep customer empathy Go broad to go narrow Rapid experiments with customers 33
Thank you! @michael_radwin
- Slides: 34