1 Causal Inference in R Ana Daglis Farfetch

  • Slides: 21
Download presentation
1 Causal Inference in R Ana Daglis, Farfetch

1 Causal Inference in R Ana Daglis, Farfetch

2 Boutique Customer Farfetch Customer Boutique

2 Boutique Customer Farfetch Customer Boutique

3 One of the most common questions we face in marketing is measuring the

3 One of the most common questions we face in marketing is measuring the incremental effects ● How much incremental revenue did the new pricing strategy drive? ● What impact did the new feature on the website have? ● How many incremental conversions were achieved by increasing the commission rate for our affiliates? ● …

4 The main gold standard method for estimating causal effects is a randomised experiment

4 The main gold standard method for estimating causal effects is a randomised experiment 10% Conversion 15% Conversion Version A 50% of visitors see Version A Version B 50% of visitors see Version B

5 However, often A/B tests are either too expensive to run or cannot be

5 However, often A/B tests are either too expensive to run or cannot be run, e. g. due to legal reasons 15% Conversion Version A Version B 100% of visitors see Version B

6 Example: financial performance of a company A Actual share price Scandal broke

6 Example: financial performance of a company A Actual share price Scandal broke

7 Approach: estimate the share price had the scandal not happened Actual share price

7 Approach: estimate the share price had the scandal not happened Actual share price Predicted share price Scandal broke

8 By comparing the actual and predicted share price, we can estimate the drop

8 By comparing the actual and predicted share price, we can estimate the drop in stock value due to the scandal Actual share price Predicted share price Drop in stock value due to scandal Scandal broke

9 Thanks to a fully Bayesian approach, we can quantify the confidence level of

9 Thanks to a fully Bayesian approach, we can quantify the confidence level of our predictions Actual share price Predicted share price 95% credible interval Scandal broke

10 How do we construct the counterfactual estimate? Training Prediction Actual share price Predicted

10 How do we construct the counterfactual estimate? Training Prediction Actual share price Predicted share price 95% credible interval Company B share price Company C share price Scandal broke

11 Causal Impact methodology is based on a Bayesian structural time series model Most

11 Causal Impact methodology is based on a Bayesian structural time series model Most general form of the model Causal Impact model Observation equation State equation

12

12

13 Priors

13 Priors

14 Priors

14 Priors

15 Priors

15 Priors

16 The inference can be performed in R with just 6 lines of code

16 The inference can be performed in R with just 6 lines of code 1 library(Causal. Impact) 2 pre. period <- as. Date(c("2011 -01 -03", "2015 -09 -14")) 3 post. period <- as. Date(c("2015 -09 -21", "2017 -03 -19")) 4 impact <- Causal. Impact(data, pre. period, post. period) 5 plot(impact) 6 summary(impact)

17 Results can be plotted and summarised in a table Cumulative panel only makes

17 Results can be plotted and summarised in a table Cumulative panel only makes sense when the metric is additive, such as clicks or the number of orders, but not in the case when it is a share price

18 The package can even write a report for you!

18 The package can even write a report for you!

19 Additional considerations ● It is important that covariates included in the model are

19 Additional considerations ● It is important that covariates included in the model are not themselves affected by the event. For each covariate included, it is critical to reason why this is the case. ● The model can be validated by running the Causal Impact analysis on an ‘imaginary event’ before the actual event. We should not be seeing any significant effect, and actual and predicted lines should match reasonably closely before the actual event.

20 References ● K. H. Brodersen, F. Gallusser, J. Koehler, N. Remy, S. L.

20 References ● K. H. Brodersen, F. Gallusser, J. Koehler, N. Remy, S. L. Scott, (2015). Inferring Causal Impact Using Bayesian Structural Time. Series Models. https: //research. google. com/pubs/pub 41854. html. ● S. L. Scott, H. Varian, (2013). Predicting the Present with Bayesian Structural Time Series. https: //people. ischool. berkeley. edu/~hal/Papers/2013/predpresent-with-bsts. pdf.

21 Thank you!

21 Thank you!

Ana Pinto Ana Mota Ana Nunes Ana JorgeAna Pinto Ana Mota Ana Nunes Ana Jorge
Causal Inference Causal Graph D Drug C CureCausal Inference Causal Graph D Drug C Cure
Causal Inference Chapter 1 Preliminaries Statistical and CausalCausal Inference Chapter 1 Preliminaries Statistical and Causal
NATIONAL REPORT FOR GREECE I A Daglis NationalNATIONAL REPORT FOR GREECE I A Daglis National
Exact Inference Inference Basic task for inference ComputeExact Inference Inference Basic task for inference Compute
Chapter 7 Statistical Inference Statistical Inference Statistical inferenceChapter 7 Statistical Inference Statistical Inference Statistical inference
Guided Reading Skills Inference Inference What is InferenceGuided Reading Skills Inference Inference What is Inference
Bayesian Inference Review 42512 Frequentist inference Bayesian inferenceBayesian Inference Review 42512 Frequentist inference Bayesian inference
Fuzzy Inference Systems Fuzzy Inference Systems Fuzzy inferenceFuzzy Inference Systems Fuzzy Inference Systems Fuzzy inference
Year 2 Reading Inference What is inference InferenceYear 2 Reading Inference What is inference Inference
ANA KARTLAR VE KASALAR ANA KARTLAR 2 ANAANA KARTLAR VE KASALAR ANA KARTLAR 2 ANA
Causal Loop Diagrams Causal loop diagrams can beCausal Loop Diagrams Causal loop diagrams can be
Causal Diagrams and the Identification of Causal EffectsCausal Diagrams and the Identification of Causal Effects
Causal Directed Acyclic Graphs DAG Causal Diagrams 2013Causal Directed Acyclic Graphs DAG Causal Diagrams 2013
Using Causal Conjunctions Causal conjunctions are used toUsing Causal Conjunctions Causal conjunctions are used to
Causal Networks Denny Borsboom This is a causalCausal Networks Denny Borsboom This is a causal
Using Causal Conjunctions Causal conjunctions are used toUsing Causal Conjunctions Causal conjunctions are used to
THEORY CONSTRUCTION 8 Causal Models Dina Abdelhafez CausalTHEORY CONSTRUCTION 8 Causal Models Dina Abdelhafez Causal
Causal Diagrams and the Identification of Causal EffectsCausal Diagrams and the Identification of Causal Effects
Experimental Design making causal inferences Causal and EffectExperimental Design making causal inferences Causal and Effect