GAGAN Generative Graph Adversarial Networks for Causal Discovery

GAGAN Generative Graph Adversarial Networks for Causal Discovery John Moss

Causal Discovery • Given some data, can we construct a Bayesian Network that captures the causal relations in the process that generated the data? • “Correlation is not causation” – but conditional independence relations (sometimes) are! • We need a few assumptions: causal sufficiency, causal faithfulness, causal Markov condition SES Wine consumption Heart Disease

Causal Discovery • Given some data, can we construct a Bayesian Network that captures the causal relations in the process that generated the data? • “Correlation is not causation” – but conditional independence relations (sometimes) are! • We need a few assumptions: causal sufficiency, causal faithfulness, causal Markov condition SES Wine consumption Heart Disease

Causal Discovery • Given some data, can we construct a Bayesian Network that captures the causal relations in the process that generated the data? • “Correlation is not causation” – but conditional independence relations (sometimes) are! • We need a few assumptions: causal sufficiency, causal faithfulness, causal Markov condition SES Wine consumption Heart Disease

Constraint-based Solutions • The conditional independence relations in the data place constraints on the Bayesian Network – let’s build a graph that satisfies those constraints • Not guaranteed to be the correct causal graph – but guaranteed to have the same structure (if not edge orientations) • This is NP-Hard, but efficient approximations exist (e. g. FGES)

NOTEARS •

GAGAN • What if we use a GAN discriminator rather than Least Squares objective? • One way to think about GANs – they learn arbitrary probability distributions from data Did I fool the discriminator? Am I acyclic? h(W) – d_loss A B Which is which? Generator C Discriminator (DNN) D Data (binary crossentropy loss)

Experiment Methodology • Generate random graph, sample data from linear Gaussian model • Instantiate empty graph as generator G • Instantiate discriminator D DNN with random weights • Feed 128 samples from G into D • Backprop train D according to binary cross-entropy loss • Update W (parameter of G) according to negative D_loss, plus cyclicity loss

GAGAN NOTEARS True

Does it work? • Surprisingly, yes! • Pros • GPUs! • Distribution agnostic • Cons • Requires large datasets to prevent overfitting discriminator (but maybe not – data augmentation a possibility) • Not particularly time efficient at low dimensionality (compared to NOTEARS, FGES) • Quantitative analysis coming

Links • Zheng et al 2018, DAGs with NOTEARS: Continuous Optimization for Structure Learning • Paper: https: //arxiv. org/abs/1803. 01422 (Poster at NIPS 2018) • Implementations • https: //github. com/jmoss 20/notears • https: //github. com/xunzheng/notears • Causal Discovery • F. Eberhardt, Introduction to the foundations of causal discovery https: //link. springer. com/article/10. 1007/s 41060 -016 -0038 -6 • Code, writeup, Jupyter notebook – http: //github. com/jmoss 20/gagan in the next few weeks

GAN Configuration •