Multiple Testing of Causal Hypotheses Samantha Kleinberg NYU

Multiple Testing of Causal Hypotheses Samantha Kleinberg NYU Bioinformatics Group, Courant Institute, NYU 9/12/08 (Jointly with Bud Mishra. )

Motivation Lung Cancer ¢ ¢ It is frequently said “smoking causes lung cancer” l But, what about other ways of developing cancer, and other conditions required to develop cancer? Goal: Find details of this relationship l How probable is it that someone will get cancer if they smoke? l How long will this take to happen?

Motivation, continued ¢ Compare: A. Smoking causes lung cancer with probability ≈ 1 after 90 years l B. Smoking causes lung cancer with probability = ½ in less than 10 years. l Different implications! l ¢ Also, consider other conditions that will make cancer more likely

Neural spike train data ¢ ¢ Simulation of neural spike trains 26 neurons, 5 causal structures At each time point: l Neuron can fire randomly (dependent on noise level) l Neuron can be triggered by one of the neurons that causes it to fire Known information l Neuron has 20 time unit refractory period l Window of 20 time units after refractory period when it can activate another neuron A B time t t+20 t+40 Data from 2006 KDD workshop on temporal data mining. K. P. Unnikrishnan, Naren Ramakrishnan, P. S. Sastry.

Patterns 1 -3 Pattern 1 Pattern 2 Pattern 3

Patterns 4 and 5 Pattern 4 Pattern 5

Desiderata ¢ A (philosophically) sound notion of causality. l It should be able to work with the kinds of data that are available, in a variety of domains: • Politics, finance, biology ¢ A (logically) rigorous method of expressing these notions of causality. l l ¢ An (algorithmic) automated method for finding all prima-facie causes l ¢ It should capture a notion of probabilistic nature of the data It should be able reason about time; time must be metric, capturing a notion of locality Model Checking A (statistically) sound method for finding all genuine causes.

Causal relationships as logical formulae ¢ ¢ Prima facie causes: earlier than effect, raise probability of effect Represent both cause and effect as formula in probabilistic temporal logic l l Example: (a^b) U c Neural spike trains: • t is 20 -40 time unit window

Causal Hypotheses ¢ ¢ Need not consider all other events; just other prima facie causes of e Why? l We are testing to see if, in the presence of some other factor, x, c is still correlated with e l For x to remove c’s influence, it must be correlated with e itself l Provides a way to narrow down the factors that must be considered Compute difference cause makes to effect But, which values of this difference are significant? 1. 2.

FDR # not rejected # rejected totals # true null H U V (F+) m 0 # non- true null H T (F-) S m 1 totals m-R R m ¢ ¢ FDR = V/R Local FDR (fdr) l For each hypothesis, compute probability of it being null

Two groups of data ¢ ¢ Two classes of prior probabilities l p 0 = Pr(uninteresting), f 0(z) density l p 1 = Pr(interesting), f 1(z) density Assume p 0 large. Mixture density: l f(z) = p 0 f 0(z) + p 1 f 1(z) Prob of being uninteresting given z-value z l fdr(z) ≈ Pr(null|z) = p 0 f 0(z) /f(z)

Steps ¢ 1. Estimate distribution of data, f(z) l ¢ 2. Define null density f 0(z) from data l ¢ ¢ E. g. splines or Poisson regression One method is to fit to central peak of data. 3. Calculate fdr(z) 4. Call Hi where fdr(zi) < threshold interesting l Common threshold is 0. 10

Causal Inference ¢ ¢ Enumerate logical formulas describing possible causes From experimental data determine prima facie causes Calculate ε for each, translate to z-values Take set of z values, calculate empirical null, label prima facie causes with z-value where fdr(z) < threshold as genuine

Examples

Neural data ¢ ¢ ¢ We used the multiple hypothesis testing framework Empirical null: N(-0. 15, -0. 39) Genuine causes have z>3

Political data ¢ ¢ Empirical null: N(0. 39, 0. 96) No genuine causes with z>0, but look at z<0 l l ¢ 3 phrases with false discovery rate, fdr<0. 1, all have z around -3 Homes, progress, lebanon What does this mean? l For example “had President Bush NOT said homes, his rating would have gone down”

Cellular data ¢ ¢ ¢ Looked at relationships between pairs of genes where relationship takes place at next unit of time Empirical null: N(-1. 00, 0. 89) Thousands of prima facie causes where f(z)< 0. 1

Conclusion ¢ ¢ ¢ New method of representing causality l Can describe probabilistic relationships with a temporal component l Allows for arbitrarily complex causes Can infer relationships with model-checking l Automated way of finding prima facie causes l Statistical method of determining genuine causality What’s next? l Inferring time between cause and effect l Magnitude of relationship l Testing on larger data sets with more complex structures Questions? samantha@cs. nyu. edu