Statistical considerations in small proofofconcept trials including crossover

  • Slides: 35
Download presentation
Statistical considerations in small proof-of-concept trials, including crossover designs Stephen Senn (c) Stephen Senn

Statistical considerations in small proof-of-concept trials, including crossover designs Stephen Senn (c) Stephen Senn 2008 1

 • People look down on marketing men • It’s not true that they

• People look down on marketing men • It’s not true that they are not scientists • They work in sell biology • I would like to take this opportunity to draw your attention to a book I rather like • In fact I wrote it myself (c) Stephen Senn 2008 2

Outline • • • Decision analysis and proof of concept Value of information perspective

Outline • • • Decision analysis and proof of concept Value of information perspective Place of cross-over trials Carry-over The potential for cross-over trials in studying individual response (c) Stephen Senn 2008 3

A Model Probability proof of concept (POC) study successful Probability proof of efficacy study

A Model Probability proof of concept (POC) study successful Probability proof of efficacy study (POE) successful if POC successful Probability POE study successful if POC unsuccessful Probability POE study successful Cost of POC including any lost sales through extra delay Cost of POE study Expected NPV revenue if POE initiated immediately and successful Value of strategy of POE only Value of strategy of POC + POE Value of POC study (c) Stephen Senn 2008 4

Model Continued (c) Stephen Senn 2008 5

Model Continued (c) Stephen Senn 2008 5

Example Value of two strategies plotted against , the probability POE successful if POC

Example Value of two strategies plotted against , the probability POE successful if POC successful (c) Stephen Senn 2008 6

(c) Stephen Senn 2008 7

(c) Stephen Senn 2008 7

Value of Biomarker Information in Terms of Posterior Variance • Suppose that over all

Value of Biomarker Information in Terms of Posterior Variance • Suppose that over all products for this indication the correlation of true therapeutic and biomarker outcomes is 0. 9 • Let the prior means be zero in this class • Let the prior variances be 1 • Let the data variance of a minimal experiment be also 1 – Implies prior information equivalent to one minimal experiment (c) Stephen Senn 2008 8

Here n is the number of minimal experiments we run Of course we expect

Here n is the number of minimal experiments we run Of course we expect a biomarker experiment to be cheaper than a therapeutic one Nevertheless note that fairly rapidly there is no interest in getting further biomarker information 9

A Serious Warning to Statisticians In the mathematical formulation of any problem it is

A Serious Warning to Statisticians In the mathematical formulation of any problem it is necessary to base oneself on some appropriate idealizations and simplification. This is, however, a disadvantage; it is a distorting factor which one should always try to keep in check, and to approach circumspectly. It is unfortunate that the reverse often happens. One loses sight of the original nature of the problem, falls in love with the idealization, and then blames reality for not conforming to it. De Finetti 1975 ‘The only way that human beings could ever have survived as a species for long as we have is that we’ve developed another kind of decisionmaking apparatus that’s capable of making very quick judgements based on very little information. Malcolm Gladwell, Blink, 2005 (c) Stephen Senn 2008 10

My Gloomy Take on This • We don’t really understand this topic • There

My Gloomy Take on This • We don’t really understand this topic • There may be less value in proof of concept studies than we propose • Therapeutic studies may be valuable even if they have low power • There is no point in undertaking POC studies unless you can see circumstance under which they would cause you to cancel projects (c) Stephen Senn 2008 11

Appropriate Attitudes for Crossover Trials • They are not suitable for all indications and

Appropriate Attitudes for Crossover Trials • They are not suitable for all indications and questions • They are extremely valuable for some indications and questions • Carry-over has to be dealt with by washout • Don’t pre-test for carry-over • Don’t rely on classical statistical approaches to carry-over • Cross-over trials have great potential in investigating individual response (c) Stephen Senn 2008 12

Carry-over Definition: Carry-over is the persistence (whether physically or in terms of effect) of

Carry-over Definition: Carry-over is the persistence (whether physically or in terms of effect) of a treatment applied in one period in a subsequent period of treatment. If carry-over applies in a cross-over trial we shall, at some stage, observe the simultaneous effects of two or more treatments on given patients. We may, however, not be aware that this is what we are observing and this ignorance may lead us to make errors in interpretation. (c) Stephen Senn 2008 13

The simple carry-over model. This is a very popular model amongst “applied” statisticians of

The simple carry-over model. This is a very popular model amongst “applied” statisticians of a mathematical bent. The model assumes that if a carry-over effect is present 1) it lasts for one period exactly 2) it depends on the engendering treatment only and not on the perturbed treatment. (c) Stephen Senn 2008 14

Three Period Bioequivalence Designs • Three formulation designs in six sequences common. • Subjects

Three Period Bioequivalence Designs • Three formulation designs in six sequences common. • Subjects randomised in equal numbers to six possible sequences. For example, 18 subjects, three on each of the sequences ABC, ACB, BAC, BCA, CAB, CBA. – A = test formulation under fasting conditions, – B = test formulation under fed conditions – C = reference formulation under fed conditions. – (c) Stephen Senn 2008 15

Weights for the Three Period Design: not Adjusting for Carry-over (c) Stephen Senn 2008

Weights for the Three Period Design: not Adjusting for Carry-over (c) Stephen Senn 2008 16

Properties of these weights • Sum to 0 in any column, – eliminates the

Properties of these weights • Sum to 0 in any column, – eliminates the period effect. • Sum to 0 in any row – eliminates patient effect • Sum to 0 over cells labelled A – A has no part in definition of contrast • Sum to 1 over the cells labelled B and to -1 over the cells labelled C – Estimate contrast B-C (c) Stephen Senn 2008 17

Weights for the Three Period Design: Adjusting for Carry-over B-C contrast: illustration of treatment

Weights for the Three Period Design: Adjusting for Carry-over B-C contrast: illustration of treatment effect and elimination of period and patient effects (c) Stephen Senn 2008 18

Weights for the Three Period Design: Adjusting for Carry-over Illustration of elimination of ‘carry-over’

Weights for the Three Period Design: Adjusting for Carry-over Illustration of elimination of ‘carry-over’ effects (c) Stephen Senn 2008 19

Have We Got Something for Nothing? • Sum of squares weights of first scheme

Have We Got Something for Nothing? • Sum of squares weights of first scheme is 1/3 (or 4/12) • Sum of squares of weights of second scheme is 5/12 • Given independent homoscedastic within- patient errors, there is thus a 25% increase in variance • Penalty for adjusting is loss of efficiency (c) Stephen Senn 2008 20

The difference between mathematical and applied statistics is that the former is full of

The difference between mathematical and applied statistics is that the former is full of lemmas whereas the latter is full of dilemmas (c) Stephen Senn 2008 21

The Dangers of Pre-testing • Situation with AB/BA design – Two-stage procedure is very

The Dangers of Pre-testing • Situation with AB/BA design – Two-stage procedure is very badly biased – CARRY and PAR are highly correlated • 1/ 2 < < 1 • Three treatment design – Same problem – Carry-over and adjusted estimates correlated • = 0. 45 (c) Stephen Senn 2008 22

The Phoenix Bioequivalence Trials • • • Analysed by D’Angelo, Potvin & Turgeon *

The Phoenix Bioequivalence Trials • • • Analysed by D’Angelo, Potvin & Turgeon * 20 drug classes 1989 -1999 12 or more subjects 96 three period designs 324 two period designs D'Angelo, G. Potvin, D. Turgeon, J. J Biopharm Stats, 11, 27 -36, 2001 (c) Stephen Senn 2008 23

Three Treatment Designs P-Values for Carry-Over “Significant” results in bold Senn, S. J. ,

Three Treatment Designs P-Values for Carry-Over “Significant” results in bold Senn, S. J. , G. D'Angelo, et al. (2004). "Carry-over in cross-over trials in bioequivalence: theoretical concerns and empirical evidence. " Pharmaceutical Statistics 3(2): 133 -142. 24

Two Treatment Designs “Significant” results in bold 25

Two Treatment Designs “Significant” results in bold 25

Test of Uniformity of P-Values (c) Stephen Senn 2008 26

Test of Uniformity of P-Values (c) Stephen Senn 2008 26

Galling as this may appear to statisticians, the cure for carry-over is more biological

Galling as this may appear to statisticians, the cure for carry-over is more biological and pharmacological understanding not more statistics (c) Stephen Senn 2008 27

Conclusions • Distribution of P-values uniform – no evidence of carry-over • Carry-over a

Conclusions • Distribution of P-values uniform – no evidence of carry-over • Carry-over a priori implausible – presence testable by assay • No point is testing for it – leads to bias • Or adjusting for it – increased variance (c) Stephen Senn 2008 28

Possible Strategy • Run multi-period cross-overs • Patient by treatment interaction becomes identifiable •

Possible Strategy • Run multi-period cross-overs • Patient by treatment interaction becomes identifiable • This provides an upper bound for gene by treatment interaction – Because patients differ by more than their genes (c) Stephen Senn 2008 29

Second cross-over Responders Non. Total Responders First cross- Responders 24 over (c) Stephen Senn

Second cross-over Responders Non. Total Responders First cross- Responders 24 over (c) Stephen Senn 2008 0 24 Non 0 Responders 8 8 Total 8 32 24 30

Second cross-over Responders Non. Total Responders First cross- Responders 18 over (c) Stephen Senn

Second cross-over Responders Non. Total Responders First cross- Responders 18 over (c) Stephen Senn 2008 6 24 Non 6 Responders 2 8 Total 8 32 24 31

Advantages and Disadvantages PRO • Cheap • Low tech • Insight into sources of

Advantages and Disadvantages PRO • Cheap • Low tech • Insight into sources of variation gained (c) Stephen Senn 2008 CON • Only suitable for chronic diseases • Demanding of patient’s time • Unglamorous • Does not produce diagnostic patents 32

An Overlooked Source of Genetic Variability • Humans may be classified into two important

An Overlooked Source of Genetic Variability • Humans may be classified into two important genetic subtypes. • One of these suffers from a massive chromosomal deficiency. • This is expressed in. – Important phenotypic differences. – A massive disadvantage in life expectancy. • Many treatment strategies take no account of this. • The names of these subtypes are. . . (c) Stephen Senn 2008 33

Men and Women (c) Stephen Senn 2008 34

Men and Women (c) Stephen Senn 2008 34

A Difficult Decision • You have $100 • Should you spend it on beer?

A Difficult Decision • You have $100 • Should you spend it on beer? – US 20 beers – UK 15 beers • Or on books? • In particular 1 book • Have I mentioned this before? (c) Stephen Senn 2008 35