Analysis of Multiple End Points in Clinical trials

Analysis of Multiple End Points in Clinical trials By Trusha Patel and Sirisha Davuluri

Article Details “An efficient method for accommodating potentially underpowered primary endpoints” ◦ By Jianjun (David) Li and Devan V. Mehrotra” 2

Contents 1. 2. 3. Introduction Procedure 1: PAAS Method Procedure 2: 4 A Method a. Independent Primary Endpoints b. Correlated Primary Endpoints 4. 5. 6. 7. 8. Performance of both Procedures (Probability of achieving positive trial) Conclusion Appendix References Questions? 3

Introduction Clinical trials generally classify the endpoints into primary, secondary and exploratory types. Primary Endpoints: � Primary endpoints address primary objectives of the trial. They are usually few but are clinically most relevant to the disease and the treatment under study. They assess the main clinical benefits of the treatment. 4

Introduction Secondary Endpoints: � Secondary endpoints characterize extra benefits of the treatment under study after it has been demonstrated that the primary endpoints show clinically meaningful benefits of the treatment. Explanatory Endpoints: � Exploratory endpoints are usually not prospectively planned and are generally not rigorously evaluated like primary and secondary endpoints. 5

Introduction Different approaches have been used to specify a clinical decision rule for trials that have more than one primary endpoint. Significant results can be required for each of several primary endpoints to consider a trial ‘‘positive. ” 6

At least One Procedure � To demonstrate the treatment’s superiority on at least one endpoint. � If each multiple endpoint is independently clinical relevant, the multiple endpoint problem can be formulated as a multiple testing problem, and the trial is declared positive if at least one significant effect is detected. � When significant results are required for more than one but not all of multiple primary endpoints for a trial to be considered positive, correction for multiplicity is also necessary, and this must take into account the total number of endpoints and the number required for the trial to be considered positive. 7

Different Methods ØWeighted Bonferroni procedure ØProspective Alpha Allocation Scheme (PAAS Method) ØAdaptive Alpha Allocation Approach (4 A Method) ØBonferroni-type parametric procedure ØFallback-type parametric procedure For this project, we are mainly focusing on PAAS Method and 4 A Method by considering two Primary Endpoints (A : well powered, B : potentially underpowered). 8

Procedure 1: PAAS Method Procedure: � Moye’(2000) proposed the ‘‘prospective alpha allocation scheme” for preserving Type I error rates at acceptable levels when there are multiple endpoints. � Assume that p-values for the individual endpoints are independent. � The experiment-wise type I error rate = α should be capped, say at 0. 05, and the fraction of the Type I error rate α allocating α 1* = α − ε, ε= 0. 01 to endpoint A α 2* = 1 - (1−α)/(1−α 1*) to endpoint B 9

PAAS Method Advantages: � PAAS is a simple and appealing method. � Prospectively allocating alpha in this method preserves the experiment-wise Type I error rate and makes it possible to consider a treatment efficacious when the null hypothesis is not rejected for the primary endpoint but is rejected for one or more of the secondary endpoints. Disadvantage: The prospective alpha allocation scheme preserves the experiment-wise Type I error rate at a higher rate than is customarily accepted. 10

Procedure 2: 4 A Method Procedure: � Li and Mehrotra (2008) proposed a multiple testing procedure, which they referred to as the adaptive alpha allocation approach or 4 A procedure. � Consider a clinical trial with m endpoints and assume that the endpoints are grouped into two families. The first family includes m 1 endpoints that are adequately powered and the second family includes m 2 potentially underpowered endpoints (m 1 + m 2 = m). 11

4 A Method - Independent Primary Endpoints Let’s consider 2 multiple points for illustration of the procedure: � Let p. A and p. B denote the p-value for endpoint A and endpoint B, respectively. � In the article, they assume that p-values are two-tailed (to mimic common practice) and that ‘statistically significant’ results are in the direction of interest; however all the methods discussed can also be used with one-tailed p-values. � In their proposed adaptive alpha allocation approach, endpoint A is tested at the pre specified level α 1 = α − ε, and endpoint B is tested at the adaptive level. 12

4 A Method - Independent Primary Endpoints � � � If A achieves statistical significance (i. e. p. A ≤α 1), then B is tested at level α 2(p. A, α 1, α) = α; If A fails to achieve statistical significance (i. e. p. A>α 1), then B is tested at level 0 <α 2(p. A, α 1, α) ≤ α 1, which is close (or equal) to α 1 if p. A is not much greater than α 1, but approaches zero as p. A increases. Sample values of α 2 as a function of observed p. A are provided in Table I (ρ = 0 column) � Table I shows the alpha allocation for Primary endpoint B. 13

Table I 14

Extending 4 A to the scenario of three or more independent primary endpoints � Let there be M primary end points out of which the first m are assumed to be powered by type A endpoints and the rest of the M-m are underpowered or type B endpoints. � Here, they suppose, the p-values for the type A endpoints are tested for statistical significance at an overall level α 1 = α – ε using Hochberg’s method. � The p-values for the type B primary endpoints are then measured at and overall adaptive level α 2(p(m), α 1, α, m) using Hochberg’s method 15

Extending 4 A to the scenario of three or more independent primary endpoints Note : (3) and (4) are generalizations of (1) and (2); setting m=1 in (3) and (4) leads to (1) and (2), respectively. Advantage: The remaining endpoints are tested at a generally higher significance level, which improves their power. 16

4 A Method- Correlated Primary Endpoints Procedure: � If the endpoints are correlated, then formula (1) and (3) cannot be used directly because the FWER may be inflated. Therefore, an adjustment may be needed in the formulas presented for correlated endpoints. � The 4 A procedure is readily implemented using Table I if ρ is known. � For the case of unknown ρ, See Appendix-I. Extending 4 A to the relatively uncommon scenario of M>2 primary endpoints is notably more challenging in the case of correlated endpoints compared with independent endpoints. 17

Performance of both Procedures (Probability of achieving Positive Trial) Let’s compare the performances of the PAAS and 4 A methods for our motivating scenario of two primary endpoints: A-well powered and B-underpowered. Goal: To improve the probability of correctly achieving a positive trial, i. e. of rejecting at least one of the two null hypotheses, while ensuring that the family wise type I error rate is at most α. Table A summarizes the probabilities of achieving a positive trial using the PAAS and 4 A methods when the two endpoints are either independent (ρ=0) or correlated (ρ=0. 5); results are based on 10, 000 simulations. (See Table III in Appendix-II) 18

Table A 19

Performance of both Procedures (Probability of achieving Positive Trial) Case 1 -7: Primary endpoint A: Adequately Powered Primary endpoint B: Underpowered Case 8 -17: Primary endpoint A: Underpowered Primary endpoint B: Underpowered For PAAS method, α 1* = 0. 04 and α 2* = 0. 010; and For 4 A method, α 1 = 0. 04 and α 2≡α 2(p. A, α 1, α, ρ) is calculated adaptively as described before. 20

Results � Use of 4 A method increased the probability of achieving a positive trial compared with PAAS. o For Independent endpoints, the absolute gain in power ranged from 0. 9 % (case 7; 77. 9 versus 78. 8%) to 3. 7% (case 8; 78. 8 versus 82. 5%) compared with PAAS. o For Correlated endpoints A and B (ρ=0. 5); the power for 4 A was numerically higher than that for PAAS, For example, the power gains of 4 A ranged from 0. 1 per cent (case 7; 77. 4 versus 77. 5 per cent) to 2. 9 per cent (case 8, case 14) compared with PAAS. 21

Results � The probability of achieving a positive trial by pre specifying both A and B as primary endpoints and using 4 A for the analysis is about as good as (cases 7 and 13) or notably better than (cases 1 and 8) the marginal power for endpoint A. o In other words, 4 A enables us to accommodate the underpowered endpoint B in the primary family while essentially preserving or substantially enhancing the likelihood of achieving a positive trial compared with the strategy of using only A as the primary endpoint with α=0. 05. 22

Conclusion � There is no optimal strategy for the use of significance testing with multiple endpoints, especially when dealing with clinically persuasive endpoints that may be underpowered, but based on the simulation results that are provided in the article, we can conclude that 4 A method performs better than PAAS method. o The power advantage of 4 A over PAAS was greater when the marginal power for endpoint B was lower (higher), regardless of whether the endpoints were independent or correlated. 23

Appendix-I 24

Appendix-II 25

References � http: //www. ncbi. nlm. nih. gov/pubmed/18759248 � Multiple Testing Problems in Pharmaceutical Statistics, by Alex Dmitrienko, Ajit Tamhane and Frank Bretz. 26

Questions 27