Testing for Both Superiority and Noninferiority FDAIndustry Workshop

Testing for Both Superiority and Noninferiority FDA/Industry Workshop September 19, 2003 William C. Blackwelder Biologics Consulting Group Bethesda, Maryland E-mail wcb@boo. net

Specific Issues: 1) Test for superiority after showing noninferiority 2) Test for noninferiority after failure to show superiority 3) Finding of both inferiority and noninferiority

References Dunnett CW, Gent M (1996). An alternative to the use of two-sided tests in clinical trials. Statistics in Medicine 15, 1729 -1738. Morikawa T, Yoshida M (1995). A useful testing strategy in phase III trials: combined test of superiority and test of equivalence. Journal of Biopharmaceutical Statistics 5, 297 -306.

E ( ) S L U 0 • E (experimental), S (standard, active control) • 0 is null value of comparative measure (e. g. , difference) • Test H 0: 0 vs H 1: < 0 at level • (L, U) is 100(1 -2 )% confidence interval (CI) for • U < 0 reject H 0 E noninferior to S • U < 0 reject H*: 0 E superior to S

1) Test for superiority after showing noninferiority E ( ) S L U 0 Objective of study: Show noninferiority ( 0) • U < 0; reject H 0, conclude E not inferior to S by as much as 0 • U also < 0: Can we now reject H*: 0 and conclude that E is superior to S?

1) Test for superiority after showing noninferiority (continued) E ( ) S L U 0 YES! • We evaluate P{U < 0 & U < 0 } • Two ways to make a type I error: 1) Conclude that < 0, when 0 2) Conclude that < 0, when 0.

1) Test for superiority after showing noninferiority (continued) E ( ) S L U 0 • Now P{U < 0 & U < 0 } = P{U < 0 }. • A priori, P{U < 0 0} , by the definition of CI. Similarly, P{U < 0 0} . Then the type I error rate is at most .

2) Test for noninferiority ( 0) after failing to show superiority E ( ) S L 0 U 0 Objective of study: Show superiority of E • 0 < U < 0; cannot reject H*: 0. • But can we at least reject H 0: 0 and conclude that E is not inferior to S by as much as 0?

2) Test for noninferiority ( 0) after failing to show superiority (continued) E ( ) S L 0 U 0 YES, but 0 should be prespecified. • Type I error in this setting is a conclusion that E is noninferior to S, when 0. • Note that P{U 0 & U < 0 0} P{U < 0 0} .

3) Finding of both inferiority and noninferiority E ( ) S 0 L U 0 • Problem with “noninferiority”: often do not explicitly include the criterion 0 , but without 0, “noninferiority” is misleading • How do we interpret this? If we really mean “noninferiority ( 0)”, must conclude that E is acceptable.

3) Finding of both inferiority and noninferiority (continued) E ( ) S 0 L U 0 One possibility: add criterion that L be < 0 Encourages smaller, not larger, study – unscientific in that less information is better than more. Unless treatment effects identical, sufficiently large study will show one superior.

3) Finding of both inferiority and noninferiority (continued) E ( ) S 0 L U 0 • If treatment effects equal or nearly so (i. e. , 0), probability that L > 0 is small (= for = 0)

3) Finding of both inferiority and noninferiority (continued) E ( ) S 0 L U 0 • When possible, choose 0 to be a difference that is not important clinically – difficult to define if endpoint is death/serious morbidity. • Possible criterion, which does not encourage a smaller study: require point estimate to be less than a specified value, say 0/2.

3) Finding of both inferiority and noninferiority (continued) E ( ) S 0 L d 0/2 U 0 • New criteria: U < 0 & d < 0/2 • For normal distributions with known & equal variances, if power = 1 - for sample size 2 n (n. E = n. S= n), < , and = 0, new criterion (d < 0/2) will hold whenever U < 0 , so that no additional sample size is needed.

3) Finding of both inferiority and noninferiority (continued) E ( ) S 0 L d 0/2 U 0 • But note that under these conditions there is no problem. It is when variability is smaller than expected and > 0 that the result is most likely to occur, and the probability of the result increases as > 0 increases.

3) Finding of both inferiority and noninferiority (continued) E ( ) S 0 L d 0/2 U 0 • If a criterion in addition to the noninferiority criterion (U < 0) is desired, more work on the properties of the resulting test, incorporating the probability of meeting both criteria, is needed.