Demonstrating that an HIV Vaccine Lowers the Risk and/or Severity of HIV infection D.Mehrotra1*, X.Li1, P.Gilbert2 Merck Research Laboratories, Blue Bell, PA 1 Fred Hutchinson Cancer Research Center & Univ. of Washington, Seattle, WA 2 Adapted from Mehrotras talk at ENAR, Austin, TX March 21, 2005 Biostat 578A Lecture 6 Outline HIV vaccine POC efficacy trial BOI vs. Simes method Adjusting for selection bias BOI vs. adjusted Simes method Concluding remarks 2 HIV Vaccine POC Efficacy Trial Design - Randomized, double-blind, multinational trial - MRKAd5 gag/pol/nef versus placebo (1:1) - 1500 subjects at high risk of becoming HIV+ - Continue until 50 events (HIV infections) accrue Co-Primary Endpoints - HIV infection status (infected/uninfected) - Viral load set-point (mean of log10 HIV RNA at 2 and 3 months after HIV+ diagnosis) Note: lowering of viral load set-point will presumably prevent or delay the onset of AIDS

3 POC Efficacy Trial (continued) Null Hypothesis: Vaccine is same as Placebo VE = 0 and = 0 Alternative Hypothesis: Vaccine is better than Placebo VE > 0 and/or > 0 true HIV infection rate for VACCINE 1 VE = true HIV infectionrate for PLACEBO = true difference in mean viral load setamong point infected subjects [placebo vaccine] Proof-of-concept (POC) is established if the composite null hypothesis is rejected at = 5% 4 POC Efficacy Trial: Data Set-Up Number randomized Number HI V inf ected Proportion inf ected Viral load set-points of inf ected subj ects (log10 copies/ ml) Vaccine Placebo Nv nv nv Nv Np y1( v )

yn( vv ) y1( p ) yn( p ) p np np Np 5 Establishing POC: BOI vs. Simes Method Burden- of- I llness (BOI ) (Chang, Guess, Heyse, 1994) nv y Difference in BOI per subj ect: Let Z BOI T E T | nv n p , H 0 Var (T | nv n p , H 0 ) np (v) i T i 1 Nv ( p) y i

i 1 Np (see CGH) [unconditional test] Rej ect null if p value Pr Z Z BOI | H 0 .05 Simes method (Simes, 1986) p1 p-value f or inf ection endpoint [unconditional test] p2 p-value f or viral load endpoint [conditional test] Rej ect null if p value min(max( p1 , p2 ), 2 min( p1 , p2 )) .05 (Similar results with Fishers combined p-value method) 6 Power (%) to Reject the Composite Null Hypothesis (assuming = 1 log10 copies/ml) VE = 0% Power (%) 100 80 80 60 60 40 40 20 20 0 0 10 30 50

70 90 10 30 50 70 90 VE = 60% 100 80 Power (%) VE = 30% 100 BOI Simes' 60 40 20 0 *Viral load reduction is 1 log10 in all plots 10 30 50 70 Number of HIV Infections 90 7 Adjusting for Selection Bias Simulations led us to choose Simes over BOI for the POC

efficacy trial. However Test for viral load component in Simes method: - Is restricted to subjects that are selected based on a post-randomization outcome (HIV infection) can suffer from selection bias. - Assesses mixture of (i) causal effect of vaccine (ii) effect of variables correlated with VL are unevenly distributed among the infected subgroups. and that References Rubin (1978), Rosenbaum (1984), Robins and Greenland (1992), Frangakis and Rubin (2002) 8 Adjusting for Selection Bias (continued) Proposed approach - Adjust the viral load test for plausible levels of selection bias such that rejection of the null hypothesis becomes harder. - If the adjusted test is significant, then we have robust evidence of a causal vaccine effect. Hudgens, Hoering, Self (Statistics in Medicine, 2003) Gilbert, Bosch, Hudgens (Biometrics, 2003) [GBH] Mehrotra, Li, Gilbert (Biometrics, 2006) Adjustment is derived via the principal stratification framework of causal inference (Frangakis and Rubin, Biometrics, 2002) 9 Adjusting for Selection Bias (continued) Subjects infected under placebo {Si(p)=1} partition into the protected and always-infected principal

strata Principal Stratum Potential infection outcome under Z Potential VL outcome under Z given Si(z) = 1 Protected Si(v) = 0, Si(p) = 1 undefined Yi(p,prot.) Always infected Si(v) = 1, Si(p) = 1 Yi(v) Yi(p,alw.inf.) Z = assigned treatment To assess a causal vaccine effect: we need to compare Yi(v) (= Yi(v, alw.inf.)) with Yi(p, alw.inf.), but the placebo VLs are a mixture of Yi(p, prot.) and Yi(p, alw. inf.) How to identify the distribution of Yi(p, alw.inf.)? 10 Adjusting for Selection Bias (continued) fp(y) = (VE)fp(prot)(y) + (1-VE)fp(alw.inf)(y) fp(alw.inf)(y) = [w(y)/(1-VE)]fp(y) where w(y) = Pr{Si(v)=1|Yi(p)=y, Si(p)=1} is the unknown probability that a placebo infectee with VL setpoint y would have been infected if given vaccine. VE and fp(y) can be estimated from the data, but not w(y). Solution: assume a known model for w(y). 11

Adjusting for Selection Bias (continued) GBH (2003) assume a logistic model for w(y): wi, = w(yi|,) = exp( + yi)/{1+exp( + yi)}, inp where is a fixed (pre-set) parameter: (i) = 0 wi, = 1 VE for all i (ii) < 0 for a 1-unit decrease in Yi(p), the odds of being in the always infected stratum increase multiplicatively by exp(-) (iii) is a constant satisfying Fp(|) = 1 For a given , fp(alw.inf) can now be estimated. 12 VL Distributions for the Protected and Always Infected Principal Strata Implied by the Logistic Model for wi(y) 2 3 4 5 log10 VL 6 7 0.4 0.3 0.2 1-VE VE VE

0.0 0.0 1-VE = - (e- = ) 0.1 0.2 probability density 0.3 0.4 1-VE 0.1 = -2 (e- = 7.4)) 0.1 probability density 0.2 0.3 VE 0.0 probability density 0.4 =0 (e- = 1) 2

3 4 5 log10 VL 6 7 2 3 4 5 6 7 log10 VL = 0: vaccine does not selectively protect subjects same distribution for Yi(p, prot.) and Yi(p, prot.) < 0: vaccine selectively protects subjects with higher VLs selection bias leads to biased estimation of the causal effect that makes the vaccine look poorer 13 Adjusting for Selection Bias (continued) Adjust the viral load test in Simes method: 1) Fix the selection bias parameter 0. 2) Adjust (reduce) all the VLs of placebo infectees: y

( p )* i , n p yi( p ) ( p) yi i1 np np i1 w yi | , y np i 1 ( p) i w yi | , is non-parametric m.l.e. of T ( p )* (v) 3) { Let = Wilcoxon rank { y }in sum statistic comparing y } i i

i nv with p 14 Adjusting for Selection Bias (continued) When VE = 0, T is the Wilcoxon rank sum statistic used for the unadjusted VL test. The distribution of T is intractable, so the p-value for the adjusted VL test ( = p2, ) is obtained using a non-parametric bootstrap. Adjusted Simes method: for the specified , reject the composite null hypothesis if max( p1 , p2, ) or min( p1 , p2, ) / 2 Robust evidence of a causal vaccine effect on either the infection or VL endpoint: reject the composite null hypothesis using the adjusted Simes method for all plausible values of . 15 BOI vs. Simes Method: Power (%) Assuming = 1 log10 copies/ml VE = 0% P o w e r (% ) 100 80 80 60 60 40 40

20 20 0 0 10 50 70 90 10 30 50 70 90 VE = 60% 100 P o w e r (% ) 30 VE = 30% 100 80 BOI Unadjusted Simes' (Beta=0) Adjusted Simes' (Beta=-1) Adjusted Simes' (Beta=-2) Adjusted Simes' (Beta=-Inf) 60

40 20 0 *Viral load reduction is 1 log10 in all plots 10 30 50 70 Number of HIV Infections 90 16 Concluding Remarks The selection bias-adjusted Simes method is more powerful than the BOI method, unless VE is large (unlikely for a CMI-based HIV vaccine). 50 events will provide at least 80% power to establish POC provided: VE 60% or 0.75 c/ml: unadjusted Simes method VE 60% or 1.0 c/ml: adjusted Simes method. An -spending interim analysis after 30 events is proposed (details omitted here). Estimated time between 30 and 50 events is 9-15 months. 17 REFERENCES 1. Chang MN, Guess HA, Heyse JF (1994). Reduction in the burden of illness: a new efficacy measure for prevention trials. Statistics in Medicine, 13, 1807-1814. 2. Fisher RA (1932). Statistical Methods for Research Workers. Oliver and Boyd, Edinburgh and London. 3. Frangakis CE, Rubin DB (2002). Principal Stratification in Causal Inference. Biometrics, 58, 21-29. 4. Gilbert PB, Bosch RJ, Hudgens MG (2003). Sensitivity analysis for the assessment of causal vaccine effects on viral load in HIV vaccine clinical trials. Biometrics, 59, 531-541.

5. Hudgens MG, Hoering A, Self SG (2003). On the analysis of viral load endpoints in HIV vaccine trials. Statistics in Medicine, 22, 2281-2298. 6. Mehrotra DV, Li X, Gilbert PB. Dual-endpoint evaluation of vaccine efficacy: Application to a proof-of-concept clinical trial of a cell mediated immunity-based HIV vaccine. Biometrics, in press. 7. Robins JM, Greenland S (1992). Identifiability and exchangeability of direct and indirect effects. Epidemiology, 3, 143-155. 8. Rosenbaum PR (1984). The consequences of adjustment for a concomitant variable that has been affected by the treatment. The Journal of the Royal Statistical Society, Series A, 147, 656-666. 9. Rubin DB (1978). Bayesian inference for causal effects: the role of randomization. The Annals of Statistics, 6, 34-58. 18 APPENDIX Arguments against a selection-bias adjustment: - POC (not phase III) trial: not essential to precisely characterize the vaccine effect. - VE (and hence selection bias) anticipated to be small. - If vaccine prevents infection only for less virulent strains, then selection bias is more likely to make placebo look better than vaccine when comparing VLs, so the unadjusted test is already conservative from a causal inference perspective! Arguments for a selection-bias adjustment: - Will we really proceed to phase III without robust evidence of a causal vaccine effect? - To satisfy statisticians who are wary of any nonrandomized comparison. 19 Hypothetical Example I nf ected/ Enrolled VL set-point (log10 copies/ ml) Vaccine Group 22/ 750 2.26 3.98

2.55 4.02 2.57 4.17 2.68 4.17 2.82 4.44 3.13 4.69 3.17 4.83 3.41 3.45 3.64 3.65 3.74 3.82 3.92 3.95 Placebo Group 28/ 750 2.79 4.45 3.26 4.57 3.32 4.58 3.51 4.66 3.72 4.92 4.02 4.99 4.08 5.18 4.10 5.19 4.10 5.20 4.14 5.23 4.20 5.52 4.21

5.60 4.24 5.62 4.26 4.40 VEobs 1 22 / 750 / 28 / 750 21% , p1 =0.240 (binomial test) obs 4.43 3.59 =0.84 log10 c/ ml, p2 =0.0001 (rank-sum test) Simes p-value =0.0002*, BOI p-value =0.062 20 Hypothetical Example (continued) Vaccine Group 22/ 750 I nf ected/ Enrolled VL set-point (log10 c/ ml) 2.26 2.55 2.57 2.68 2.82 3.13 3.17 3.41 3.45 3.64 3.65 3.74 3.82 3.92 X 0 0 0 0 0 0 0 0 0

0 0 0 0 0 3.95 3.98 4.02 4.17 4.17 4.44 4.69 4.83 Placebo Group 28/ 750 X 0 0 0 1 1 0 1 1 2.79 3.26 3.32 3.51 3.72 4.02 4.08 4.10 4.10 4.14 4.20 4.21 4.24 4.26 X 0 0

0 0 0 0 0 0 0 0 0 0 0 0 4.40 4.45 4.57 4.58 4.66 4.92 4.99 5.18 5.19 5.20 5.23 5.52 5.60 5.62 X 1 1 0 1 0 1 1 0 1 1 1 1 1 1 X =unobserved covariate (e.g., a genetic trait) A higher proportion of placebo inf ectees have X=1, and

subj ects with X =1 tend to have higher viral loads. Did vaccine cause lower VLs or is the observed vaccine effect an artifact of the imbalance in the X distribution? 21 Hypothetical Example Revisited Assigning weights to the VLs in the Placebo Group log10 VL 2.79 3.26 3.32 3.51 3.72 4.02 4.08 4.10 4.10 4.14 4.20 4.21 4.24 4.26 weight (wi,) 0 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 2 0.99 0.98 0.98

0.97 0.96 0.93 0.92 0.92 0.92 0.92 0.91 0.90 0.90 0.89 1 1 1 1 1 1 1 1 1 1 1 1 1 1 log10 VL 4.40 4.45 4.57 4.58 4.66 4.92 4.99 5.18 5.19 5.20 5.23 5.52 5.60 5.62 weight (wi,)

0 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 2 0.86 0.85 0.82 0.82 0.79 0.69 0.66 0.57 0.57 0.56 0.55 0.41 0.37 0.36 1 1 1 1 1 1 1 1 0 0 0 0 0 0

22 Hypothetical Example Revisited 0.1 1-tailed p-value 0.075 Robust evidence of a causal vaccine effect on VL 0.05 0.025 = Inf 0 1 2 3 4 5 6 7 8 9 10 OR=exp 23