Optimising clinical trial programs to enable more robust ...
MBSW 2012 Assessing the Impact of Unmeasured Confounding in Comparative Observational Research Doug Faries Sr Research Scientist, Eli Lilly & Company Based on Work Done by: Karen Price, Xiaomei Peng, Manjiri Pawaskar, James Stamey, John Seaman Outline The Observational Research Problem
(or Challenge) Selection Bias Confounders Physicians/patients did not select treatment at random but based on a variety of factors so Groups A and B differ in some aspects other than treatment A variable is a Confounder if it is associated with both treatment selection
and outcome The Observational Research Challenge Selection Selection Bias Bias Confounders Physicians/patients did not select treatment at random but based on a variety of factors so Groups A and B differ in some aspects other than treatment
A variable is a Confounder if it is associated with both treatment selection and outcome Measured: Information is collected within the study and statistical adjustment is possible Unmeasured: Information on the confounder is not available from the study Can We Get Causal Inference From Observational (non-randomized) Data? YES IF 3 key assumptions hold .. 1. No Unmeasured Confounders
An ASSUMPTION! Can not be definitively verified. 2. No Perfect Confounding 3. Correct Models are used Hierarchy of Evidence Vandenbrouke (2008), Concato (2000), . Increasing the Quality of Observational Research (Rubin 2007, 2008) Keep core statistical (RCT) design principles in mind that are sometimes overlooked in observational research ..
Prospective Specification Multiplicity Replication Sensitivity Analyses Current State of the Union Regarding Unmeasured Confounding What should I do about unmeasured confounding? Current State of the Union Regarding Unmeasured Confounding
What should I do about unmeasured confounding? Just mention it as a limitation in the Discussion Section and move on! There are new methods in the literature! Best Practices include sensitivity analyses EXPERT Figure 1: Unmeasured Confounding Options
Unmeasured Confounding Information Available Method None 1) Rule Out 2) IV
External 1) Bayesian 2) Algebraic Internal 1) Bayesian 2) Multiple Imputation 3) Propensity Calibration Example for Today Pawaskar M, Zagar A, Sugihara T, Shi L (2011). Healthcare
resource utilization and costs assessment of type 2 diabetes ... J Med Econ. 2011;14(1):16-27. No direct measure of glycemic control was available in the original claims database. However, after linking with a laboratory file, A1C values were obtained in a subset (about 20%) of the sample; A1C was a significant predictor of treatment selection (p<.001) but only modestly related to outcome (costs) Our Work: Sensitivity Analysis using this Internal Information Information Available: None Rule Out CONCEPT: Quantify how strong and imbalanced a confounder would need to be in order to explain (rule
out) the observed treatment difference Instrumental Variables CONCEPT: Use of an Instrument (variable associated with treatment selection but not with outcome) allows one to mimic randomization Rule-out Method Concept: Quantify how strong and imbalanced a confounder would need to be in order to explain (rule out) the observed result. This approach attempts to find all combinations of
1) the confounder-outcome relationship and 2) the confounder-treatment relationship,necessary to move the observed point estimate to zero. Rule Out Simple Model Basic (additive) Model: AMD = TTD + Bias AMD is the apparent (observed) mean treatment difference
TTD is the true (fully adjusted) mean treatment difference Bias is a function of: The imbalance of the unmeasured confounding factor between treatment groups The strength of the association between the unmeasured confounder and the outcome Rule Out Simple Spreadsheet Bias
Pc0 Pc1 AMD (Estimate) CE AMD (Lower 95% CL) CE
.95 $ 2,597 $ 2,734 $ 690 $ 726 .90 $ 2,597
$ 2,885 $ 690 $ 767 $ 2,597 $ 51,940 $ 690
$ 13,800 .. .05 Fixed Value 1 Fixed Value 2 Rule-out Method Example So, a confounder occurring in 20% more patients in Cohort A (compared to Cohort B) which
results in $15,000 higher cost per patient would eliminate the observed difference Trt A is Not Less Costly Trt A remains Less Costly Confounder - Outcome Association
File name/location Company Confidential Copyright 2000 Eli Lilly and Company Rule Out Example Information Available: External Concept: Use information external to the study (e.g. data from literature or other databases) to estimate parameters regarding unmeasured confounding (e.g. strength of association with outcome and treatment).
Bayesian Models: Incorporate the external information through a prior distribution and account for the uncertainty surrounding the external estimates External Adjustment (ctd) Some Issues: 1) Transportability 2) Correlation of Unmeasured Confounder with variables already accounted for in the analysis model
Algebraic External Adjustment Examples: - Schneeweiss et al JAGS 2005 - Schneeweiss et al CNS Drugs 2009 Information Available: Internal Concept: Use information from the patients in the study (e.g. subsample of chart review data for a retrospective claims database study) to estimate parameters regarding unmeasured confounding With Internal data can avoid transportability assumption and can account for correlation between unmeasured confounder and measured confounders
Information Available: Internal Methods 1)Propensity Score Calibration Sturmer et al, Am J Epi 2005 2)Bayesian Modeling McCandless Stat Med 2007 3)Multiple Imputation Faries (submitted) Propensity Score Calibration Concept: Utilize additional data - variables not in full sample but available for a subset of patients - to modify
the propensity score adjustment Two propensity scores (PS) are calculated for the validation data, - Error Prone PS: utilizes only covariates available for the full sample of patients - Gold Standard PS: calculated using both the covariates in the main study along with the additional confounding covariates. Regression calibration (measurement error modeling) is then applied to adjust the regression coefficients and thus compensate for the unmeasured confounding. Propensity Score Calibration (ctd) Error Prone Propensity Score Model (PSEP)
PSEP Pr( X 1 | z1, z 2,..., zn) Gold Standard Propensity Score Model (PSGS) PSGS Pr( X 1 | z1, z 2,..., zn, ) Calibration Model: E[ PSGS ] 0 1 X 2 PSEP File name/location Company Confidential Copyright 2000 Eli Lilly and Company
Propensity Score Calibration Validity relies on surrogacy of the error prone propensity for the gold standard propensity. error prone PS must be independent of the outcome given gold standard PS and treatment. For our example surrogacy assumption not clearly satisfied Correlations of A1C & Outcome was negative Correlations of Other Covariates & Outcome was positive Bayesian Twin Regression Models
Concept: Bayesian models naturally incorporate additional sources of information such as internal subset data or external information from other studies - through prior distributions Outcome 0 1 * Treatment 2 * UnmConf 3 * MeasConf Logit P ( UnmConf 1) Treatment MeasConf Internal data serves in essence as informative prior information for parameters relating to unmeasured confounder Implementation: WinBUGS (SAS 9.3 code upcoming) Bayesian Twin Regression Models Outcome 0 1 * Treatment * UnmConf * MeasConf
logit P( UnmConf 1) TRT MeasConf Priors: Uninformative: Informative: , , , 0 , 1 , 2 Keys to Bayesian Approach Incorporation of best available information through
Informative Priors Best available data whether internal (via subset data) or External data (e.g. literature) Informative Priors not just a way to add uncertainty (McCandless 2007) Yields a posterior distribution (point and interval estimates) for the treatment effect adjusted for the unmeasured confounder U. Fixed Modeling (Schneeweiss 2006) failed to incorporate variability Flexible data driven model No restrictions on relationships on associations between variables as in measurement error approaches (Sturmer 2007).
Missing Data Multiple Imputation (for internal data) Concept: This is a missing data problem use a well accepted method -- Multiple Imputation! Imputation Model: Treatment, Measured Covariates, and Outcome (in order to account for he association between confounder/outcome and confounder / treatment) Used > 5 replications due to amount of missing data Implementation: PROC MI in SAS Example: Summary of Sensitivity Analyses with Internal A1C Data
Conclusions Other References Goodman M, Barraj LM, Mink PJ, Britton NL, Yager JW, Flanders D, Kelsh MA (2007). Estimating uncertainty in observational studies of associations between continuous variables: example of methylmercury and neuropsychological testing in children Epi Perspec Innovat 4:9. Gustafson P, McCandless LC (2010). Probabilistic Approaches to Better Quantifying the Results of Epidemiologic Studies. Int. J Environ Res Pub Hlth 7:1520-1539. McCandless, L.C. Gustafson, P. Levy, A.R. Bayesian sensitivity analysis for unmeasured confounding in observational studies. Statist. Med. 2007, 26, 2331-2347. McCandless, L.C. Gustafson, P. Levy, A.R. A sensitivity analysis using information about measured confounders yielded improved assessments of uncertainty from unmeasured confounding. J. Clin. Epidemiol. 2008, 61, 247255.
Pawaskar M, Zagar A, Sugihara T, Shi L (2011). Healthcare resource utilization and costs assessment of type 2 diabetes patients initiating exenatide BID or glargine: a retrospective database analysis. J Med Econ. 2011;14(1):16-27. Epub 2010 Dec 15. Schneeweiss S, Setoguchi S, Brookhard MA, Kaci L, Wang PS (2009). Assessing Residual Confouding of the Association between Antipsychotic Medications and Risk of Death Using Survey Data. CNS Drugs, 23(2):171-180. Schneeweiss S, Wang PS (2005). Claims Data Studies of Sedative-Hypnotics and Hip Fractures in Older People: Exploring Residual Confounding Using Survey Information. JAGS 53:948-955. Schneeweiss S, Wang PS (2005). Claims Data Studies of Sedative-Hypnotics and Hip Fractures in Older People: Exploring Residual Confounding Using Survey Information. JAGS 53:948-955. Sturmer T, Schneeweiss S, Rothman KJ, Avorn J, Glynn RJ (2007). Performance of Propensity Score Calibation A Simulation Study. Am J Epi 165(10):1110-1118. Sturmer T, Schneeweiss S, Avorn J, Glynn RJ (2005). Adjusting Effect Estimates for Unmeasured Confounding with Validation Data using Propensity Score Calibration. Am J Epi 162(3):279-289. Weiner MG, Xie D, Tannen RL (2008). Replication of the Scandinavian Simvastatin Survival Study using a primary care medical record
database prompted exploration of a new method to address unmeasured confounding. Pharmacoepi Drug Safety DOI: 10.1002/pds Xue F, Strombom I, Turnbull B, Zhu S, Seeger JD (2011). Duloxetine for Depression and the Incidence of Hepatic Events in Adults. J Clin Psych 31:517-522/
…thinking about what we want the students to be able to do Bloom's Taxonomy (1956) Anderson & Krathwohl, eds. (2000) Remembering [recall information] Understanding [explain ideas and concepts] Applying [use new knowledge in another situation] Analyzing [distinguish importance, outline,determine point...
Repayers give because of what they have received in life, which could involve gratitude for medical or educational benefits that they have received. They insist on cost-effectiveness and want nonprofit organizations to focus on client needs. Dynasts see philanthropy as...
You are invited to the 6th Annual CNH District's Missions Conference, "Healthy for Mission" Saturday, March 16, 2019 St. Paul's Lutheran Church. 1635 Chester Drive. Tracy, CA 95376. To register, go to www.cnh-lcms.org.
Thomas & Collier Study Next Steps . . . 2010 Data Analyses Spring 2014. 2011 & 2012 Analyses plus 5-year longitudinal study. Establish protocol and procedures for NCDPI to apply study analyses in future years
Many fancy things like quantum teleportation. Not important for us. Shor's algorithm (1994) Quantum computers can do FFT very efficiently. Can be used to find period of a function. ... Only signatures (new proposal for encryption exists but too recent)
Which part is the ratio decidendi and which the obiter dicta? Which rule is to be followed given the various opinions or judgments in a case? Precendence Donoghue v Stevenson  A.C. 562 Find out the legal problem that you...
"Surely God is good to Israel, to those who are pure in heart!But as for me, my feet came close to stumbling; My steps had almost slipped.For I was envious of the arrogant, as I saw the prosperity of the...
Ready to download the document? Go ahead and hit continue!