A review and evaluation of doubly robust approaches for estimating average treatment effects

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: Doubly robust methods, particularly regression weighted by inverse propensity score, showed superior performance

• Primary Outcome: Bias reduction in average treatment effect estimates

• Nnt Or Sensitivity: Methods provide protection when either outcome model or propensity score model is correctly specified

• Confidence Interval: Not applicable - simulation study

Clinical Application

Moderate - requires statistical expertise but methods are implementable with standard software Highly relevant for Australian health services research, pharmaceutical outcomes research, and real-world evidence studies required by TGA. Particularly valuable for PBS effectiveness evaluations and RACGP guideline development using observational data Researchers conducting observational studies to estimate treatment effects in healthcare settings

Abstract

In nonexperimental studies, obtaining an unbiased estimate of the average treatment effect (ATE) typically requires two key assumptions: that all relevant covariates are measured (i.e., no unmeasured confounding) and that the statistical model used for covariate adjustment is correctly specified. Two common approaches for adjustment are specifying an outcome model and propensity score weighting. To mitigate bias from model misspecification, doubly robust methods combine both approaches, ensuring unbiased ATE estimates if either the outcome model or the propensity score model is correctly specified. In this study, we review four doubly robust methods that have received considerable attention in the methodological literature but remain underutilized in psychological research: augmented inverse probability weighting, regression weighted by the inverse propensity score, regression incorporating the inverse propensity score as a covariate, and calibrated propensity score weighting. Using two simulation studies, we compare these methods with regression estimation and inverse probability weighting estimators. Our results suggest that doubly robust methods-particularly regression weighted by the inverse propensity score-offer greater protection against bias from model misspecification across various data-generating scenarios. We also discuss practical considerations for implementing doubly robust methods, including weight normalization, propensity score truncation, and potential efficiency losses due to overfitting. The different methods for estimating the ATE are illustrated in a data example.

References

1. 1.Zhang, J., Lüdtke, O., & Robitzsch, A. (2026). A review and evaluation of doubly robust approaches for estimating average treatment effects. Behavior Research Methods. https://doi.org/10.1080/01621459.2015.1023805