Approaches to observational study designs and analytical options to evaluate the safety of multi-dose vaccines: a systematic review

Clinical Snapshot

Key Findings

• P Value: Not applicable - no statistical testing performed

• Effect Size: 28% of cohort studies failed to report time zero definition

• Primary Outcome: Distribution of study designs: cohort (46%), SCCS/SCRI (40%), case-control (12%)

• Nnt Or Sensitivity: 68% consistency rate when multiple designs used in same study

• Confidence Interval: Not applicable - descriptive systematic review

Clinical Application

Highly feasible - provides methodological guidance that can be immediately implemented in study design and reporting Directly relevant to TGA post-market surveillance activities, NCIRS vaccine safety monitoring, and Australian immunisation research. Findings support improved methodological standards for vaccine safety studies submitted to Australian regulatory bodies. Researchers conducting post-licensure vaccine safety studies, regulatory agencies, and systematic reviewers

Abstract

INTRODUCTION: Observational studies require careful considerations when evaluating the safety of multidose vaccines. We reviewed design and analytical approaches in observational studies evaluating the safety of multidose vaccines in the post-licensure phase. METHODS: EMBASE, MEDLINE, Web of Science, and Scopus (2018-2022) were searched for hypothesis-testing studies evaluating the safety of multidose vaccines. Key features from frequently used designs were extracted. RESULTS: Among 123 eligible studies, cohort (46%) and self-controlled case series (SCCS)/self-controlled risk interval (SCRI) (40%) followed by case-control (12%) were the most common designs, and 15% of studies used multiple designs. Among cohort studies evaluating multiple doses, vaccination date (36%) and cohort entry with time-updated exposure status (32%) were frequent approaches used to define time zero. Twenty-eight percent of cohort studies did not report time zero; all but one evaluated COVID-19 vaccine effect on post-delivery and fertility-related outcomes. For SCCS/SCRI, 64% of studies accounted for event-dependent exposures, mainly by including pre-exposure periods (53%) and modified SCCS model (48%), while 20% employed multiple correction strategies. Among studies using multiple designs, 68% reached consistent conclusions. CONCLUSIONS: SCCS/SCRI and cohort designs dominate multidose vaccine safety studies. Clear reporting on time zero in pregnancy and fertility-related cohort studies, and on addressing event-dependent exposures in SCCS/SCRI studies is needed, along with guidance on interpreting results from multiple designs. Observational studies are important for assessing safety of vaccines especially for rare health outcomes that cannot be easily detected in randomized trials. Commonly used approaches include comparing these health outcomes between vaccinated and unvaccinated individuals or between different time points in relation to vaccination timing within individuals. However, both approaches have important design challenges. For example, comparison between vaccinated and unvaccinated individuals requires careful consideration of alignment of the time of starting to observe outcomes with the time when individuals become eligible and vaccination time for both comparison groups (time zero). Within-individual comparison requires that earlier occurrence of health outcomes does not influence whether someone receives later vaccine dose(s). These challenges become more complex to address when vaccines are given in multiple doses.In this review, we assessed how important design and analytical issues are handled and reported in published studies evaluating the safety of multidose vaccines and assessed agreement between different study designs when more than one is used.We found that 28% of studies comparing vaccinated and unvaccinated groups did not clearly report time alignment (time zero) mostly with studies on pregnancy and fertility outcomes. Similarly, about one-third of studies using within-individual comparison did not report checking and addressing the possibility that earlier outcomes could influence future vaccinations. When multiple designs were used, two-thirds of studies reached similar conclusions, although comparisons were mainly qualitative. Future work is needed to guide clear reporting of these issues to improve transparency in design and analysis and when comparing results across different designs.

References

1. 1.Mtei, M., Trang, H., Pham, T., Navarro-Torné, A., Douglas, I. J., & Schultze, A. (2026). Approaches to observational study designs and analytical options to evaluate the safety of multi-dose vaccines: a systematic review. Expert Review of Vaccines. https://doi.org/10.1080/14760584.2026.2667740