Report cards and outcome measurements to improve the safety of surgical care (North America): an updated systematic review from Making Healthcare Safer IV

Clinical Snapshot

Key Findings

• P Value: Statistically significant mortality benefit in 1 of 4 controlled studies

• Effect Size: Variable effects depending on study design and implementation approach

• Primary Outcome: Surgical safety outcomes and quality improvement measures

• Nnt Or Sensitivity: Not calculated; effectiveness dependent on active quality improvement implementation

• Confidence Interval: Not reported for overall findings

Clinical Application

Moderate - requires institutional commitment to quality improvement initiatives beyond just data collection Relevant to Australian surgical quality initiatives, ACHS programs, and state-based surgical outcome reporting systems. Aligns with ACSQHC National Safety and Quality Health Service Standards Surgical patients in hospital and outpatient settings where quality improvement programs can be implemented

Abstract

OBJECTIVES: In the USA, an estimated 40-50 million operations are performed annually, with high rates of adverse events. Since the 1980s, report cards have been used for outcome measures and to improve safety of surgical care. As part of Making Healthcare Safer IV-an initiative aimed at publishing evidence-based reviews as they are completed to help healthcare leaders, researchers and policymakers act more quickly on evidence-supported practices-we performed an updated review on the certainty of evidence on patient safety practices related to the use of surgical report cards and outcome measurements. DESIGN: Systematic review using the Grade of Recommendations Assessment, Development and Evaluation (GRADE) approach. DATA SOURCES: PubMed, Web of Science, Scopus and the Cochrane Library were searched from November 2011 to May 2023. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: We included primary research studies (randomised control trials or observational studies with a comparison group, including pre-post studies) or observational studies that investigated a surgical report card in adult or paediatric surgical patients at the hospital or surgeon level in inpatient or outpatient settings. Excluded studies included: narrative reviews, scoping reviews, editorials, commentaries, abstracts, studies that measured only patient knowledge or levels of engagement or studies using local surgical dashboard data. DATA EXTRACTION AND SYNTHESIS: Screening and eligibility were done in duplicate, while data extraction was done by one reviewer and checked by a second reviewer. Specific items in the Risk Of Bias In Non-randomised Studies - of Interventions tool and a modification of the National Institutes of Health Tool were used to assess for bias in studies. Two reviewers assessed each study for risk of bias. A modified version of the GRADE framework was used to assess the certainty of evidence. RESULTS: We identified 19 studies that met the inclusion criteria: 13 primary research studies and 6 descriptive studies of surgical collaboratives. Of the primary studies, nine used a pre-post or longitudinal study design and four used a regression discontinuity or concurrent controlled design. Seven of the studies were about the American College of Surgeons National Surgical Quality Improvement Project. Five studies were from single institutions and the remainder included nine to greater than 700 hospitals. Pre-post studies of report cards that prompted quality improvement (QI) programmes all reported improvements in outcomes, longitudinal studies reported benefits in some but not all outcomes and one in four controlled before-and-after studies reported a statistically significant mortality benefit. All studies, except for one, were at moderate or high risk of bias. Six collaboratives were identified with preliminary data. CONCLUSIONS: Based on the above evidence, the theoretical rationale and parallel evidence in other settings, we judged that it was moderate certainty that report cards and outcomes measurements can improve surgical outcomes. However, given the evidence from studies where report cards were actively linked to institutional QI initiatives, we recommend that outcome data must be paired with actionable QI efforts to meaningfully improve patient outcomes.

References

1. 1.Huy, T., Blegen, M. B., Tupper, H., Premji, A., Motala, A., Lawson, E., Shekelle, P. G., Girgis, M., & Maggard-Gibbons, M. (2026). Report cards and outcome measurements to improve the safety of surgical care (North America): an updated systematic review from Making Healthcare Safer IV. BMJ Open, 16(4), e101718. https://doi.org/10.1136/bmjopen-2025-101718