Large Language Models as Physician Recommenders: Limitations, Alternatives, and the Path Toward Hybrid AI Systems

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: Not applicable — no quantitative effect estimates reported; qualitative comparative assessment only

• Primary Outcome: Comparative evaluation of LLMs versus four alternative physician-selection strategies across dimensions of accuracy, transparency, equity, and clinical validity; proposal of a hybrid system integrating auditable performance measures with patient-preference filters and verifiable explanations

• Nnt Or Sensitivity: Not applicable — no diagnostic, therapeutic, or prognostic data generated; framework paper only

• Confidence Interval: Not reported — narrative review with no meta-analytic synthesis

Clinical Application

The conceptual hybrid framework is theoretically feasible but practically unvalidated. Implementation would require: (1) access to risk-adjusted, physician-level outcome data of sufficient quality; (2) transparent and auditable AI infrastructure; (3) patient-facing explainability mechanisms; and (4) robust governance frameworks. These prerequisites represent significant resource and data infrastructure challenges for most health systems. Australian applicability requires substantial contextual adaptation. The TGA regulates software as a medical device (SaMD) under the Software as a Medical Device framework, which would govern any AI-based physician recommender system deployed clinically. The RACGP's Standards for General Practices and the Australian Commission on Safety and Quality in Health Care's frameworks for clinical governance would be relevant to implementation. Consumer-facing physician rating platforms (e.g., Healthengine, RateMDs) operate in Australia but are not subject to the same regulatory scrutiny as clinical decision support tools. The My Health Record system and AIHW national datasets could theoretically support outcome-informed components of a hybrid system, though physician-level performance data remain limited and politically sensitive. PBS considerations are not directly relevant to physician selection per se, but equitable access to specialist care — a known challenge in rural and remote Australia — is directly implicated by the equity design principle the authors propose. RACGP guidance on digital health tools and the Australian Digital Health Agency's national digital health strategy provide relevant governance context. Patients using digital tools to select clinicians; health system administrators designing patient navigation platforms; clinical informaticists and health technology governance bodies evaluating AI-assisted care navigation tools

Abstract

Digital tools are increasingly used by patients to access health information and navigate care, including choosing clinicians, but the evidence supporting "best doctor" recommendations varies widely across available methods. This manuscript compares the emerging use of consumer-facing large language models (LLMs) for this purpose with four alternative physician-selection strategies: patient networks, expert referrals, consumer rating/search platforms, and outcome-informed artificial intelligence / machine learning (AI/ML) recommender systems. Consumer rating platforms mainly reflect patient experiences rather than technical skills and often show weak or inconsistent links with objective performance measures. General-purpose LLMs may produce convincing rationales that highlight visibility and reputation, but they remain limited by hallucinations, prompt sensitivity, and uncertainty about whether their explanations truly reflect the underlying reasoning. Outcome-informed AI/ML recommender systems could, in theory, incorporate risk-adjusted clinical performance, but they face significant challenges, including unreliable physician-level measurement, incomplete data, residual confounding, and limited transparency, which can undermine trust and informed consent. We suggest that a hybrid system should be seen as a guiding concept and a testable approach rather than a final solution. Such a system could integrate proven, auditable performance measures with transparent patient-preference filters and verifiable, patient-facing explanations. We present key design principles, including clinical relevance, explainability, autonomy, equity, and governance, in line with current US Food and Drug Administration (FDA) guidance related to clinical decision support and adaptive AI life cycle management.

References

1. 1.Bagheri Asl, M. M., & Spiotta, A. M. (2026). Large language models as physician recommenders: Limitations, alternatives, and the path toward hybrid AI systems. Joint Commission Journal on Quality and Patient Safety. https://doi.org/10.1016/j.jcjq.2026.03.011