Guide · Core, Pathways and documents

Hallucinated citations in medicine: the problem, and how to check a reference

Written for
Hospital physicians, residents and anyone who relies on AI-generated references
Reading time
7 minutes
Published
Updated

Written by the Sana Health team for clinicians. It summarises published guidance, is not a guideline and has not been peer reviewed. Check current local protocols before acting.

Key points

  • A hallucinated citation looks like a real reference but describes a paper that does not exist, or attaches wrong details to one that does.
  • Published studies of general-purpose chatbots found high rates of fabricated and inaccurate references in medical content.
  • The cause is structural: a language model that writes a reference list from patterns can produce a plausible one without retrieving it.
  • Errors are not new: human-written medical papers also misquote sources and keep citing retracted work.
  • Any reference that will support a clinical decision should be opened and checked: the record, the DOI, the retraction status and what the paper actually says.

What a hallucinated citation is

Reference hallucination describes AI-generated citations that appear legitimate but are fabricated or carry significant errors in their details [7]. A fabricated reference usually looks convincing: plausible authors, a real journal, a credible year and title, sometimes a PMID or DOI that leads nowhere or to an unrelated paper. A subtler form attaches a real paper to a claim it does not support.

These errors follow from how large language models work. They generate text by predicting likely sequences of words rather than by looking facts up, and the causes identified include gaps in training data, facts that appear only once in that data, errors with named entities such as authors and journals, and poorly designed prompts [8]. A reference list is exactly the kind of structured, detail-heavy text that such prediction can imitate well while getting the details wrong [9].

What the studies found

Soon after general-purpose chatbots became widely available, researchers began testing the references they produced. The studies below used the versions available at the time, and newer models may behave differently, but the mechanism is the same whenever a model writes a reference list itself.

  • ChatGPT-3.5 was asked to write 30 short medical papers with at least three references each; checking the references against Medline, Google Scholar and other sources found high rates of fabricated and inaccurate references [1].
  • When ChatGPT was asked 20 medical questions and prompted for references, the researchers had to change their planned evaluation because most of the references it provided were fabricated [2].
  • ChatGPT-3.5 and ChatGPT-4 wrote short literature reviews on 42 topics; the 636 citations in the 84 papers were searched to measure the prevalence of fabricated citations and the errors in citations to real papers [3].
  • ChatGPT and Bard were tested on reproducing the references of human-conducted systematic reviews, with hallucination rates and reference accuracy compared between them [4].
  • A cross-disciplinary study checked both the citations and the DOIs that ChatGPT produced when asked to write the introduction of a manuscript [5].
  • An early report described artificial hallucinations when ChatGPT was asked to write about the pathogenesis of two rare metabolic conditions for case reports [6].

Why it matters at the bedside

A reference is often the reason a doctor trusts a dose, a threshold or a change of practice. A fabricated reference removes that safeguard while appearing to provide it, and an invented citation copied into a guideline, a teaching slide or a discharge letter can travel far from its source.

The problem is not unique to AI. Reviews of the medical literature have estimated that roughly 20 to 25% of assertions cited from original research are quoted inaccurately [10]. Retracted articles also continue to be cited as if they were valid, often because readers do not know they have been retracted [11, 12]. Checking a reference is a habit worth having whatever its source.

How to check a reference in two minutes

  1. Search PubMed for the exact title. A real paper indexed in PubMed should appear at once; if nothing close appears, treat the reference as unverified.
  2. Open the PubMed record and compare it with the citation: title, first author, journal, year and volume should all match.
  3. If a PMID is given, open it directly and confirm it is the same paper. A real PMID attached to the wrong paper is a common form of fabrication.
  4. Resolve the DOI through doi.org and confirm it leads to the same article on the journal's site.
  5. Look for a retraction or expression of concern on the PubMed record and on the journal page.
  6. Read the abstract, and the full text where the claim depends on detail, to confirm the paper says what it is cited for: the same population, intervention, dose and outcome.
  7. Check what kind of evidence it is and when it was published: a case report is not a trial, and a guideline may have been superseded.
Warning signWhat it may mean
Title not found in PubMed or on the journal's siteLikely fabricated
PMID or DOI opens a different paperFabricated or mismatched reference
Authors, year or journal differ from the recordReal paper with corrupted details, or a fabrication built on a real one
Paper exists but does not mention the claimMisattribution: the claim is unsupported
Retraction notice on the recordDo not rely on the findings

Using AI tools with references safely

  • Prefer tools that retrieve their references from bibliographic databases at the time of the question rather than writing them.
  • Even with retrieved references, check that the cited paper supports the specific sentence it is attached to.
  • Never copy a reference into a note, guideline or teaching material without opening it first.
  • Be most careful with precise claims, such as a dose, a percentage or a threshold, because these are where an unsupported citation does most harm.

Verifying that a paper exists is only the first step. The clinical question is whether the evidence applies to the patient being treated, and that judgement stays with the clinician.

How Sana Health helps

The Evidence Review engine answers a focused question from the trials and guidelines, and like every Sana Health engine it ends with a reference list read from PubMed, ClinicalTrials.gov and, for named medicines, DailyMed when the question is asked. The model does not write that list, so its links cannot be invented. Retracted papers are excluded, the answer cites the list by number, and any PubMed link the model writes in the text is checked against PubMed. Each entry links to PubMed, to the journal's page and, where one exists, to the free full text, so every claim can be checked.

Questions

Do AI chatbots make up medical references?

General-purpose chatbots that write their own reference lists have been shown to produce fabricated and inaccurate references in medical content. Tools that retrieve references from databases avoid fabrication, but the link between a claim and its source still needs checking.

How can I tell whether a PubMed citation is real?

Search PubMed for the exact title and open the record. Check that the title, authors, journal and year match, that any PMID or DOI opens the same paper, and that there is no retraction notice.

Is a real reference always a reliable one?

No. A real paper can be misquoted, may not support the specific claim, may be a weak study design, or may have been retracted. Reading the abstract or full text is the only way to be sure.

References

  1. High Rates of Fabricated and Inaccurate References in ChatGPT-Generated Medical Content (opens in a new tab)Cureus, 2023
  2. Learning to Fake It: Limited Responses and Fabricated References Provided by ChatGPT for Medical Questions (opens in a new tab)Mayo Clin Proc Digit Health, 2023
  3. Fabrication and errors in the bibliographic citations generated by ChatGPT (opens in a new tab)Sci Rep, 2023
  4. Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis (opens in a new tab)J Med Internet Res, 2024
  5. Evaluation of Large Language Model Performance and Reliability for Citations and References in Scholarly Writing: Cross-Disciplinary Study (opens in a new tab)J Med Internet Res, 2024
  6. Artificial Hallucinations in ChatGPT: Implications in Scientific Writing (opens in a new tab)Cureus, 2023
  7. Citation integrity in the age of AI: evaluating the risks of reference hallucination in maxillofacial literature (opens in a new tab)J Craniomaxillofac Surg, 2025
  8. AI hallucinations in academic writing: implications for research integrity (opens in a new tab)Naunyn Schmiedebergs Arch Pharmacol, 2026
  9. Large language models in medicine (opens in a new tab)Nat Med, 2023
  10. Accuracy of cited "facts" in medical research articles: A review of study methodology and recalculation of quotation error rate (opens in a new tab)PLoS One, 2017
  11. Reducing the Inadvertent Spread of Retracted Science: recommendations from the RISRS report (opens in a new tab)Res Integr Peer Rev, 2022
  12. Citations of microRNA Biomarker Articles That Were Retracted: A Systematic Review (opens in a new tab)JAMA Netw Open, 2024