Explainer · AI and citations

Does AI Create Fake Citations? Why It Happens and How to Avoid It

Published 22 October 2026 · 7-minute read · By the CiteLoom team

If you have asked a chatbot for sources and later found that some of them don't exist, you have met one of the best-known failures of generative AI. Researchers call these hallucinated citations. They matter more in academic writing than almost anywhere else, because a citation is a claim of fact: this source exists and it says this.

This explainer covers why AI creates fake citations, how widespread the problem is, the subtler errors that are even more common, and how to protect your work.

Why language models invent references

A large language model is trained to continue text in a way that looks right. It learns that academic sentences are often followed by something like "(Smith et al., 2019)" and that reference lists contain an author, a year, a title, a journal and a DOI. When you ask for a reference, it produces text with exactly that shape.

What it does not do, unless the tool connects it to a database, is look anything up. The model has no list of papers to choose from. It assembles a reference from fragments that commonly appear together: a researcher who publishes in the area, a title built from the key terms of your question, a journal in the right field. The result can be indistinguishable from a real reference until you search for it.

Three features make the problem worse:

• Confidence. Models present invented references in the same tone as real ones.
• Partial truth. A fake reference often contains real elements, such as an author who exists, which makes it pass a quick glance.
• Specificity. Requests for precise details such as page numbers or exact statistics push the model to fill gaps with invention.

How common are fake citations?

Precise rates vary by tool, topic and how the question is asked, and many published figures come from vendors with something to sell. Two independent signals show the scale clearly:

• At ICLR 2026, a major machine-learning conference, a scan of 300 submissions found that 50 contained at least one hallucinated citation, and those papers had already passed peer review.
• An analysis of 100 fabricated citations at NeurIPS 2025 concluded that peer reviewers focus on methods and results rather than checking whether each cited source exists.

If fabricated references get past expert reviewers at top conferences, they can certainly get into a student thesis. The safe assumption is that any reference you have not checked might be wrong.

Fake citations are only the first problem

Invented papers get attention because they are dramatic. In practice, three other errors are at least as common, especially in tools that do use real databases.

Misattribution

The paper is real, but it doesn't say what your sentence claims. A tool might attach a real study on urban parks and stress to a sentence about parks and depression. Examiners who know the literature spot this quickly.

Overstatement

The paper supports a weaker version of the claim. A cross-sectional survey that found an association is cited as if it proved causation, or a finding about one country is generalised to all.

Retracted sources

The paper was real and relevant but has since been retracted. Retracted papers stay in databases and keep being cited unless someone checks.

How to tell if a citation is fake

1. Search the exact title in Google Scholar, Semantic Scholar or OpenAlex. A real paper usually appears immediately.
2. Resolve the DOI at doi.org. A fake DOI returns an error or a different paper.
3. Check the author's publication list, on their university page or Google Scholar profile.
4. Check the journal: does that volume and issue exist, and does it contain the article?
5. Read the passage that supposedly supports your sentence.

Warning signs include a title that matches your question suspiciously well, a journal outside the author's usual field, a DOI that doesn't resolve, or page numbers that don't fit the journal's issue.

Are AI tools getting better at citations?

Yes, but unevenly. Tools that connect language models to scholarly databases have reduced invented references, and research groups are building dedicated detectors for hallucinated citations. But three things haven't changed: models can still summarise a real source inaccurately, abstracts don't contain every detail a sentence might rely on, and a tool's marketing claims are not the same as independent evaluation. Treat improvements as reducing your checking work, not removing it.

How to use AI without fake citations

The fix is not to avoid AI but to use it in a way that keeps references grounded.

In practice:

• Use tools that retrieve before they write. References should come from real records such as Semantic Scholar, OpenAlex, Crossref or your own library, never from the model's memory.
• Give the tool your own sources. Upload the PDFs and reference library your supervisor expects you to use.
• Check claims, not just existence. Prefer tools that compare each cited sentence with the source and show the exact passage.
• Screen for retractions.
• Verify the references your argument depends on by reading them yourself.

CiteLoom was built around this approach: it cites only real papers from scholarly databases and your own files, excludes retracted papers, checks every cited sentence against the source, and shows a supporting quote only if it is found word for word.

A worked example: checking a suspicious reference

Imagine a chatbot gives you this reference for a claim about student wellbeing: Okafor, L. & Brandt, M. (2021). "Green campus design and postgraduate stress." Journal of Environmental Psychology, 74, 101582.

1. Search the exact title in Google Scholar and Semantic Scholar. No result. First warning sign.
2. Resolve the DOI or look up the volume. Volume 74 of that journal exists, but article 101582 is a different paper by different authors. Second warning sign.
3. Search the authors. Neither appears to have published on campus design. Third warning sign.

Conclusion: the reference is fabricated. It combines a real journal, a realistic volume, plausible names and a title built from your search terms. The fix is to search for real studies on your topic and cite what they actually found. (The reference above is an invented illustration, not a real paper.)

Why web search alone doesn't solve the problem

Connecting a chatbot to web search reduces invented references but doesn't eliminate errors. Web results include low-quality pages, and the model may still summarise a source inaccurately or attach it to the wrong claim. For academic work, retrieval from scholarly databases plus claim-level checking against the source text is what makes references dependable.

What happens if you submit a fake citation?

Universities generally treat fabricated references as a serious academic-integrity issue, whatever the intent, because they misrepresent the evidence behind your work. Many institutions also require you to disclose AI use, and undisclosed use in assessed work is commonly treated as misconduct. The safest position is simple: every reference in your work should be one you have verified.

Frequently asked questions

Sources

1. Vectara, ICLR 2026 hallucinated citations case study.
2. Compound Deception in Elite Peer Review: a failure-mode taxonomy of 100 fabricated citations at NeurIPS 2025, arXiv.
3. HalluCiteChecker: a toolkit for hallucinated citation detection and verification, arXiv.
4. Thesify, Generative AI policies at the world's top universities: 2026 update.