Why a well-written AI answer can still be wrong

Two words first

• Hallucination: when an AI states something false as if it were true, for example a made-up date, name or quote.
• Fluent: smooth and natural, like a person who writes well.

What the researchers did

Researchers at Radboud University in the Netherlands showed people one of four chatbot conversations about The Lord of the Rings. Everyone who took part already knew the story.

• Two chatbots wrote in a fluent, human-like way. Two gave short, businesslike replies.
• In each style, one chatbot was correct and one slipped in small invented facts.

People then rated how smart and pleasant the chatbot seemed, and how much they trusted it.

What they found

• The fluent chatbot with false facts was rated about as smart and pleasant as the short chatbot that was correct.
• Up to 80% of the people did not notice the invented facts at all.
• The fluent chatbot that was correct got the best ratings.
• People who saw a chatbot as smarter also trusted it more and wanted to use it more.

A language model is the AI system behind a chatbot: it learns from huge amounts of text to predict which words come next. The researchers explain that these models are trained mainly to write text that sounds right and reads well. Being correct is a separate goal.

They also argue that one general warning, such as "AI can make mistakes", is not enough, and that chatbots should show when they are unsure.

Limits of the study

The study used one topic, a well-known book series, and short conversations. Results may be different for other topics, like health or school work.

What this means for you when you chat with ChatPrime

Answers from any AI chat tool, ChatPrime included, can read well and still contain mistakes.

• Judge the facts, not the style. A confident tone does not mean the answer is right.
• Check what matters, like health, money, law or school work, with a trusted source.
• Ask which parts might be wrong. This can help you spot weak points, but the reply can be wrong too.
• Open the sources you are given and make sure they say what the answer claims. A source named by an AI can also be wrong or made up.

Source

Marianna A. de Sá Siqueira, Tibor Bosse and Barbara C. N. Müller (Radboud University), "Users' perception of chatbot language fluency and hallucination", Computers in Human Behavior: Artificial Humans, 2026: doi.org/10.1016/j.chbah.2026.100412

Short summary from the university, October 29, 2026: Radboud University news