New study: Scary, chatbots make up up to 40 percent of answers and present them as facts

They ask ChatGPT about the election results, ask Claude to summarize the news, or ask Perplexity to give them basic information about the conflict in the Middle East: hundreds of millions of people rely on AI chatbots as sources of information every day. ChatGPT alone is used by 800 million people worldwide every week. For many, these digital assistants are already replacing traditional Google searches.

But that trust is risky, as a new study by the European Broadcasting Union (EBU) shows. The association of 68 public broadcasters from 56 countries systematically tested the reliability of the most popular artificial intelligence systems.

The result is frightening: ChatGPT, Claude, Gemini and other chatbots make up up to 40 percent of their answers and present them as facts.

Hallucinations: Artificial Intelligence Lies Convincingly

The popular chatbot ChatGPT firmly claims that Pope Francis is still alive. Microsoft's Copilot, which is integrated into the office programs Word and Excel, does not know that Sweden is in NATO. And Google's Gemini believes that Donald Trump's re-election is "possible," even though it happened a long time ago.

"The systems sound convincing, even though they repeatedly claim completely false things," warns economist Peter Poš of the Technical University of Dortmund. "This makes them particularly dangerous for inexperienced users, because errors are often not immediately obvious."

There is a phenomenon that experts call "hallucination": artificial intelligence fantasizes about information that seems coherent but has no factual basis. This is especially common with regional events, current affairs, or when multiple pieces of information need to be connected.

Threat to democracy

But what does this mean for a society where more and more people get their information from chatbots? The consequences are already noticeable: false information spreads quickly on social networks, as users share "facts" generated by artificial intelligence without verification. Schoolchildren and students include fabricated information in their work. Citizens can make decisions about who to vote for based on false claims.

What's particularly problematic is that many users are not even aware that chatbots can hallucinate. They assume that the technology operates objectively and factually, which is a dangerous misconception. AI systems warn about potential errors in their terms of use, but – who reads that anymore?

Damage to the reputation of the media

Another problem concerns the credibility of established media. Chatbots regularly claim that their fabricated information comes from sources such as German public broadcasters ARD or ZDF, even though they have never reported it – or have reported it quite differently. Users lose trust in reputable sources when artificial intelligence misuses their names to spread false information.

The EBU study tested chatbots with hundreds of factual questions: about historical events, scientific findings, and current news. Depending on the topic, the error rate ranged from 15 to as much as 40 percent. None of the AIs tested performed flawlessly.

Why does artificial intelligence make mistakes?

The problem is in the system: chatbots don't actually understand what they're saying. They calculate which words are likely to match based on vast amounts of text and can't verify whether the statement that results from such calculations is true. They have no knowledge of facts, only statistical patterns.

Technology companies are aware of these shortcomings and are working on solutions. They are integrating databases, improving citations, and retraining systems. Billions are being invested in development. Yet hallucinations remain a fundamental, unsolved problem with the technology.

What can users do?

The EBU recommends clear rules for dealing with chatbots: never trust blindly, always verify important information, and rely on established media for news and facts, not artificial intelligence. Caution is particularly advised when it comes to political topics, health issues, or financial decisions.

Schools and universities must teach media literacy: how do I recognize disinformation generated by artificial intelligence, which sources are reliable? The German government is planning awareness campaigns – but they are late. Millions of people have been using the technology for a long time and every day.

Until it becomes more reliable, here's the thing: chatbots can be useful for creative tasks or as writing aids, but they're not suitable as fact-checkers or news sources. And no one should rely on them 100 percent.

Anyone who wants to be informed cannot avoid reputable media outlets that are staffed by human editors, who check sources and evaluate claims and evidence. The digital revolution may change many things, but the need for careful research and fact-checking remains.