ChatGPT ‘hallucinates.’ Some researchers worry it isn’t fixable. (2023)


10 min


Gift Article

SAN FRANCISCO — Recently, researchers asked two versions of OpenAI’s ChatGPT artificial intelligence chatbot where Massachusetts Institute of Technology professor Tomás Lozano-Pérez was born.

One bot said Spain and the other said Cuba. Once the system told the bots to debate the answers, the one that said Spain quickly apologized and agreed with the one with the correct answer, Cuba.

Tech is not your friend. We are. Sign up for The Tech Friend newsletter.ArrowRight

The finding, in a paper released by a team of MIT researchers last week, is the latest potential breakthrough in helping chatbots to arrive at the correct answer. The researchers proposed using different chatbots to produce multiple answers to the same question and then letting them debate each other until one answer won out. The researchers found using this “society of minds” method made them more factual.

“Language models are trained to predict the next word,” said Yilun Du, a researcher at MIT who was previously a research fellow at OpenAI, and one of the paper’s authors. “They are not trained to tell people they don’t know what they’re doing.” The result is bots that act like precocious people-pleasers, making up answers instead of admitting they simply don’t know.

(Video) ChatGPT's HUGE Problem


The researchers’ creative approach is just the latest attempt to solve for one of the most pressing concerns in the exploding field of AI. Despite the incredible leaps in capabilities that “generative” chatbots like OpenAI’s ChatGPT, Microsoft’s Bing and Google’s Bard have demonstrated in the last six months, they still have a major fatal flaw: they make stuff up all the time.

Figuring out how to prevent or fix what the field is calling “hallucinations” has become an obsession among many tech workers, researchers and AI skeptics alike. The issue is mentioned in dozens of academic papers posted to the online database Arxiv and Big Tech CEOs like Google’s Sundar Pichai have addressed it repeatedly. As the tech gets pushed out to millions of people and integrated into critical fields including medicine and law, understanding hallucinations and finding ways to mitigate them has become even more crucial.

Most researchers agree the problem is inherent to the “large language models” that power the bots because of the way they’re designed. They predict what the most apt thing to say is based on the huge amounts of data they’ve digested from the internet, but don’t have a way to understand what is factual or not.


Still, researchers and companies are throwing themselves at the problem. Some firms are using human trainers to rewrite the bots’ answers and feed them back into the machine with the goal of making them smarter. Google and Microsoft have started using their bots to give answers directly in their search engines, but still double check the bots with regular search results. And academics around the world have suggested myriad clever ways to decrease the rates of false answers, like MIT’s proposal to get multiple bots to debate each other.

The drive to improve the hallucinations problem is urgent for a reason.

Already, when Microsoft launched its Bing chatbot, it quickly started making false accusations against some of its users, like telling a German college student that he was a threat to its safety. The bot adopted an alter-ego and started calling itself “Sydney.” It was essentially riffing off the student’s questions, drawing on all the science fiction it had digested from the internet about out-of-control robots.


(Video) How to fix ChatGPT hallucination ☁️, statement on AI risk 🚨, ControlNet for videos 📹

Microsoft eventually had to limit the number of back-and-forths a bot could engage in with a human to prevent it from happening more.

In Australia, a government official threatened to sue OpenAI after ChatGPT said he had been convicted of bribery, when in reality he was a whistleblower in a bribery case. And last week a lawyer admitted to using ChatGPT to generate a legal brief after he was caught because the cases cited so confidently by the bot simply did not exist, according to the New York Times.

Even Google and Microsoft, which have pinned their futures on AI and are in a race to integrate the tech into their wide range of products, have missed hallucinations their bots made during key announcements and demos.

None of that is stopping the companies from rushing headlong into the space. Billions of dollars in investment are going into developing smarter and faster chatbots and companies are beginning to pitch them as replacements or aids for human workers. Earlier this month OpenAI CEO Sam Altman testified at Congress saying AI could “cause significant harm to the world” by spreading disinformation and emotionally manipulating humans. Some companies are already saying they want to replace workers with AI, and the tech also presents serious cybersecurity challenges.


On Tuesday, Altman joined hundreds of other AI researchers and executives, including some senior leaders from Google and Microsoft, in signing a statement saying AI poses an existential risk to humanity on par with pandemics and nuclear war.

Hallucinations have also been documented in AI-powered transcription services, adding words to recordings that weren’t spoken in real life. Microsoft and Google using the bots to answer search queries directly instead of sending traffic to blogs and news stories could erode the business model of online publishers and content creators who work to produce trustworthy information for the internet.

“No one in the field has yet solved the hallucination problems. All models do have this as an issue,” Pichai said in an April interview with CBS. Whether it’s even possible to solve it is a “matter of intense debate,” he said.


(Video) The Dumbest Use of ChatGPT EVER! | Shelly Palmer LIVE | May 31, 2023

Depending on how you look at hallucinations, they are both a feature and a bug of large language models. Hallucinations are part of what allows the bots to be creative and generate never-before-seen stories. At the same time they reveal the stark limitations of the tech, undercutting the argument that chatbots are intelligent in a way similar to humans by suggesting that they do not have an internalized understanding of the world around them.

“There is nothing in there that tells the model that whatever it’s saying should be actually correct in the world,” said Ece Kamar, a senior researcher at Microsoft. The model itself also trains on a set amount of data, so anything that happens after the training is done doesn’t factor into its knowledge of the world, Kamar said.

Hallucinations are not new. They’ve been an inherent problem of large language models since their inception several years ago, but other problems such as the AIs producing nonsensical or repetitive answers were seen as bigger issues. Once those were largely solved, though, hallucinations have now become a key focus for the AI community.


I asked ChatGPT to write a short bio of @billmurphyjr (aka, me)

Apparently I:
* went to @BrownUniversity* & @MITSloan*
* worked in investment banking* & management consulting*
* wrote a book about medical breakthroughs*
(*not true)

When challenged it insisted it was correct.

— Bill Murphy Jr. (@BillMurphyJr) February 15, 2023

Potsawee Manakul was playing around with ChatGPT when he asked it for some simple facts about tennis legend Roger Federer. It’s a straightforward request, easy for a human to look up on Google or Wikipedia in seconds, but the bot kept giving contradictory answers.

“Sometimes it says he won Wimbledon five times; sometimes it says he won Wimbledon eight times,” Manakul, an AI researcher at the University of Cambridge and ardent tennis fan, said in an interview. (The correct answer is eight.)

Manakul and a group of other Cambridge researchers released a paper in March suggesting a system they called “SelfCheckGPT” that would ask the same bot a question multiple times, then tell it to compare the different answers. If the answers were consistent, it was likely the facts were correct, but if they were different, they could be flagged as probably containing made-up information.


(Video) This AI Killed ChatGPT - You Just Don't Know It Yet

When humans are asked to write a poem, they know it’s not necessarily important to be factually correct. But when asking them for biographical details about a real person, they automatically know their answer should be rooted in reality. Because chatbots are simply predicting what word or idea comes next in a string of text, they don’t yet have that contextual understanding of the question.

“It doesn’t have the concept of whether it should be more creative or if it should be less creative,” Manakul said. Using their method, the researchers showed that they could eliminate factually incorrect answers and even rank answers based on how factual they were.

It’s likely a whole new method of AI learning that hasn’t been invented yet will be necessary, Manakul said. Only by building systems on top of the language model can the problem really be mitigated.


“Because it blends information from lots of things it will generate something that looks plausible,” he said. “But whether it’s factual or not, that’s the issue.”

That’s essentially what the leading companies are already doing. When Google generates search results using its chatbot technology, it also runs a regular search in parallel, then compares whether the bot’s answer and the traditional search results match. If they don’t, the AI answer won’t even show up. The company has tweaked its bot to be less creative, meaning it’s not very good at writing poems or having interesting conversations, but is less likely to lie.

By limiting its search-bot to corroborating existing search results, the company has been able to cut down on hallucinations and inaccuracies, said Google spokeswoman Jennifer Rodstrom. A spokesperson for OpenAI pointed to a paper the company had produced where it showed how its latest model, GPT4, produced fewer hallucinations than previous versions.

Companies are also spending time and money improving their models by testing them with real people. A technique called reinforcement learning with human feedback, where human testers manually improve a bot’s answers and then feed them back into the system to improve it, is widely credited with making ChatGPT so much better than chatbots that came before it. A popular approach is to connect chatbots up to databases of factual or more trustworthy information, such as Wikipedia, Google search or bespoke collections of academic articles or business documents.

Some leading AI researchers say hallucinations should be embraced. After all, humans have bad memories as well and have been shown to fill-in the gaps in their own recollections without realizing it.

“We’ll improve on it but we’ll never get rid of it,” Geoffrey Hinton, whose decades of research helped lay the foundation for the current crop of AI chatbots, said of the hallucinations problem. He worked at Google until recently, when he quit to speak more publicly about his concerns that the technology may get out of human control. “We’ll always be like that and they’ll always be like that.”

(Video) How I Learned to Stop Worrying and Love ChatGPT


What is the problem with hallucinations in ChatGPT? ›

AI hallucination gained prominence around 2022 alongside the rollout of certain large language models (LLMs) such as ChatGPT. Users complained that such bots often seemed to "sociopathically" and pointlessly embed plausible-sounding random falsehoods within their generated content.

What are examples of hallucinations in ChatGPT? ›

Here are two examples of what hallucinations in ChatGPT might look like: User input: "When did Leonardo da Vinci paint the Mona Lisa?" AI-generated response: "Leonardo da Vinci painted the Mona Lisa in 1815." (Incorrect: The Mona Lisa was painted between 1503 and 1506, or perhaps continuing until 1517.)

What is an example of a hallucination chatbot? ›

When it comes to AI, hallucinations refer to erroneous outputs that are miles apart from reality or do not make sense within the context of the given prompt. For example, an AI chatbot may give a grammatically or logically incorrect response or misidentify an object due to noise or other structural factors.

How do you stump an AI chatbot? ›

How to Break a Chatbot - Eight Ways
  1. 1 - Tell the Chatbot to Reset or Start Over. ...
  2. 2 - Use Filler Language. ...
  3. 3 - Ask Whatever Is on the Display Button. ...
  4. 4 - Answering Outside the Pre-Selected Responses. ...
  5. 5 - Ask for Help or Assistance. ...
  6. 6 - Answer the Question with Non-Traditional Answers. ...
  7. 7 - Say Goodbye. ...
  8. 8 - Ask Odd Questions.

What triggers hallucinations? ›

There are many causes of hallucinations, including: Being drunk or high, or coming down from such drugs like marijuana, LSD, cocaine (including crack), PCP, amphetamines, heroin, ketamine, and alcohol. Delirium or dementia (visual hallucinations are most common)

What is the percentage of accuracy with ChatGPT? ›

Using the National Cancer Institute's (NCI) common myths and misconceptions about cancer webpage as a comparator, the research team found that 97% of the answers provided by ChatGPT were correct.

What is the most common hallucinations? ›

Auditory (sound) hallucinations: These are the most common type of hallucinations. They involve hearing sounds that aren't real, like music, footsteps or doors banging. Some people hear voices when no one has spoken. The voices may be positive, negative or neutral.

What are the 5 types of hallucinations? ›

Hallucination Types
  • Visual hallucinations. Here, a person sees something that does not exist or sees something that does exist but sees it incorrectly. ...
  • Auditory hallucinations. ...
  • Olfactory hallucination. ...
  • Tactile hallucination. ...
  • Gustatory hallucination. ...
  • General somatic hallucination. ...
  • Further Reading.

What are 3 different hallucinations? ›

Types of hallucinations
  • Visual hallucinations. Visual hallucinations involve seeing things that aren't there. ...
  • Olfactory hallucinations. Olfactory hallucinations involve your sense of smell. ...
  • Gustatory hallucinations. ...
  • Auditory hallucinations. ...
  • Tactile hallucinations.

What are 2 examples of hallucinations? ›

hear sounds or voices that nobody else hears. see things that are not there like objects, shapes, people or lights. feel touch or movement in your body that is not real like bugs are crawling on your skin or your internal organs are moving around. smell things that do not exist.

How can you tell if someone is chatbot? ›

A bot's tweets may reveal its algorithmic logic: they may be formulaic or repetitive, or use responses common in chatbot programs. Missing an obvious joke and rapidly changing the subject are other telltale traits (unfortunately, they are also quite common among human Twitter users).

Does ChatGPT give the same answers? ›

No, ChatGPT does not give the exact same answer and wording to everyone who asks the same question. While it may generate similar responses for identical or similar queries, it can also produce different responses based on the specific context, phrasing, and quality of input provided by each user.

Is ChatGPT free? ›

Can I Use Chat GPT for Free? The short answer is yes. OpenAI has made ChatGPT free to use.

Can chatbot be tracked? ›

The nature of a chatbot means that it will always reveal information about the user, regardless of how the service is used, says Moscona. “Even if you use a chatbot through an anonymous account or a VPN, the content you provide over time could reveal enough information to be identified or tracked down.”

Can you be aware of your own psychosis? ›

People who have psychotic episodes are often totally unaware their behaviour is in any way strange or that their delusions or hallucinations are not real. They may recognise delusional or bizarre behaviour in others, but lack the self-awareness to recognise it in themselves.

Why do I have 2 voices in my head? ›

There are many significant factors that can cause hearing voices. The major factors that contribute to this condition are stress, anxiety, depression, and traumatic experiences. In some cases, there might be environmental and genetic factors that cause such hearing of voices.

What mental illness causes hallucinations? ›

Hallucinations occur frequently in psychiatric conditions such as schizophrenia, bipolar disorder, post-traumatic disorder and borderline personality disorder, as well as in other disorders such as dementia and Parkinson's.

What are the weaknesses of ChatGPT? ›

What Are the Limitations of ChatGPT?
  • Incorrect answers.
  • Biased answers.
  • Lack of human insight.
  • Overly long (wordy) answers.
Apr 20, 2023

Is ChatGPT reliable for research? ›

ChatGPT is not a useful tool for writing reliable scientific texts without strong human intervention. It lacks the knowledge and expertise necessary to accurately and adequately convey complex scientific concepts and information.” (23).

How much does ChatGPT cost per day? ›

SemiAnalysis' Chief Analyst Dylan Patel released the report this week. According to his analysis, running ChatGPT costs approximately $700,000 a day. That breaks down to 36 cents for each question. ChatGPT creator OpenAI currently uses Nvidia GPUs for computing power for the bot as well as its other partner brands.

Can overthinking cause hallucinations? ›

Stress can exacerbate the symptoms of psychotic, mood, anxiety, and trauma disorders. And when these disorders are at a severe level is when the risk of psychosis is heightened. So, in a way, stress can indirectly cause hallucinations.

What are the rarest hallucinations? ›

Gustatory Hallucinations

Gustatory (taste) hallucinations are rare. Like olfactory hallucinations, they sometimes happen in conjunction with brain damage and seizures. Like olfactory hallucinations, they can pose particular distress when coupled with delusions.

How do you know if a hallucination is real? ›

A patient's reaction to hallucinations can be an indicator of authenticity. If the patient tries to get rid of the voices on his own, by playing music or humming, or seeking extra medication, this is a sign they are real.

What do most schizophrenics hallucinate? ›

Hearing voices or other sounds is the most common hallucination.

Do schizophrenics know they are hallucinating? ›

It is possible to experience hallucinations while being aware that they aren't real. As with delusions, this would require a meta-awareness of the unreality of what appears to be a real experience.

What are very vivid hallucinations? ›

Peduncular hallucinosis is a rare neurological phenomenon typically characterized by vivid, brightly colored visual hallucinations of people and objects. Most frequently, lesions are localized to the thalamus, midbrain, and brainstem.

What is the first stage of hallucination? ›

Stage 1. Also referred to as the comforting stage,a person may begin to experience a sense of anxiety, loneliness or guilt that can cause them to focus obsessively on thoughts that will relieve those feelings. However, the sufferer realizes the thoughts are their own and finds that they can control them.

What is a Pseudohallucination? ›

Pseudohallucinations are vivid mental images, which however differ from true hallucinations in that they lack the full qualities of true perceptions. For example, an auditory hallucination may be heard within one's own head rather than perceived as coming from the outside world.

What is Alogia? ›

Alogia is a symptom, not a stand-alone diagnosis. It's defined in the Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-5) as “poverty of speech” and “poverty of content.”

What is the smartest AI chat? ›

The best overall AI chatbot is the new Bing due to its exceptional performance, versatility, and free availability. It uses OpenAI's cutting-edge GPT-4 language model, making it highly proficient in various language tasks, including writing, summarization, translation, and conversation.

What will ChatGPT be used for? ›

ChatGPT is a natural language processing tool driven by AI technology that allows you to have human-like conversations and much more with the chatbot. The language model can answer questions and assist you with tasks, such as composing emails, essays, and code.

What are simple visual hallucinations? ›

Visual hallucinations caused by seizures have often been described as simple, brief, and consistent for each patient; they usually consist of small, brightly colored spots or shapes that flash.

Can anxiety make you see things that aren't there? ›

People with anxiety and depression may experience periodic hallucinations. The hallucinations are typically very brief and often relate to the specific emotions the person is feeling.

What are four types of delusions? ›

Types of delusions include persecutory, erotomanic , grandiose , jealous, somatic, and mixed/unspecific.

Are chat bots real people? ›

Chatbots – also known as “conversational agents” – are software applications that mimic written or spoken human speech for the purposes of simulating a conversation or interaction with a real person. There are two primary ways chatbots are offered to visitors: via web-based applications or standalone apps.

Are chatbots self aware? ›

Technologists broadly agree that AI chatbots are not self-aware just yet, but there is some thought that we may have to re-evaluate how we talk about sentience. ChatGPT and other new chatbots are so good at mimicking human interaction that they've prompted a question among some: Is there any chance they're conscious?

Can you get caught using ChatGPT? ›

The free version of ChatGPT relies on the GPT-3.5 language model, which is easy to catch by most AI detectors. GPT-4, which is currently only available through ChatGPT Plus, is more difficult to detect. Still, many AI detectors can detect GPT-4, so using it to write your assignments isn't foolproof.

How do you know if my students are using ChatGPT? ›

Turnitin for detecting Chat GPT.

One of the most effective tools is Turnitin , widely used by educational institutions to detect plagiarism. Turnitin uses a combination of machine learning and human expertise to identify text that chat GPT models have generated.

Does ChatGPT show up on Turnitin? ›

Yes, Turnitin can detect Chat GPT and other AI-assisted writing. Turnitin's AI detection technology has been developed to keep up with the advancements in AI writing tools. Educators and lecturers can use this technology to check students' work for plagiarism.

Who owns ChatGPT? ›

Who Created ChatGPT? Chat GPT is owned and developed by OpenAI, a leading artificial intelligence research and deployment company based in San Francisco that was launched in December 2015.

How long is ChatGPT free? ›

Yes, ChatGPT is unlimited in use and free to use just as long as you can access it. This may be restricted by demand – like when the ChatGPT is at capacity – and you may experience slower response times. However, there appears no hard limit on use when in the application.

Is ChatGPT better than Google? ›

Chat GPT wins on overall feature set, but Google Bard wins on computational resources. And don't expect Google to accept overall second place. It is throwing a lot of resources at the AI competition.

What are the risks of chatbot? ›

But the way these products work—receiving instructions from users and then scouring the internet for answers—creates a ton of new risks. With AI, they could be used for all sorts of malicious tasks, including leaking people's private information and helping criminals phish, spam, and scam people.

How do I make chatbot text undetectable? ›

How to make ChatGPT text undetectable
  1. Rewrite the following text so that it's harder to detect as coming from ChatGPT by a ChatGPT detector.
  2. Rewrite the following text. Write like a human, more spoken language, more line breaks.
  3. Paraphrase the following, use random words and rare words. Write more like a human.
May 4, 2023

How do I know if a text was written by AI? ›

GLTR is currently the most visual way to predict if casual portions of text have been written with AI. To use GLTR, simply copy and paste a piece of text into the input box and hit "analyze."

What issues can you have with hallucinations? ›

Hallucinations occur frequently in psychiatric conditions such as schizophrenia, bipolar disorder, post-traumatic disorder and borderline personality disorder, as well as in other disorders such as dementia and Parkinson's.

What is the disorder of seeing hallucinations? ›

Psychosis (schizophrenia/schizoaffective disorder).

While the majority of hallucinations reported in primary psychotic disorders are auditory, they may also be visual, olfactory, tactile, or gustatory. Visual hallucinations have been reported in 16%–72% of patients with schizophrenia and schizoaffective disorder.

What is an example of hallucinations in GPT? ›

Examples of Hallucinations

Hallucinations in ChatGPT can range from minor inaccuracies to completely erroneous responses. For example, ChatGPT might generate a plausible-sounding answer to a factual question that is completely incorrect, such as an erroneous date for the creation of the Mona Lisa (source).

What are the risks of hallucinations? ›

In some severe cases, fear and paranoia triggered by hallucinations can lead to dangerous actions or behaviors. Stay with the person at all times and go with them to the doctor for emotional support. You may also be able to help answer questions about their symptoms and how often they occur.

What brain damage causes hallucinations? ›

Posttraumatic delirium

The typical symptoms shown are restlessness, confusion, disorientation, hallucinations, agitation, and delusions. Delirium occurs by the effect of injury on brain tissue chemicals. However, there are other mechanisms that can cause posttraumatic delirium in TBI patients.

What causes scary hallucinations? ›

Common causes of hallucinations include: mental health conditions like schizophrenia or a bipolar disorder. drugs and alcohol. Alzheimer's disease or Parkinson's disease.

What is the most common form of hallucination? ›

Hearing voices in the mind is the most common type of hallucination in people with mental health conditions such as schizophrenia.

Can extreme anxiety cause hallucinations? ›

Hallucinations are not among the more common symptoms of anxiety disorders. But they can occur in people who have various types of anxiety. A study in the January 2016 edition of the journal Consciousness and Cognition documented a connection between anxiety and auditory hallucinations.

What are the most common psychotic hallucinations? ›

[2] The most common hallucinations in schizophrenia are auditory, followed by visual. Tactile, olfactory and gustatory are reported less frequently [Table 1]. [3] Visual hallucinations in schizophrenia have a predominance of denatured people, parts of bodies, unidentifiable things and superimposed things.

What are factual hallucinations? ›

Abstract. State-of-the-art abstractive summarization systems often generate hallucinations; i.e., content that is not directly inferable from the source text. Despite being assumed to be incorrect, we find that much hallucinated content is actually consistent with world knowledge, which we call factual hallucinations.

What are OCD hallucinations like? ›

Many people with OCD also experience quasi-hallucinations. With a quasi-hallucination, you know it's not real, but the feeling is still strong. For example, you might feel dirt on your skin and have a compulsion to wash it off, even though you know there isn't really dirt on your skin.

Are hallucinations a mental health issue? ›

When not related to substance abuse, hallucinating can be a symptom of a mental illness. Hallucinations are experienced most commonly in schizophrenia, but can also be found in schizoaffective disorder and bipolar disorder. For more information please see our section on Hearing voices.

What happens in the brain during hallucinations? ›

Hallucinations are conscious perception-like experiences that are a common symptom of schizophrenia spectrum disorders (SSD). Current neuroscience evidence suggests several brain areas are involved in the generation of hallucinations including the sensory cortex, insula, putamen, and hippocampus.


1. Did ChatGPT Become Useless Already?!
(Simon Høiberg)
2. AI Jobpocalypse: How tech like ChatGPT will impact your life by 2030 (and what to do about it)
(David Shapiro ~ AI)
3. Stanford Webinar - How [You] Can Use ChatGPT to Increase Your Creative Output
(Stanford Online)
4. Here's Why ChatGPT Should Be DESTROYED..
(Tech Splash)
5. AI and ChatGPT. Should We Be Worried?
(National Association of Scholars)
6. A chat on ChatGPT | Conversation with Pro ChatGPT data scientist
(Elyas Tech)


Top Articles
Latest Posts
Article information

Author: Catherine Tremblay

Last Updated: 29/09/2023

Views: 5943

Rating: 4.7 / 5 (47 voted)

Reviews: 86% of readers found this page helpful

Author information

Name: Catherine Tremblay

Birthday: 1999-09-23

Address: Suite 461 73643 Sherril Loaf, Dickinsonland, AZ 47941-2379

Phone: +2678139151039

Job: International Administration Supervisor

Hobby: Dowsing, Snowboarding, Rowing, Beekeeping, Calligraphy, Shooting, Air sports

Introduction: My name is Catherine Tremblay, I am a precious, perfect, tasty, enthusiastic, inexpensive, vast, kind person who loves writing and wants to share my knowledge and understanding with you.