🤪 Why Does Generative AI Make Mistakes?

"Hallucinate" was named the Word of the Year for 2023. In the context of AI, it refers to instances where a neural network generates false or nonsensical information. Let's explore why this happens, using examples from large language models (LLMs) like GPT-4 or Claude.

😒 How Do LLMs Search for Answers and Why Do They Lie?

Large language models are similar to T9, the predictive text function on smartphones. However, unlike T9, LLMs learn from vast amounts of data and consider the entire context of the text.

While LLMs can construct grammatically correct sentences, they lack an understanding of the meaning of their output. They choose words based on probability, "randomly" selecting the most suitable option. The internal workings of the model are complex and not entirely transparent, often described as a "black box."

In essence, LLMs know how to write but don't comprehend what they write. So, when ChatGPT outputs something nonsensical, the LLM deems that word combination most suitable, not because it "thinks" so.

➕ What Else Influences AI Hallucinations?

→ Errors, inaccuracies, and fakes in training datasets.

→ Incorrect context in the query.

→ Insufficient or excessive training.

👎 How Often Do Neural Networks Make Things Up?

According to research, ChatGPT fabricates ~3% of information. Meta's models ~5%. Claude 2 — 8%. Older models make more errors.

❌ What Can Be Done About Hallucinations?

→ Feedback — humans should comment on vulnerable topics. Development companies handle this.

→ Craft queries skillfully, providing sufficient context. Look at the guide on writing prompts.

OpenAI CEO Sam Altman believes the hallucination issue will be resolved within the next 2 years but added, "There is a balance between creativity and perfect accuracy, and models will have to learn when you want one or the other."

@hiaimediaen