Free online consultation available. Please feel free to contact us.

Hallucination

Hallucination

This glossary explains various keywords that will help you understand the mindset necessary for data utilization and successful DX.

This time, we will explain hallucination, a phenomenon in generative AI that should be taken into consideration when using it in business.

Hallucination is

Hallucination is a term derived from the English word "hallucination," and refers to the phenomenon in which generating AI provides answers that are contrary to the facts, or are entirely fictional, as if they were true.

This is a distinctive behavior where they not only make mistakes, but also confidently state incorrect information, and sometimes tell strangely realistic fabricated stories that, to someone unfamiliar with the field, could easily be believed to be true. Many believe this phenomenon is an inevitable consequence of the workings of large-scale language models, and that it is difficult to completely eliminate this behavior.

A common problem when using generative AI in business is how to address hallucination, and how to deal with it is often a crucial point in the successful application of generative AI.

What is hallucination? (Examples of hallucination)

The current boom in generative AI began with the public release of ChatGPT on November 30, 2022. From its very beginnings, along with its astonishingly realistic user experience that made it seem as if it were having a conversation with a human, its characteristic of "casually creating fabricated stories" also became widely known.

In particular, the large-scale language models (LLMs) used in early generative AI were not at the level they are today, so they often produced blatantly fabricated stories. The following is not an actual conversation, but a fictionalized example of the "blatant hallucination" seen in the early days of generative AI (i.e., it does not reflect historical facts), but I think it was quite common for things to turn out like this.

Please tell me about a famous Sengoku period warlord who was based in what is now Ishikawa Prefecture.

When it comes to Sengoku period warlords based in Ishikawa Prefecture, Oda Nobunaga comes to mind. He was a famous Sengoku daimyo who was active with Kanazawa Castle as his base.
*Incorrect: The Sengoku period warlord based in Ishikawa Prefecture was Maeda Toshiie.

Please tell me about some famous historical events involving Oda Nobunaga.

One of Oda Nobunaga's most famous historical events is the Battle of Okehazama, where he fought against Shibata Katsuie. Oda Nobunaga ambushed Shibata's unsuspecting army and won, after which he was granted Kanazawa Castle.
*Incorrect: The opponent in the Battle of Okehazama was Imagawa Yoshimoto.

This characteristic phenomenon of calmly giving incorrect answers is called "hallucination."

Currently, the large-scale language models used by generative AI are trained using more data, and measures have been taken to suppress hallucination in the output by performing multiple processes internally (such as using multiple agents), so it is less common for them to give such blatantly incorrect answers.

However, hallucination hasn't completely disappeared. Sometimes, even when someone gives a fluent and detailed answer, you might find factual errors scattered throughout, or they might refer to non-existent sources. While it's good that the number of incorrect answers has decreased, it can also be seen as a more difficult situation because it's harder to spot the incorrect answers mixed in.

The origins of the Hallucinogenic religion

The exact cause of hallucination is not fully understood. Furthermore, it is considered difficult to completely eliminate hallucination through any countermeasures.

It's not just a problem with the training data.

I recall seeing explanations that hallucination occurs (especially in the early stages) because the training data contains errors. Indeed, training a large-scale language model with only incorrect data will result in incorrect output, and it is true that internet text data, which may contain errors, is used in training large-scale language models.

However, even if the data used for training is deemed problem-free, hallucination does not completely disappear; rather, it is thought to be a phenomenon that arises from the training process and response generation process of large-scale language models themselves.

For example, there's this incident: Facebook's Meta Inc. publicly released its own large-scale language model, "Galactica," in November 2022, but it was taken down after only three days.

⇒ MIT Tech Review: Why Meta's language AI "Battlestar Galactica" was withdrawn from public view after only 3 days

Galactica was a large-scale language model created with the aim of supporting scientists, trained on carefully selected data (academic papers, textbooks, encyclopedias, etc.).

However, while the output was highly plausible and sounded like it came from an expert, it frequently exhibited behaviors such as providing incorrect information or citing non-existent papers as sources. As a result, it was criticized for potentially presenting misinformation in an authoritative manner, leading to this outcome.

This is not a problem that arose from the advent of generative AI.

Furthermore, hallucination is not a term that has come into use since the advent of generative AI; it has been used for some time as a name for strange phenomena that occur in systems utilizing neural networks.

Before the boom in generative AI, there was a period when deep learning was popular. In 2018, Google researchers published a paper reporting the discovery of a strange phenomenon in neural machine translation (NMT) using deep learning: "text that was not at all present in the source text appeared in the translated text."

⇒Hallucinations in Neural Machine Translation

A troublesome problem has been discovered where nonsensical content that does not exist at all in the original text sometimes appears in the translation, and moreover, it appears in fluent sentences. This is why the title of the paper is "Hallucinations in Neural Machine Translation (NMT)".

Neural machine translation doesn't directly perform the same tasks as large-scale language models (at least it's not a "guess the next word" task), but similar strange phenomena occur.

Hallucination is often considered a problem inherent in large-scale language models themselves.

Research and development on countermeasures against hallucination, as well as research into why hallucination occurs, continues worldwide. However, the prevailing outlook is that hallucination is an inherent problem of large-scale language models themselves, and it may be impossible to completely eliminate it.

Generally, computers are perceived as operating logically and consistently, following the rules set by their programs. It's almost unthinkable that, when querying an employee database, a relational database (RDB) would output employee data that isn't registered in the database.

However, the large-scale language models that make generative AI possible operate probabilistically, "outputting the word with the highest probability of appearing next," so their operating principles are significantly different from those of typical IT systems.

​⇒Large Language Model (LLM) | Glossary

Let's talk a little more about the causes of hallucinations.

For example, it's assumed that the response isn't based on a logical understanding that "Oda Nobunaga was a Sengoku daimyo of Owari Province (not Kaga Province)," so it might fail to consider contexts unrelated to Owari Province and assume that "Oda Nobunaga" is the most likely answer following "Speaking of famous Sengoku daimyo," leading to this kind of behavior.

In addition to not actually understanding the meaning itself, the output is produced by the accumulation of processes that calculate the probability of the next word (token). Therefore, if a calculation error occurs along the way (for example, if the conversation proceeds with Oda Nobunaga), the output will continue based on that error, which can lead to a phenomenon where fluent responses continue with incorrect information.

In the example above, the model failed to adequately learn about the Sengoku-era daimyo of present-day Ishikawa Prefecture, and the probability distribution was not properly trained. As a result, it was influenced by the general statement that Oda Nobunaga is a Sengoku-era daimyo, leading to this outcome. While this can be attributed to insufficient training data, it's unrealistic to expect the model to have perfect training on every aspect of the universe, so such problems can occur in certain areas of study.

It's not easy to answer "I don't know."

In such cases, we would like the generating AI to respond with "I don't know," but there are reasons why this is difficult. The model that calculates the probability of the next word does not directly handle "do not output" or "output should be stopped due to insufficient data."

Furthermore, there are difficulties in training generating AI from user feedback. If the response is "I don't know," the conversation ends there. However, if the response is "plausible but actually incorrect," and "the user fails to notice this, the conversation continues." As a result, it is considered that the user was satisfied with the answer and the conversation continued, and there is a problem that "if the response allows the conversation to continue, the AI tends to learn in that direction."

Furthermore, this phenomenon does not occur when the model simply responds with the learned text, and it represents an undesirable aspect of the creativity (novelty) that is a useful feature of large-scale language models. For example, in the widely used application of generative AI, "using it to generate ideas," we can consider its ability to "skillfully create things that don't exist" to be beneficial to us.

Hallucination countermeasures

Various methods have been proposed to counter hallucination. While each of these measures is considered to have some potential effectiveness, it is also thought that each has its own limitations in terms of deterrent effect.

Please answer "I don't know."

One possible countermeasure to suppress hallucination is to add the prompt, "If you don't know, please answer 'I don't know'." However, since large-scale language models are not models that can directly understand that their own learning is insufficient, it seems unlikely that this countermeasure will completely suppress hallucination.

Clarify the intent of the prompt.

Some proposed strategies involve clarifying the intent of prompts to make it easier to understand what the generating AI should output.

While this method is generally useful for improving output and can be expected to clarify what kind of output is desired from the generating AI, it is unlikely to address hallucination, which occurs because the content to be output itself is problematic from the perspective of a large-scale language model. Therefore, this alone will not provide a complete solution.

Perform self-verification by showing the source (origin) and outputting the result.

There are also known methods to suppress unsubstantiated output by prompting the AI to "provide sources for its claims and output accordingly," thereby allowing it to self-verify its output.

However, this can lead to other forms of hallucination, such as suppressing useful output that the generating AI could have produced, fabricating non-existent sources, or referencing sources unrelated to the claims.

Instructing the generating AI on how to think (e.g., making it think step by step)

There are proposed methods to suppress hallucination by instructing the generative AI on the process by which it should think, such as by using "Chain-of-Thought prompting."

⇒ Chain-of-Thought Prompting | Glossary

For example, in the first example, the student is mistaken because they are influenced by the idea that "Oda Nobunaga is synonymous with Sengoku warlords." The problem could be mitigated by first having them list famous historical figures from Ishikawa Prefecture, and then instructing them to select a Sengoku warlord who was based in Ishikawa Prefecture from that list (or to answer "none" if there are none).

However, for prompt creators, it's not always clear what thought process the generating AI should follow to be effective in preventing hallucination, and some of these measures don't directly address the root cause of hallucination itself.

Verification by other AI agents

One method to suppress hallucination is to have other AI agents verify the output for errors. This mechanism is often used internally in generative AI services, often without our knowledge.

While this is an effective measure, it cannot be achieved solely through prompts; it requires the combined use of multiple generating AIs through custom development and manual work. Furthermore, the AI agent performing the verification operates on the same principle, which can lead to it overlooking incorrect outputs. Additionally, if the verification AI agent is made to make overly strict judgments and suppress the output, it may reject the output excessively, resulting in no output at all.

Import external data using search extension generation (RAG), etc.

There are measures to suppress hallucination, such as using Search Aggregation Generation (RAG) to import data from external sources that is necessary for the generating AI to produce appropriate answers.

⇒ Retrieval Augmented Generation (RAG) | Glossary

In some cases, hallucination can be caused by insufficient training data for large-scale language models. If the missing data can be supplemented from external sources as needed, a suppressive effect on this behavior can be expected.

However, considering the general usage of RAGs, while it is possible to improve responses to company-related issues by feeding company-related data to large-scale language models, it does not seem to be a means of generally suppressing hallucination.

Human-in-the-Loop (HITL)

This countermeasure assumes that the output of the generating AI may contain errors due to hallucination and other factors, and therefore, instead of using the output directly from the generating AI, it is used only after "human verification." Because it involves human intervention during the processing rather than relying on automated processing, this countermeasure is also called "Human-in-the-Loop" (HITL).

This method is effective as a countermeasure to many challenges in generative AI, not just hallucination. However, it can easily lead to a heavy workload for verification, causing staff to become exhausted and work to stagnate (despite the introduction of generative AI). Furthermore, problems can arise where people assume that the AI output is mostly correct and click OK without proper verification, leading to accidents.

Furthermore, hallucinations can be written in fluent language, but they can also contain subtle errors in points that are difficult for humans to understand or in highly specialized content, making it difficult to expect that a human reviewing them would lead to an accurate judgment.

Using the execution environment

For example, in applications where a generating AI is used to output program code, you can verify whether the outputted source code actually works by compiling and running it. There are also other cases where you can test the output of a generating AI, such as robot operation procedures, by actually executing them.

However, preparing and using an execution environment can easily become a high-cost undertaking, and while it may be possible to detect errors that prevent execution altogether, it may not be able to detect subtle errors. Furthermore, providing an execution environment to the generating AI may lead to unintended and dangerous behavior (such as unauthorized access to external IT systems), creating security risks.

Find creative ways to use it.

Alternatively, when using large-scale language models, hallucination is considered unavoidable, and one possible solution is to adjust the use case to match the characteristics that cause hallucination.

For example, one widely used application of generative AI is to "generate ideas," and in such applications, hallucination becomes less of a problem.

Generative AI needs to be used with comprehensive ingenuity, assuming that hallucination will occur.

As described above, there are various methods to combat hallucination, and each has its own effectiveness. However, unfortunately, it is impossible to completely eliminate hallucination, and we must consider that the only option is to reduce the probability of its occurrence.

It is necessary to adapt well to the characteristics of the generation AI.

The practical application of real-world generative AI involves devising ways to use it effectively in applications where hallucination is not a problem, or considering how to combine it effectively with human verification, assuming that hallucination exists.

Rather than being a technology that magically makes business desires and expectations a reality, it will require businesses to consider the technical characteristics of generative AI such as hallucination, devise ways to utilize it effectively, and engage in efforts to leverage what works well through trial and error to achieve results.

Trial and error becomes more important.

Given these characteristics of generative AI, traditional IT system development methods (such as the waterfall model), which involve defining requirements based on business needs and then determining the specifications of the IT system to be developed, often prove ineffective.

Even if you thoroughly analyze and plan beforehand to clarify "what needs to be made," if the actual creation often doesn't turn out as predicted, it becomes difficult to conduct effective analysis and planning. Alternatively, with generative AI, you might end up spending a lot of money and time trying to realize "what needs to be made" according to plan, which is technically difficult, but the results are often unsatisfactory.

Rather than spending time planning and analyzing, an approach that involves immediately creating and running an idea as soon as it comes to mind, and then using that feedback to consider what can be achieved from a business perspective (an agile approach) is more likely to lead to results.

Beyond simply determining whether a particular use seems impractical or potentially useful but requires careful consideration of its application area and operational ingenuity, the results of actual use can reveal unexpected applications that can lead to significant business success.

An environment that allows for trial and error and enables flexible collaboration is necessary.

In other words, to achieve results with generative AI, it is necessary to have an environment that allows for trial and error. Furthermore, in order to be able to devise various business applications, it is also necessary to have an environment that allows generative AI to be freely integrated with external systems and data.

Even in the techniques for preventing hallucination, some require collaboration with external parties and trial and error, such as adding cleverly designed prompts to the instructions of the generating AI, making decisions in conjunction with the results of some kind of verification work performed externally, supplying appropriate data from the company's IT system to achieve RAG, and experimenting with when and how to embed human judgment into the processing performed by the generating AI.

You might imagine a situation where you hire engineers and have them develop using Python, but in addition, these trials and errors need to be carefully considered from a business perspective, meaning that it is desirable for them to be led by non-engineers on the front lines (or even by the management themselves).

Connecting technologies that create an environment for successful use of generative AI

You might be wondering how we can create an environment where, as soon as you come up with an idea for how to use something, you can immediately "try and error" and "collaborate freely," and where non-engineers on the front lines of the business can take the lead in implementing it. But in fact, there are already ways to realize such an environment for use.

Please utilize "connecting" technology

There are ways to efficiently develop these various "integration processes" using only a GUI. These are "connecting" technologies such as "DataSpider" and "HULFT Square," also known as "EAI," "ETL," and "iPaaS." By utilizing these, it is possible to smoothly and efficiently integrate old and new systems.

The advantage of being able to use and develop using only a GUI.

Unlike regular programming, there is no need to write code. By placing and configuring icons on the GUI, you can achieve integration with a wide variety of systems, data, and cloud services.

No-code development using only a GUI might seem like a simplified and compromised approach compared to full-fledged programming. However, being able to develop using only a GUI makes it possible for "business personnel on the ground to proactively engage in Cloud integration themselves."

Full-scale processing can be implemented

Many products advertise that development can be done solely with a GUI, but some people may have a negative impression of such products, thinking they can only do basic things.

It is true that things like "it's easy to make, but it can only do simple things," "when I tried to execute a full-scale process it couldn't process and crashed," or "it didn't have the high reliability or stable operating capacity to support business operations, which caused problems" tend to occur.

"DataSpider" and "HULFT Square" are easy to use, but they also allow for the creation of processing at a level comparable to full-fledged programming. They have high processing power similar to full-fledged programming, such as being internally converted to Java and executed, and have a proven track record of supporting corporate IT for many years. They combine the advantages of a "GUI-only" interface with the proven track record and full capabilities of professional use.

It can be operated on-premises and can also be used as an i PaaS that does not require on-premises management.

With DataSpider, you can reliably operate it within your own managed systems. With HULFT Square, a cloud service (iPaaS), you can use this "connecting" technology itself as a cloud service, eliminating the hassle of in-house implementation and system operation.

For more information about data integration software "DataSpider," please see here.
⇒ DataSpider Servista | Services

For more information about HULFT Square, a cloud service that enables you to "connect" things, please see here.
⇒ HULFT Square | Services

Related keywords (for further understanding)

Keywords related to Generative AI/ChatGPT

Machine learning related keywords

Keywords related to data integration and system integration

  • EAI
    • This approach involves "connecting" systems through data integration, providing a means to freely link various data and systems. It's a concept that has been active long before the cloud era as a way to effectively utilize IT.
  • ETL
    • In the recent trend of actively working on data utilization, the majority of the work is not the data analysis itself, but rather the collection and preprocessing of data scattered in various places, from on-premise to cloud.
  • iPaaS
    • A cloud service that "connects" various clouds with external systems and data simply by operating on a GUI is called iPaaS.

Are you interested in "iPaaS" and "connecting" technologies?

Try out our products that enable you to seamlessly integrate various data and systems, from on-premises IT systems to cloud services, and successfully leverage IT.

The ultimate "connecting" tool: data integration software "DataSpider" and data integration platform "HULFT Square"

"DataSpider," data integration tool developed and sold by our company, is a "connecting" tool with a long history of success. "HULFT Square," a data integration platform, is a "connecting" cloud service developed using DataSpider technology.

Another feature is that development can be done using only the GUI (no code) without writing code like in regular programming, so business staff who have a good understanding of their company's business can take the initiative to use it.

Try out DataSpider/ HULFT Square 's "connecting" technology:

There are many simple collaboration tools on the market, but this tool can be used with just a GUI, is easy enough for even non-programmers to use, and has "high development productivity" and "full-fledged performance that can serve as the foundation for business (professional use)."

This solution smoothly resolves the problem of "connecting disparate systems and data," which often hinders the success of IT utilization. We regularly hold free trial sessions and hands-on workshops where you can try it out for free, so we encourage you to give it a try.


Why not try a PoC to see if "HULFT Square" can transform your business?

Why not try verifying how "connecting" can be utilized in your business, the feasibility of solving problems using data integration, and the benefits that can be obtained?

  • I want to automate data integration with SaaS, but I want to confirm the feasibility of doing so.
  • We want to move forward with data utilization, but there are challenges in system integration.
  • I want to consider data integration platform to achieve DX.

Glossary Column List

Alphanumeric characters and symbols

A row

Ka row

Sa row

Ta row

Na row

Ha row

Ma row

Ya row

Ra row

Wa row

»Data Utilization Column List

Recommended Content

Related Columns

Recommended Articles