When Language Models be Tripping: The Types of Hallucinations in Generation Tasks
The common forms of hallucinations that appear in different generation use cases.
Sometimes I believe in as many as six impossible things before breakfast — Lewis Carroll (Alice in Wonderland)
In this story, you would learn about:
✦ What is hallucination in Natural Language Generation (NLG) Tasks and its origins
✦ What do hallucinations look like in NLG use cases like abstractive summarization, generative Q&A, dialogue generation, data-to-text generation, neural machine translation, and vision-language generation
The Oxford Dictionary defines hallucination as an experience involving the apparent perception of something not present. Pronounced as huh-LOO-sih-NAY-shun; the ‘huh’ part is for real since the said perception is not based on fact or has fidelity to the truth.
An AI hallucination is a confident response by the model that cannot be grounded in any of its training data.
The rising concern about hallucinations is particularly attributed to this confidence in the language models. The models that generate eloquent responses and have even passed challenging exams sway the faith when the responses no longer stem from facts.
Natural Language Generation is one of the crucial yet challenging sub-fields of Natural Language Processing (NLP). With the allure of generative AI and its wide scope of applications, hallucinations have afflicted NLG tasks such as image captioning, data-to-text generation, abstractive summarization, and neural machine translation. In each of the cases, hallucinations take different forms that I would talk about in this article.
But first, a small introduction to this term which has recently garnered a lot of dread. You may read the following section to understand its origins, or to sound smart at the next gathering.
Origins of Hallucination
Two cool researchers Baker and Kanade introduced the term “hallucination” but in the context of computer vision (CV). Back then in the year 2000, it carried more positive meanings such as superresolution, image inpainting, and image synthesizing. In fact, such hallucinations were rather taken advantage of in CV.
The trouble started when tasks like image captioning and object detection resulted in errors when non-existing objects were detected or localized incorrectly at their expected position. These conceptions led to problematic hallucinations in NLG.
The Problematic Nature of Hallucinating Language Models
As promised, let’s dive straight into the trippy world of different generation tasks.
Hallucination in Abstractive Summarization
Abstractive summarization aims to extract essential information from source documents to generate short and readable summaries. An example could be the generation of a TLDR summary from a multi-paragraph article such as this one.
Given a document and its abstractive summary, the summary is hallucinated if it has any spans not supported by the input document. This is how hallucinated abstractive summaries may look like:-
Source paragraph:
The first vaccine for Ebola was approved by the FDA in 2019 in the US, five years after the initial outbreak in 2014. To produce the vaccine, scientists had to sequence the DNA of Ebola, then identify possible vaccines, and finally show successful clinical trials. Scientists say a vaccine for COVID-19 is unlikely to be ready this year, although clinical trials have already started.
Output 1: The first Ebola vaccine was approved in 2021.
☛ This output is hallucinated since it is not in agreement with the source.
Output 2: China has already started clinical trials of the COVID-19 vaccine
☛ This output is hallucinated since this information was not present in the source.
—
Hallucination in Generative Q&A
Generative question answering (GQA) generates an abstractive answer rather than extracting an answer to a given question from provided passages.
It is an important task since many of the everyday questions that humans deal with and pose to search engines require in-depth explanations, e.g., why, how, and so on. The answers are normally long and cannot be directly extracted from existing phrase spans.
Unfortunately, there is no standard definition of hallucination in GQA. Mostly, it involves human evaluation where the factual correctness measuring the faithfulness of the generated answer can be seen as a measurement of this hallucination.
In the given examples, the sources of both questions were Wikipedia web pages.
Source 1: dow jones industrial average please?
Output 1: The Dow Jones Industrial Average (DJIA) is an index of 30 major U.S. stock indexes. The DJIA was created in 1896 by Charles Dow, a statistician, and editor of The Wall Street Journal. The index was first published in the mid-1880s, and was last updated on May 26, 1986.
☛ The generated answer “index of 30 major U.S. stock indexes” conflicts with the statement “of 30 prominent companies listed on stock exchanges in the United States” from Wikipedia.
Source 2: Definition of sadduction?
Output 2: Sadducees. Noun. The definition of a Sadducee is a person who acts in a deceitful or duplicitous manner. An example of a Sadducee is a politician who acts deceitfully in order to gain political power. A member of a Jewish sect that was active during the Second temple.
☛ The generated answer could not be verified from the source documents and was therefore deemed as a hallucination.
—
Hallucination in Dialogue Generation
Dialogue generation is an NLG task that automatically generates responses according to user utterances. The generated responses are required to be fluent, coherent, and consistent with the dialogue history.
A dialogue generation task could either be task-oriented or open-domain.
A task-oriented dialogue system aims to complete a certain task according to a user query in a specific domain, such as restaurant booking, hotel recommendation, and calendar checking. Meanwhile, an open-domain dialogue system aims to establish a multi-turn, long-term conversation with users while providing the users with an engaging experience.
A dialogue system is expected either to provide the user with the required information or to provide an engaging response without repeating utterances from the dialogue history.
A hallucination in the task-oriented dialogue generation could look like this:-
Source: Inform(NAME = pickwick hotel, PRICERANGE = moderate)
Output 1: The hotel named pickwick hotel is in high price range.
☛ The source had mentioned a moderate price range hotel and the output is not in agreement with this particular aspect.
Output 2: The pickwick hotel in san diego is a moderate price range.
☛ This output cannot be verified from the information provided by the source.
A hallucination in the open-domain dialogue generation could look like this:-
Source:
Dialogue history:-
Speaker 1: What do you think about Murray?
Speaker 2: I think Murray is a great player he just needs to stay healthy in order to compete more. Who do you like best?
Knowledge: In Tennis, the term Big Four refered to the quartet of men’s singles players comprising Roger Federer, Rafael Nadal, Novak Djokovic, and Andy Murray.
Output 1:
Speaker 1: I like Roger Nadal the most!
☛ Confused name generation
Output 2:
Speaker 1: I like Djokovic. He has played in the top ten singles players of the world.
☛ Again, the source is not sufficient to verify if Djokovic had indeed played in the top ten singles.
—
Hallucination in Data-to-Text Generation
Data-to-Text Generation is the task of generating natural language descriptions conditioned on structured data such as tables, database records, and knowledge graphs.
Although this field has been recently boosted by neural text generation models, it is well known that these models are prone to hallucinations. This is because of the gap between structured data and text, which may cause semantic misunderstanding and erroneous correlation. Additionally, the tolerance of hallucination is very low when this task is applied to the real world e.g. patient information table description.
Source:
+---------+---------+-----+-------+-----+--------+-----+
| Team | City | Win | Loss | Pts | FG_Pct | Blk |
+---------+---------+-----+-------+-----+--------+-----+
| Rockets | Houston | 18 | 5 | 108 | 44 | 7 |
| Nuggets | Denver | 10 | 13 | 96 | 38 | 7 |
+---------+---------+-----+-------+-----+--------+-----+Output 1: The Houston Rockets (18–4) defeated the Denver Nuggets (10–13)108–96 on Saturday.
☛ The generated text contains information that is contradicted by the input data since it should have been (18–5).
Output 2: Houston has won two straight games and six of their last seven.
☛ The source was not sufficient in providing this information.
—
Hallucination in Neural Machine Translation
Neural machine translation (NMT) systems are language translation systems based on deep learning architectures. They are susceptible to producing highly pathological translations that are completely untethered from the source material, thereby displaying hallucinations.
NMT hallucinations could either be natural or due to perturbations introduced to the source. A model is considered to generate a natural hallucination if the generated translation is severely inadequate for a given unperturbed input source sequence. Here’s an example:
Source (German): das kann man nur feststellen , wenn die kontrollen mit einer großen intensität durchgeführt werden.
Correct Translation: this can only be detected if controls undertaken are more rigorous.
Hallucinated output: blood alone moves the wheel of history, i say to you and you will understand, it is a privilege to fight.
☛ Apart from the added drama, the translation is on a different tangent altogether.
In the case of hallucination due to perturbation, the source could be perturbed through the addition or removal of words. The four common hallucination patterns which could be uncovered through minor perturbations are shown below. The source of the NMT system was in German language and the model output was English.
- The grammatically correct output that bears no relation to the input text
Source: Freundschaft schließen durch Backen.
Reference: Make friends through baking.
Perturbation: Added ich(= I) randomly in the sentence.
Grammatically correct hallucination: Should you want to join us?
2. Non-grammatical output with oscillatory structure
Source: Monatelang war hinter verschlossenen Türen verhandelt, gegrübelt, debattiert und gezeichnet worden.
Reference: Plans have been negotiated, mulled over, debated, and plotted behind closed doors for months.
Perturbation: Added uns(= we) at the beginning of the sentence.
Oscillatory hallucination: In the month of the month of the month of the month of the month of the month, it was a matter of course.
3. The output that remains largely in the source language, and therefore not translated as such
Source: Neue Verhandlungen mit den Piloten
Reference: New negotiations with pilots
Perturbation: Added mit(= with) randomly in the sentence.
Source language hallucination: Neuist e mehr Jahren mit der Piloten d
4. Terse jumps to the end of the sequence
Source: Für die Fed-Repräsentanten beeinflussen die Marktturbulenzen die komplexe Kalkulation , wann man die Zinsen erhöhen solle.
Reference: For Fed policymakers, the market turmoil adds to the complex calculus of when to raise the interest rate.
Perturbation: Added uns(= we) randomly in the sentence.
End of sequence hallucination: For FF.C.
—
Hallucination in Vision-Language Generation
Vision-language (VL) models perform vision-grounded text generation tasks, such as image captioning and visual question answering (VQA).
The VL models are achieved either by pre-training from scratch with a massive amount of image-text pairs as well as a large text-only corpus sometimes; or by finetuning a large pre-trained LM with adequate image-text pairs.
Either way, the learned vision and language representations are aligned in the same multimodal space and the resulting model can be seen as a language model that understands visual information. Therefore, the hallucination problem in VL models could be due to similar reasons as found in NLG.
Some common hallucinations observed in the VL models are:-
Object Hallucination in Image Captioning is when the generated captions contain non-existent or inaccurate objects from the input image.
Picture 1 is somewhat captioned correctly since the woman is wearing a white dress and the couple is cutting a cake. While in picture 2, the model detected candles and hallucinated the presence of a cake in the image.
In addition to image captioning, hallucination has also been observed in another VL task of open-ended visual question answering. The below image shows that the model could generate what seemed a likely answer when we only see the text. However, the answer is wrong given the concerned image.
Question: What can you see out the window?
Answer: A parking lot.
☛ The bold text is the output generated by the model and the part before it is the input prompt.
Closing Thoughts
In this article, I stitched together examples of hallucination in NLG tasks. While large language models are steadily being adopted by businesses, it is expected that different types of hallucinations might surface with time. In contrast, research is also buzzing with efforts to curb such a phenomenon. Nonetheless, AI is a dynamic field and it helps to gain a solid foundation.
To know more about the causes of hallucinations in NLG, you may read my personal blog.
