  While large language models (LLMs) have taken great strides towards helping
humans with a plethora of tasks like search and summarization, hallucinations
remain a major impediment towards gaining user trust. The fluency and coherence
of model generations even when hallucinating makes it difficult to detect
whether or not a model is hallucinating. In this work, we explore if the
artifacts associated with the model generations can provide hints that the
generation will contain hallucinations. Specifically, we probe LLMs at 1) the
inputs via Integrated Gradients based token attribution, 2) the outputs via the
Softmax probabilities, and 3) the internal state via self-attention and
fully-connected layer activations for signs of hallucinations on open-ended
question answering tasks. Our results show that the distributions of these
artifacts differ between hallucinated and non-hallucinated generations.
Building on this insight, we train binary classifiers that use these artifacts
as input features to classify model generations into hallucinations and
non-hallucinations. These hallucination classifiers achieve up to 0.80 AUROC.
We further show that tokens preceding a hallucination can predict the
subsequent hallucination before it occurs.
