  Generative image reconstruction algorithms such as measurement conditioned
diffusion models are increasingly popular in the field of medical imaging.
These powerful models can transform low signal-to-noise ratio (SNR) inputs into
outputs with the appearance of high SNR. However, the outputs can have a new
type of error called hallucinations. In medical imaging, these hallucinations
may not be obvious to a Radiologist but could cause diagnostic errors.
Generally, hallucination refers to error in estimation of object structure
caused by a machine learning model, but there is no widely accepted method to
evaluate hallucination magnitude. In this work, we propose a new image quality
metric called the hallucination index. Our approach is to compute the Hellinger
distance from the distribution of reconstructed images to a zero hallucination
reference distribution. To evaluate our approach, we conducted a numerical
experiment with electron microscopy images, simulated noisy measurements, and
applied diffusion based reconstructions. We sampled the measurements and the
generative reconstructions repeatedly to compute the sample mean and
covariance. For the zero hallucination reference, we used the forward diffusion
process applied to ground truth. Our results show that higher measurement SNR
leads to lower hallucination index for the same apparent image quality. We also
evaluated the impact of early stopping in the reverse diffusion process and
found that more modest denoising strengths can reduce hallucination. We believe
this metric could be useful for evaluation of generative image reconstructions
or as a warning label to inform radiologists about the degree of hallucinations
in medical images.
