  Visual information extraction (VIE), which aims to simultaneously perform OCR
and information extraction in a unified framework, has drawn increasing
attention due to its essential role in various applications like understanding
receipts, goods, and traffic signs. However, as existing benchmark datasets for
VIE mainly consist of document images without the adequate diversity of layout
structures, background disturbs, and entity categories, they cannot fully
reveal the challenges of real-world applications. In this paper, we propose a
large-scale dataset consisting of camera images for VIE, which contains not
only the larger variance of layout, backgrounds, and fonts but also much more
types of entities. Besides, we propose a novel framework for end-to-end VIE
that combines the stages of OCR and information extraction in an end-to-end
learning fashion. Different from the previous end-to-end approaches that
directly adopt OCR features as the input of an information extraction module,
we propose to use contrastive learning to narrow the semantic gap caused by the
difference between the tasks of OCR and information extraction. We evaluate the
existing end-to-end methods for VIE on the proposed dataset and observe that
the performance of these methods has a distinguishable drop from SROIE (a
widely used English dataset) to our proposed dataset due to the larger variance
of layout and entities. These results demonstrate our dataset is more practical
for promoting advanced VIE algorithms. In addition, experiments demonstrate
that the proposed VIE method consistently achieves the obvious performance
gains on the proposed and SROIE datasets.
