  Text-based image captioning is an important but under-explored task, aiming
to generate descriptions containing visual objects and scene text. Recent
studies have made encouraging progress, but they are still suffering from a
lack of overall understanding of scenes and generating inaccurate captions. One
possible reason is that current studies mainly focus on constructing the
plane-level geometric relationship of scene text without depth information.
This leads to insufficient scene text relational reasoning so that models may
describe scene text inaccurately. The other possible reason is that existing
methods fail to generate fine-grained descriptions of some visual objects. In
addition, they may ignore essential visual objects, leading to the scene text
belonging to these ignored objects not being utilized. To address the above
issues, we propose a DEpth and VIsual ConcEpts Aware Transformer (DEVICE) for
TextCaps. Concretely, to construct three-dimensional geometric relations, we
introduce depth information and propose a depth-enhanced feature updating
module to ameliorate OCR token features. To generate more precise and
comprehensive captions, we introduce semantic features of detected visual
object concepts as auxiliary information. Our DEVICE is capable of generalizing
scenes more comprehensively and boosting the accuracy of described visual
entities. Sufficient experiments demonstrate the effectiveness of our proposed
DEVICE, which outperforms state-of-the-art models on the TextCaps test set. Our
code will be publicly available.
