  Multimodal knowledge graph link prediction aims to improve the accuracy and
efficiency of link prediction tasks for multimodal data. However, for complex
multimodal information and sparse training data, it is usually difficult to
achieve interpretability and high accuracy simultaneously for most methods. To
address this difficulty, a new model is developed in this paper, namely
Interpretable Multimodal Knowledge Graph Answer Prediction via Sequence
Modeling (IMKGA-SM). First, a multi-modal fine-grained fusion method is
proposed, and Vgg16 and Optical Character Recognition (OCR) techniques are
adopted to effectively extract text information from images and images. Then,
the knowledge graph link prediction task is modelled as an offline
reinforcement learning Markov decision model, which is then abstracted into a
unified sequence framework. An interactive perception-based reward expectation
mechanism and a special causal masking mechanism are designed, which "converts"
the query into an inference path. Then, an autoregressive dynamic gradient
adjustment mechanism is proposed to alleviate the insufficient problem of
multimodal optimization. Finally, two datasets are adopted for experiments, and
the popular SOTA baselines are used for comparison. The results show that the
developed IMKGA-SM achieves much better performance than SOTA baselines on
multimodal link prediction datasets of different sizes.
