  Abductive reasoning aims to make the most likely inference for a given set of
incomplete observations. In this paper, we introduce a novel research task
known as "abductive action inference" which addresses the question of which
actions were executed by a human to reach a specific state shown in a single
snapshot. The research explores three key abductive inference problems: action
set prediction, action sequence prediction, and abductive action verification.
To tackle these challenging tasks, we investigate various models, including
established ones such as Transformers, Graph Neural Networks, CLIP, BLIP, GPT3,
end-to-end trained Slow-Fast, Resnet50-3D, and ViT models. Furthermore, the
paper introduces several innovative models tailored for abductive action
inference, including a relational graph neural network, a relational bilinear
pooling model, a relational rule-based inference model, a relational GPT-3
prompt method, and a relational Transformer model. Notably, the newly proposed
object-relational bilinear graph encoder-decoder (BiGED) model emerges as the
most effective among all methods evaluated, demonstrating good proficiency in
handling the intricacies of the Action Genome dataset. The contributions of
this research offer significant progress toward comprehending the implications
of human actions and making highly plausible inferences concerning the outcomes
of these actions.
