  The ability of reasoning beyond data fitting is substantial to deep learning
systems in order to make a leap forward towards artificial general
intelligence. A lot of efforts have been made to model neural-based reasoning
as an iterative decision-making process based on recurrent networks and
reinforcement learning. Instead, inspired by the consciousness prior proposed
by Yoshua Bengio, we explore reasoning with the notion of attentive awareness
from a cognitive perspective, and formulate it in the form of attentive message
passing on graphs, called neural consciousness flow (NeuCFlow). Aiming to
bridge the gap between deep learning systems and reasoning, we propose an
attentive computation framework with a three-layer architecture, which consists
of an unconsciousness flow layer, a consciousness flow layer, and an attention
flow layer. We implement the NeuCFlow model with graph neural networks (GNNs)
and conditional transition matrices. Our attentive computation greatly reduces
the complexity of vanilla GNN-based methods, capable of running on large-scale
graphs. We validate our model for knowledge graph reasoning by solving a series
of knowledge base completion (KBC) tasks. The experimental results show
NeuCFlow significantly outperforms previous state-of-the-art KBC methods,
including the embedding-based and the path-based. The reproducible code can be
found by the link below.
