Relational-Grid-World: A Novel Relational Reasoning Environment and An
  Agent Model for Relational Information Extraction
  Reinforcement learning (RL) agents are often designed specifically for a
particular problem and they generally have uninterpretable working processes.
Statistical methods-based agent algorithms can be improved in terms of
generalizability and interpretability using symbolic Artificial Intelligence
(AI) tools such as logic programming. In this study, we present a model-free RL
architecture that is supported with explicit relational representations of the
environmental objects. For the first time, we use the PrediNet network
architecture in a dynamic decision-making problem rather than image-based
tasks, and Multi-Head Dot-Product Attention Network (MHDPA) as a baseline for
performance comparisons. We tested two networks in two environments ---i.e.,
the baseline Box-World environment and our novel environment,
Relational-Grid-World (RGW). With the procedurally generated RGW environment,
which is complex in terms of visual perceptions and combinatorial selections,
it is easy to measure the relational representation performance of the RL
agents. The experiments were carried out using different configurations of the
environment so that the presented module and the environment were compared with
the baselines. We reached similar policy optimization performance results with
the PrediNet architecture and MHDPA; additionally, we achieved to extract the
propositional representation explicitly ---which makes the agent's statistical
policy logic more interpretable and tractable. This flexibility in the agent's
policy provides convenience for designing non-task-specific agent
architectures. The main contributions of this study are two-fold ---an RL agent
that can explicitly perform relational reasoning, and a new environment that
measures the relational reasoning capabilities of RL agents.
