  In this work, we propose and explore Deep Graph Value Network (DeepGV) as a
promising method to work around sample complexity in deep
reinforcement-learning agents using a message-passing mechanism. The main idea
is that the agent should be guided by structured non-neural-network algorithms
like dynamic programming. According to recent advances in algorithmic
alignment, neural networks with structured computation procedures can be
trained efficiently. We demonstrate the potential of graph neural network in
supporting sample efficient learning by showing that Deep Graph Value Network
can outperform unstructured baselines by a large margin in solving the Markov
Decision Process (MDP). We believe this would open up a new avenue for
structured agent design. See
https://github.com/drmeerkat/Deep-Graph-Value-Network for the code.
