  Neural algorithmic reasoning studies the problem of learning algorithms with
neural networks, especially with graph architectures. A recent proposal, XLVIN,
reaps the benefits of using a graph neural network that simulates the value
iteration algorithm in deep reinforcement learning agents. It allows model-free
planning without access to privileged information about the environment, which
is usually unavailable. However, XLVIN only supports discrete action spaces,
and is hence nontrivially applicable to most tasks of real-world interest. We
expand XLVIN to continuous action spaces by discretization, and evaluate
several selective expansion policies to deal with the large planning graphs.
Our proposal, CNAP, demonstrates how neural algorithmic reasoning can make a
measurable impact in higher-dimensional continuous control settings, such as
MuJoCo, bringing gains in low-data settings and outperforming model-free
baselines.
