  Neural algorithmic reasoning is an emerging area of machine learning focusing
on building models that can imitate the execution of classic algorithms, such
as sorting, shortest paths, etc. One of the main challenges is to learn
algorithms that are able to generalize to out-of-distribution data, in
particular with significantly larger input sizes. Recent work on this problem
has demonstrated the advantages of learning algorithms step-by-step, giving
models access to all intermediate steps of the original algorithm. In this
work, we instead focus on learning neural algorithmic reasoning only from the
input-output pairs without appealing to the intermediate supervision. We
propose simple but effective architectural improvements and also build a
self-supervised objective that can regularise intermediate computations of the
model without access to the algorithm trajectory. We demonstrate that our
approach is competitive to its trajectory-supervised counterpart on tasks from
the CLRS Algorithmic Reasoning Benchmark and achieves new state-of-the-art
results for several problems, including sorting, where we obtain significant
improvements. Thus, learning without intermediate supervision is a promising
direction for further research on neural reasoners.
