  Graph Neural Networks (GNNs) are a powerful representational tool for solving
problems on graph-structured inputs. In almost all cases so far, however, they
have been applied to directly recovering a final solution from raw inputs,
without explicit guidance on how to structure their problem-solving. Here,
instead, we focus on learning in the space of algorithms: we train several
state-of-the-art GNN architectures to imitate individual steps of classical
graph algorithms, parallel (breadth-first search, Bellman-Ford) as well as
sequential (Prim's algorithm). As graph algorithms usually rely on making
discrete decisions within neighbourhoods, we hypothesise that
maximisation-based message passing neural networks are best-suited for such
objectives, and validate this claim empirically. We also demonstrate how
learning in the space of algorithms can yield new opportunities for positive
transfer between tasks---showing how learning a shortest-path algorithm can be
substantially improved when simultaneously learning a reachability algorithm.
