  We evaluate ChatGPT's ability to solve algorithm problems from the CLRS
benchmark suite that is designed for GNNs. The benchmark requires the use of a
specified classical algorithm to solve a given problem. We find that ChatGPT
outperforms specialist GNN models, using Python to successfully solve these
problems. This raises new points in the discussion about learning algorithms
with neural networks and how we think about what out of distribution testing
looks like with web scale training data.
