  Many widely used datasets for graph machine learning tasks have generally
been homophilous, where nodes with similar labels connect to each other.
Recently, new Graph Neural Networks (GNNs) have been developed that move beyond
the homophily regime; however, their evaluation has often been conducted on
small graphs with limited application domains. We collect and introduce diverse
non-homophilous datasets from a variety of application areas that have up to
384x more nodes and 1398x more edges than prior datasets. We further show that
existing scalable graph learning and graph minibatching techniques lead to
performance degradation on these non-homophilous datasets, thus highlighting
the need for further work on scalable non-homophilous methods. To address these
concerns, we introduce LINKX -- a strong simple method that admits
straightforward minibatch training and inference. Extensive experimental
results with representative simple methods and GNNs across our proposed
datasets show that LINKX achieves state-of-the-art performance for learning on
non-homophilous graphs. Our codes and data are available at
https://github.com/CUAI/Non-Homophily-Large-Scale.
