  Graph Neural Networks (GNNs) have shown excellent performance on graphs that
exhibit strong homophily with respect to the node labels i.e. connected nodes
have same labels. However, they perform poorly on heterophilic graphs. Recent
approaches have typically modified aggregation schemes, designed adaptive graph
filters, etc. to address this limitation. In spite of this, the performance on
heterophilic graphs can still be poor. We propose a simple alternative method
that exploits Truncated Singular Value Decomposition (TSVD) of topological
structure and node features. Our approach achieves up to ~30% improvement in
performance over state-of-the-art methods on heterophilic graphs. This work is
an early investigation into methods that differ from aggregation based
approaches. Our experimental results suggest that it might be important to
explore other alternatives to aggregation methods for heterophilic setting.
