  Training graph neural networks (GNNs) on large-scale graphs can be
challenging due to the high computational expense caused by the massive number
of nodes and high-dimensional nodal features. Existing graph condensation
studies tackle this problem only by reducing the number of nodes in the graph.
However, the resulting condensed graph data can still be cumbersome.
Specifically, although the nodes of the Citeseer dataset are reduced to 0.9%
(30 nodes) in training, the number of features is 3,703, severely exceeding the
training sample magnitude. Faced with this challenge, we study the problem of
joint condensation for both features and nodes in large-scale graphs. This task
is challenging mainly due to 1) the intertwined nature of the node features and
the graph structure calls for the feature condensation solver to be
structure-aware; and 2) the difficulty of keeping useful information in the
condensed graph. To address these challenges, we propose a novel framework
TinyGraph, to condense features and nodes simultaneously in graphs.
Specifically, we cast the problem as matching the gradients of GNN weights
trained on the condensed graph and the gradients obtained from training over
the original graph, where the feature condensation is achieved by a trainable
function. The condensed graph obtained by minimizing the matching loss along
the training trajectory can henceforth retain critical information in the
original graph. Extensive experiments were carried out to demonstrate the
effectiveness of the proposed TinyGraph. For example, a GNN trained with
TinyGraph retains 98.5% and 97.5% of the original test accuracy on the Cora and
Citeseer datasets, respectively, while significantly reducing the number of
nodes by 97.4% and 98.2%, and the number of features by 90.0% on both datasets.
