  In recent years, machine-learning methods have become increasingly important
for the experiments at the Large Hadron Collider (LHC). They are utilised in
everything from trigger systems to reconstruction and data analysis. The recent
UCluster method is a general model providing unsupervised clustering of
particle physics data, that can be easily modified to provide solutions for a
variety of different decision problems. In the current paper, we improve on the
UCluster method by adding the option of training the model in a scalable and
distributed fashion, and thereby extending its utility to learn from
arbitrarily large data sets. UCluster combines a graph-based neural network
called ABCnet with a clustering step, using a combined loss function in the
training phase. The original code is publicly available in TensorFlow v1.14 and
has previously been trained on a single GPU. It shows a clustering accuracy of
81% when applied to the problem of multi-class classification of simulated jet
events. Our implementation adds the distributed training functionality by
utilising the Horovod distributed training framework, which necessitated a
migration of the code to TensorFlow v2. Together with using parquet files for
splitting data up between different compute nodes, the distributed training
makes the model scalable to any amount of input data, something that will be
essential for use with real LHC data sets. We find that the model is well
suited for distributed training, with the training time decreasing in direct
relation to the number of GPU's used. However, further improvements by a more
exhaustive and possibly distributed hyper-parameter search is required in order
to achieve the reported accuracy of the original UCluster method.
