  Web automation holds the potential to revolutionize how users interact with
the digital world, offering unparalleled assistance and simplifying tasks via
sophisticated computational methods. Central to this evolution is the web
element nomination task, which entails identifying unique elements on webpages.
Unfortunately, the development of algorithmic designs for web automation is
hampered by the scarcity of comprehensive and realistic datasets that reflect
the complexity faced by real-world applications on the Web. To address this, we
introduce the Klarna Product Page Dataset, a comprehensive and diverse
collection of webpages that surpasses existing datasets in richness and
variety. The dataset features 51,701 manually labeled product pages from 8,175
e-commerce websites across eight geographic regions, accompanied by a dataset
of rendered page screenshots. To initiate research on the Klarna Product Page
Dataset, we empirically benchmark a range of Graph Neural Networks (GNNs) on
the web element nomination task. We make three important contributions. First,
we found that a simple Convolutional GNN (GCN) outperforms complex
state-of-the-art nomination methods. Second, we introduce a training refinement
procedure that involves identifying a small number of relevant elements from
each page using the aforementioned GCN. These elements are then passed to a
large language model for the final nomination. This procedure significantly
improves the nomination accuracy by 16.8 percentage points on our challenging
dataset, without any need for fine-tuning. Finally, in response to another
prevalent challenge in this field - the abundance of training methodologies
suitable for element nomination - we introduce the Challenge Nomination
Training Procedure, a novel training approach that further boosts nomination
accuracy.
