  In the context of End-to-End testing of web applications, automated
exploration techniques (a.k.a. crawling) are widely used to infer state-based
models of the site under test. These models, in which states represent features
of the web application and transitions represent reachability relationships,
can be used for several model-based testing tasks, such as test case
generation. However, current exploration techniques often lead to models
containing many near-duplicate states, i.e., states representing slightly
different pages that are in fact instances of the same feature. This has a
negative impact on the subsequent model-based testing tasks, adversely
affecting, for example, size, running time, and achieved coverage of generated
test suites. As a web page can be naturally represented by its tree-structured
DOM representation, we propose a novel near-duplicate detection technique to
improve the model inference of web applications, based on Tree Kernel (TK)
functions. TKs are a class of functions that compute similarity between
tree-structured objects, largely investigated and successfully applied in the
Natural Language Processing domain. To evaluate the capability of the proposed
approach in detecting near-duplicate web pages, we conducted preliminary
classification experiments on a freely-available massive dataset of about 100k
manually annotated web page pairs. We compared the classification performance
of the proposed approach with other state-of-the-art near-duplicate detection
techniques. Preliminary results show that our approach performs better than
state-of-the-art techniques in the near-duplicate detection classification
task. These promising results show that TKs can be applied to near-duplicate
detection in the context of web application model inference, and motivate
further research in this direction.
