  Flaky tests are tests that yield different outcomes when run on the same
version of a program. This non-deterministic behaviour plagues continuous
integration with false signals, wasting developers' time and reducing their
trust in test suites. Studies highlighted the importance of keeping tests
flakiness-free. Recently, the research community has been pushing forward the
detection of flaky tests by suggesting many static and dynamic approaches.
While promising, those approaches mainly focus on classifying tests as flaky or
not and, even when high performances are reported, it remains challenging to
understand the cause of flakiness. This part is crucial for researchers and
developers that aim to fix it. To help with the comprehension of a given flaky
test, we propose FlakyCat, the first approach for classifying flaky tests based
on their root cause category. FlakyCat relies on CodeBERT for code
representation and leverages a Siamese network-based Few-Shot learning method
to train a multi-class classifier with few data. We train and evaluate FlakyCat
on a set of 343 flaky tests collected from open-source Java projects. Our
evaluation shows that FlakyCat categorises flaky tests accurately, with a
weighted F1 score of 70%. Furthermore, we investigate the performance of our
approach for each category, revealing that Async waits, Unordered collections
and Time-related flaky tests are accurately classified, while
Concurrency-related flaky tests are more challenging to predict. Finally, to
facilitate the comprehension of FlakyCat's predictions, we present a new
technique for CodeBERT-based model interpretability that highlights code
statements influencing the categorization.
