  With the upcoming large-scale surveys like LSST, we expect to find
approximately $10^5$ strong gravitational lenses among data of many orders of
magnitude larger. In this scenario, the usage of non-automated techniques is
too time-consuming and hence impractical for science. For this reason, machine
learning techniques started becoming an alternative to previous methods. In our
previous work, we proposed a new machine learning architecture based on the
principle of self-attention, trained to find strong gravitational lenses on
simulated data from the Bologna Lens Challenge. Self-attention-based models
have clear advantages compared to simpler CNNs and highly competing performance
in comparison to the current state-of-art CNN models. We apply the proposed
model to the Kilo Degree Survey, identifying some new strong lens candidates.
However, these have been identified among a plethora of false positives, which
made the application of this model not so advantageous. Therefore, throughout
this paper, we investigate the pitfalls of this approach, and possible
solutions, such as transfer learning, are proposed.
