  A new modification of Isolation Forest called Attention-Based Isolation
Forest (ABIForest) for solving the anomaly detection problem is proposed. It
incorporates the attention mechanism in the form of the Nadaraya-Watson
regression into the Isolation Forest for improving solution of the anomaly
detection problem. The main idea underlying the modification is to assign
attention weights to each path of trees with learnable parameters depending on
instances and trees themselves. The Huber's contamination model is proposed to
be used for defining the attention weights and their parameters. As a result,
the attention weights are linearly depend on the learnable attention parameters
which are trained by solving the standard linear or quadratic optimization
problem. ABIForest can be viewed as the first modification of Isolation Forest,
which incorporates the attention mechanism in a simple way without applying
gradient-based algorithms. Numerical experiments with synthetic and real
datasets illustrate outperforming results of ABIForest. The code of proposed
algorithms is available.
