  Deep neural networks have become a mainstream approach to interactive
segmentation. As we show in our experiments, while for some images a trained
network provides accurate segmentation result with just a few clicks, for some
unknown objects it cannot achieve satisfactory result even with a large amount
of user input. Recently proposed backpropagating refinement (BRS) scheme
introduces an optimization problem for interactive segmentation that results in
significantly better performance for the hard cases. At the same time, BRS
requires running forward and backward pass through a deep network several times
that leads to significantly increased computational budget per click compared
to other methods. We propose f-BRS (feature backpropagating refinement scheme)
that solves an optimization problem with respect to auxiliary variables instead
of the network inputs, and requires running forward and backward pass just for
a small part of a network. Experiments on GrabCut, Berkeley, DAVIS and SBD
datasets set new state-of-the-art at an order of magnitude lower time per click
compared to original BRS. The code and trained models are available at
https://github.com/saic-vul/fbrs_interactive_segmentation .
