  In many control problems that include vision, optimal controls can be
inferred from the location of the objects in the scene. This information can be
represented using feature points, which is a list of spatial locations in
learned feature maps of an input image. Previous works show that feature points
learned using unsupervised pre-training or human supervision can provide good
features for control tasks. In this paper, we show that it is possible to learn
efficient feature point representations end-to-end, without the need for
unsupervised pre-training, decoders, or additional losses. Our proposed
architecture consists of a differentiable feature point extractor that feeds
the coordinates of the estimated feature points directly to a soft actor-critic
agent. The proposed algorithm yields performance competitive to the
state-of-the art on DeepMind Control Suite tasks.
