  This paper presents a novel attention-based algorithm for achieving adaptive
computation called DACT, which, unlike existing ones, is end-to-end
differentiable. Our method can be used in conjunction with many networks; in
particular, we study its application to the widely known MAC architecture,
obtaining a significant reduction in the number of recurrent steps needed to
achieve similar accuracies, therefore improving its performance to computation
ratio. Furthermore, we show that by increasing the maximum number of steps
used, we surpass the accuracy of even our best non-adaptive MAC in the CLEVR
dataset, demonstrating that our approach is able to control the number of steps
without significant loss of performance. Additional advantages provided by our
approach include considerably improving interpretability by discarding useless
steps and providing more insights into the underlying reasoning process.
Finally, we present adaptive computation as an equivalent to an ensemble of
models, similar to a mixture of expert formulation. Both the code and the
configuration files for our experiments are made available to support further
research in this area.
