  Estimating the forces acting between instruments and tissue is a challenging
problem for robot-assisted minimally-invasive surgery. Recently, numerous
vision-based methods have been proposed to replace electro-mechanical
approaches. Moreover, optical coherence tomography (OCT) and deep learning have
been used for estimating forces based on deformation observed in volumetric
image data. The method demonstrated the advantage of deep learning with 3D
volumetric data over 2D depth images for force estimation. In this work, we
extend the problem of deep learning-based force estimation to 4D
spatio-temporal data with streams of 3D OCT volumes. For this purpose, we
design and evaluate several methods extending spatio-temporal deep learning to
4D which is largely unexplored so far. Furthermore, we provide an in-depth
analysis of multi-dimensional image data representations for force estimation,
comparing our 4D approach to previous, lower-dimensional methods. Also, we
analyze the effect of temporal information and we study the prediction of
short-term future force values, which could facilitate safety features. For our
4D force estimation architectures, we find that efficient decoupling of spatial
and temporal processing is advantageous. We show that using 4D spatio-temporal
data outperforms all previously used data representations with a mean absolute
error of 10.7mN. We find that temporal information is valuable for force
estimation and we demonstrate the feasibility of force prediction.
