  Vestibular schwannomas (VS) are benign tumors that are generally managed by
active surveillance with MRI examination. To further assist clinical
decision-making and avoid overtreatment, an accurate prediction of tumor growth
based on longitudinal imaging is highly desirable. In this paper, we introduce
DeepGrowth, a deep learning method that incorporates neural fields and
recurrent neural networks for prospective tumor growth prediction. In the
proposed method, each tumor is represented as a signed distance function (SDF)
conditioned on a low-dimensional latent code. Unlike previous studies that
perform tumor shape prediction directly in the image space, we predict the
latent codes instead and then reconstruct future shapes from it. To deal with
irregular time intervals, we introduce a time-conditioned recurrent module
based on a ConvLSTM and a novel temporal encoding strategy, which enables the
proposed model to output varying tumor shapes over time. The experiments on an
in-house longitudinal VS dataset showed that the proposed model significantly
improved the performance ($\ge 1.6\%$ Dice score and $\ge0.20$ mm 95\%
Hausdorff distance), in particular for top 20\% tumors that grow or shrink the
most ($\ge 4.6\%$ Dice score and $\ge 0.73$ mm 95\% Hausdorff distance). Our
code is available at ~\burl{https://github.com/cyjdswx/DeepGrowth}
