  The safe deployment of autonomous vehicles relies on their ability to
effectively react to environmental changes. This can require maneuvering on
varying surfaces which is still a difficult problem, especially for slippery
terrains. To address this issue we propose a new approach that learns a
surface-aware dynamics model by conditioning it on a latent variable vector
storing surface information about the current location. A latent mapper is
trained to update these latent variables during inference from multiple
modalities on every traversal of the corresponding locations and stores them in
a map. By training everything end-to-end with the loss of the dynamics model,
we enforce the latent mapper to learn an update rule for the latent map that is
useful for the subsequent dynamics model. We implement and evaluate our
approach on a real miniature electric car. The results show that the latent map
is updated to allow more accurate predictions of the dynamics model compared to
a model without this information. We further show that by using this model, the
driving performance can be improved on varying and challenging surfaces.
