  In this paper, we propose a novel approach for agent motion prediction in
cluttered environments. One of the main challenges in predicting agent motion
is accounting for location and context-specific information. Our main
contribution is the concept of learning context maps to improve the prediction
task. Context maps are a set of location-specific latent maps that are trained
alongside the predictor. Thus, the proposed maps are capable of capturing
location context beyond visual context cues (e.g. usual average speeds and
typical trajectories) or predefined map primitives (such as lanes and stop
lines). We pose context map learning as a multi-task training problem and
describe our map model and its incorporation into a state-of-the-art trajectory
predictor. In extensive experiments, it is shown that use of learned maps can
significantly improve predictor accuracy. Furthermore, the performance can be
additionally boosted by providing partial knowledge of map semantics.
