  This work tackles scene understanding for outdoor robotic navigation, solely
relying on images captured by an on-board camera. Conventional visual scene
understanding interprets the environment based on specific descriptive
categories. However, such a representation is not directly interpretable for
decision-making and constrains robot operation to a specific domain. Thus, we
propose to segment egocentric images directly in terms of how a robot can
navigate in them, and tailor the learning problem to an autonomous navigation
task. Building around an image segmentation network, we present a generic
affordance consisting of 3 driveability levels which can broadly apply to both
urban and off-road scenes. By encoding these levels with soft ordinal labels,
we incorporate inter-class distances during learning which improves
segmentation compared to standard "hard" one-hot labelling. In addition, we
propose a navigation-oriented pixel-wise loss weighting method which assigns
higher importance to safety-critical areas. We evaluate our approach on
large-scale public image segmentation datasets ranging from sunny city streets
to snowy forest trails. In a cross-dataset generalization experiment, we show
that our affordance learning scheme can be applied across a diverse mix of
datasets and improves driveability estimation in unseen environments compared
to general-purpose, single-dataset segmentation.
