  Scene understanding is paramount in robotics, self-navigation, augmented
reality, and many other fields. To fully accomplish this task, an autonomous
agent has to infer the 3D structure of the sensed scene (to know where it looks
at) and its content (to know what it sees). To tackle the two tasks, deep
neural networks trained to infer semantic segmentation and depth from stereo
images are often the preferred choices. Specifically, Semantic Stereo Matching
can be tackled by either standalone models trained for the two tasks
independently or joint end-to-end architectures. Nonetheless, as proposed so
far, both solutions are inefficient because requiring two forward passes in the
former case or due to the complexity of a single network in the latter,
although jointly tackling both tasks is usually beneficial in terms of
accuracy. In this paper, we propose a single compact and lightweight
architecture for real-time semantic stereo matching. Our framework relies on
coarse-to-fine estimations in a multi-stage fashion, allowing: i) very fast
inference even on embedded devices, with marginal drops in accuracy, compared
to state-of-the-art networks, ii) trade accuracy for speed, according to the
specific application requirements. Experimental results on high-end GPUs as
well as on an embedded Jetson TX2 confirm the superiority of semantic stereo
matching compared to standalone tasks and highlight the versatility of our
framework on any hardware and for any application.
