  Vision, as an inexpensive yet information rich sensor, is commonly used for
perception on autonomous mobile robots. Unfortunately, accurate vision-based
perception requires a number of assumptions about the environment to hold --
some examples of such assumptions, depending on the perception algorithm at
hand, include purely lambertian surfaces, texture-rich scenes, absence of
aliasing features, and refractive surfaces. In this paper, we present an
approach for introspective vision for obstacle avoidance (IVOA) -- by
leveraging a supervisory sensor that is occasionally available, we detect
failures of stereo vision-based perception from divergence in plans generated
by vision and the supervisory sensor. By projecting the 3D coordinates where
the plans agree and disagree onto the images used for vision-based perception,
IVOA generates a training set of reliable and unreliable image patches for
perception. We then use this training dataset to learn a model of which image
patches are likely to cause failures of the vision-based perception algorithm.
Using this model, IVOA is then able to predict whether the relevant image
patches in the observed images are likely to cause failures due to vision (both
false positives and false negatives). We empirically demonstrate with extensive
real-world data from both indoor and outdoor environments, the ability of IVOA
to accurately predict the failures of two distinct vision algorithms.
