  Precisely tracking uncertainties is crucial for robots to successfully and
safely operate in unstructured and dynamic environments. We present a
probabilistic framework to precisely keep track of uncertainties throughout the
entire manipulation process. In agreement with common manipulation pipelines,
we decompose the process into two subsequent stages, namely perception and
physical interaction. Each stage is associated with different sources and types
of uncertainties, requiring different techniques. We discuss which
representation of uncertainties is the most appropriate for each stage (e.g. as
probability distributions in SE(3) during perception, as weighted particles
during physical interactions), how to convert from one representation to
another, and how to initialize or update the uncertainties at each step of the
process (camera calibration, image processing, pushing, grasping, etc.).
Finally, we demonstrate the benefit of this fine-grained knowledge of
uncertainties in an actual assembly task.
