  Liquids are an important part of many common manipulation tasks in human
environments. If we wish to have robots that can accomplish these types of
tasks, they must be able to interact with liquids in an intelligent manner. In
this paper, we investigate ways for robots to perceive and reason about
liquids. That is, a robot asks the questions What in the visual data stream is
liquid? and How can I use that to infer all the potential places where liquid
might be? We collected two datasets to evaluate these questions, one using a
realistic liquid simulator and another on our robot. We used fully
convolutional neural networks to learn to detect and track liquids across
pouring sequences. Our results show that these networks are able to perceive
and reason about liquids, and that integrating temporal information is
important to performing such tasks well.
