  The ability to interact and understand the environment is a fundamental
prerequisite for a wide range of applications from robotics to augmented
reality. In particular, predicting how deformable objects will react to applied
forces in real time is a significant challenge. This is further confounded by
the fact that shape information about encountered objects in the real world is
often impaired by occlusions, noise and missing regions e.g. a robot
manipulating an object will only be able to observe a partial view of the
entire solid. In this work we present a framework, 3D-PhysNet, which is able to
predict how a three-dimensional solid will deform under an applied force using
intuitive physics modelling. In particular, we propose a new method to encode
the physical properties of the material and the applied force, enabling
generalisation over materials. The key is to combine deep variational
autoencoders with adversarial training, conditioned on the applied force and
the material properties. We further propose a cascaded architecture that takes
a single 2.5D depth view of the object and predicts its deformation. Training
data is provided by a physics simulator. The network is fast enough to be used
in real-time applications from partial views. Experimental results show the
viability and the generalisation properties of the proposed architecture.
