  Due to the limitations of the robotic sensors, during a robotic manipulation
task, the acquisition of the object's state can be unreliable and noisy.
Combining an accurate model of multi-body dynamic system with Bayesian
filtering methods has been shown to be able to filter out noise from the
object's observed states. However, efficiency of these filtering methods
suffers from samples that violate the physical constraints, e.g., no
penetration constraint.
  In this paper, we propose a Rao-Blackwellized Particle Filter (RBPF) that
samples the contact states and updates the object's poses using Kalman filters.
This RBPF also enforces the physical constraints on the samples by solving a
quadratic programming problem. By comparing our method with methods that does
not consider physical constraints, we show that our proposed RBPF is not only
able to estimate the object's states, e.g., poses, more accurately but also
able to infer unobserved states, e.g., velocities, with higher precision.
