  Robot manipulation in a physically-constrained environment requires compliant
manipulation. Compliant manipulation is a manipulation skill to adjust hand
motion based on the force imposed by the environment. Recently, reinforcement
learning (RL) has been applied to solve household operations involving
compliant manipulation. However, previous RL methods have primarily focused on
designing a policy for a specific operation that limits their applicability and
requires separate training for every new operation. We propose a
constraint-aware policy that is applicable to various unseen manipulations by
grouping several manipulations together based on the type of physical
constraint involved. The type of physical constraint determines the
characteristic of the imposed force direction; thus, a generalized policy is
trained in the environment and reward designed on the basis of this
characteristic. This paper focuses on two types of physical constraints:
prismatic and revolute joints. Experiments demonstrated that the same policy
could successfully execute various compliant-manipulation operations, both in
the simulation and reality. We believe this study is the first step toward
realizing a generalized household-robot.
