  Choosing an appropriate representation of the environment for the underlying
decision-making process of the reinforcement learning agent is not always
straightforward. The state representation should be inclusive enough to allow
the agent to informatively decide on its actions and disentangled enough to
simplify policy training and the corresponding sim2real transfer. Given this
outlook, this work examines the effect of various representations in
incentivizing the agent to solve a specific robotic task: antipodal and planar
object grasping. A continuum of state representations is defined, starting from
hand-crafted numerical states to encoded image-based representations, with
decreasing levels of induced task-specific knowledge. The effects of each
representation on the ability of the agent to solve the task in simulation and
the transferability of the learned policy to the real robot are examined and
compared against a model-based approach with complete system knowledge. The
results show that reinforcement learning agents using numerical states can
perform on par with non-learning baselines. Furthermore, we find that agents
using image-based representations from pre-trained environment embedding
vectors perform better than end-to-end trained agents, and hypothesize that
separation of representation learning from reinforcement learning can benefit
sim2real transfer. Finally, we conclude that incentivizing the state
representation with task-specific knowledge facilitates faster convergence for
agent training and increases success rates in sim2real robot control.
