  Current end-to-end deep Reinforcement Learning (RL) approaches require
jointly learning perception, decision-making and low-level control from very
sparse reward signals and high-dimensional inputs, with little capability of
incorporating prior knowledge. This results in prohibitively long training
times for use on real-world robotic tasks. Existing algorithms capable of
extracting task-level representations from high-dimensional inputs, e.g. object
detection, often produce outputs of varying lengths, restricting their use in
RL methods due to the need for neural networks to have fixed length inputs. In
this work, we propose a framework that combines deep sets encoding, which
allows for variable-length abstract representations, with modular RL that
utilizes these representations, decoupling high-level decision making from
low-level control. We successfully demonstrate our approach on the robot
manipulation task of object sorting, showing that this method can learn
effective policies within mere minutes of highly simplified simulation. The
learned policies can be directly deployed on a robot without further training,
and generalize to variations of the task unseen during training.
