  Combining learned policies in a prioritized, ordered manner is desirable
because it allows for modular design and facilitates data reuse through
knowledge transfer. In control theory, prioritized composition is realized by
null-space control, where low-priority control actions are projected into the
null-space of high-priority control actions. Such a method is currently
unavailable for Reinforcement Learning. We propose a novel, task-prioritized
composition framework for Reinforcement Learning, which involves a novel
concept: The indifferent-space of Reinforcement Learning policies. Our
framework has the potential to facilitate knowledge transfer and modular design
while greatly increasing data efficiency and data reuse for Reinforcement
Learning agents. Further, our approach can ensure high-priority constraint
satisfaction, which makes it promising for learning in safety-critical domains
like robotics. Unlike null-space control, our approach allows learning globally
optimal policies for the compound task by online learning in the
indifference-space of higher-level policies after initial compound policy
construction.
