  Reinforcement learning can greatly benefit from the use of options as a way
of encoding recurring behaviours and to foster exploration. An important open
problem is how can an agent autonomously learn useful options when solving
particular distributions of related tasks. We investigate some of the
conditions that influence optimality of options, in settings where agents have
a limited time budget for learning each task and the task distribution might
involve problems with different levels of similarity. We directly search for
optimal option sets and show that the discovered options significantly differ
depending on factors such as the available learning time budget and that the
found options outperform popular option-generation heuristics.
