  Optimization of hyper-parameters in reinforcement learning (RL) algorithms is
a key task, because they determine how the agent will learn its policy by
interacting with its environment, and thus what data is gathered. In this work,
an approach that uses Bayesian optimization to perform a two-step optimization
is proposed: first, categorical RL structure hyper-parameters are taken as
binary variables and optimized with an acquisition function tailored for such
variables. Then, at a lower level of abstraction, solution-level
hyper-parameters are optimized by resorting to the expected improvement
acquisition function, while using the best categorical hyper-parameters found
in the optimization at the upper-level of abstraction. This two-tier approach
is validated in a simulated control task. Results obtained are promising and
open the way for more user-independent applications of reinforcement learning.
