  A Reinforcement Learning (RL) system depends on a set of initial conditions
(hyperparameters) that affect the system's performance. However, defining a
good choice of hyperparameters is a challenging problem.
  Hyperparameter tuning often requires manual or automated searches to find
optimal values. Nonetheless, a noticeable limitation is the high cost of
algorithm evaluation for complex models, making the tuning process
computationally expensive and time-consuming.
  In this paper, we propose a framework based on integrating complex event
processing and temporal models, to alleviate these trade-offs. Through this
combination, it is possible to gain insights about a running RL system
efficiently and unobtrusively based on data stream monitoring and to create
abstract representations that allow reasoning about the historical behaviour of
the RL system. The obtained knowledge is exploited to provide feedback to the
RL system for optimising its hyperparameters while making effective use of
parallel resources.
  We introduce a novel history-aware epsilon-greedy logic for hyperparameter
optimisation that instead of using static hyperparameters that are kept fixed
for the whole training, adjusts the hyperparameters at runtime based on the
analysis of the agent's performance over time windows in a single agent's
lifetime. We tested the proposed approach in a 5G mobile communications case
study that uses DQN, a variant of RL, for its decision-making. Our experiments
demonstrated the effects of hyperparameter tuning using history on training
stability and reward values. The encouraging results show that the proposed
history-aware framework significantly improved performance compared to
traditional hyperparameter tuning approaches.
