  This note lays part of the theoretical ground for a definition of
differential systems modeling reinforcement learning in continuous time
non-Markovian rough environments. Specifically we focus on optimal relaxed
control of rough equations (the term relaxed referring to the fact that
controls have to be considered as measure valued objects). With reinforcement
learning in view, our reward functions encompass forms that involve an
entropy-type term favoring exploration. In this context, our contribution
focuses on a careful definition of the corresponding relaxed
Hamilton-Jacobi-Bellman (HJB)-type equation. A substantial part of our endeavor
consists in a precise definition of the notion of test function and viscosity
solution for the rough relaxed PDE obtained in this framework. Note that this
task is often merely sketched in the rough viscosity literature, in spite of
the fact that it gives a proper meaning to the differential system at stake. In
the last part of the paper we prove that the natural value function in our
context solves a relaxed rough HJB equation in the viscosity sense.
