  We propose a comprehensive framework for policy gradient methods tailored to
continuous time reinforcement learning. This is based on the connection between
stochastic control problems and randomised problems, enabling applications
across various classes of Markovian continuous time control problems, beyond
diffusion models, including e.g. regular, impulse and optimal
stopping/switching problems. By utilizing change of measure in the control
randomisation technique, we derive a new policy gradient representation for
these randomised problems, featuring parametrised intensity policies. We
further develop actor-critic algorithms specifically designed to address
general Markovian stochastic control issues. Our framework is demonstrated
through its application to optimal switching problems, with two numerical case
studies in the energy sector focusing on real options.
