  This paper studies continuous-time reinforcement learning for controlled
jump-diffusion models by featuring the q-function (the continuous-time
counterpart of Q-function) and the q-learning algorithms under the Tsallis
entropy regularization. Contrary to the conventional Shannon entropy, the
general form of Tsallis entropy renders the optimal policy not necessary a
Gibbs measure, where some Lagrange multiplier and KKT multiplier naturally
arise from certain constraints to ensure the learnt policy to be a probability
distribution. As a consequence,the relationship between the optimal policy and
the q-function also involves the Lagrange multiplier. In response, we establish
the martingale characterization of the q-function under Tsallis entropy and
devise two q-learning algorithms depending on whether the Lagrange multiplier
can be derived explicitly or not. In the latter case, we need to consider
different parameterizations of the q-function and the policy and update them
alternatively. Finally, we examine two financial applications, namely an
optimal portfolio liquidation problem and a non-LQ control problem. It is
interesting to see therein that the optimal policies under the Tsallis entropy
regularization can be characterized explicitly, which are distributions
concentrate on some compact support. The satisfactory performance of our
q-learning algorithm is illustrated in both examples.
