  We propose an optimistic model-based algorithm, dubbed SMRL, for
finite-horizon episodic reinforcement learning (RL) when the transition model
is specified by exponential family distributions with $d$ parameters and the
reward is bounded and known. SMRL uses score matching, an unnormalized density
estimation technique that enables efficient estimation of the model parameter
by ridge regression. Under standard regularity assumptions, SMRL achieves
$\tilde O(d\sqrt{H^3T})$ online regret, where $H$ is the length of each episode
and $T$ is the total number of interactions (ignoring polynomial dependence on
structural scale parameters).
