  We consider online reinforcement learning in episodic Markov decision process
(MDP) with unknown transition function and stochastic rewards drawn from some
fixed but unknown distribution. The learner aims to learn the optimal policy
and minimize their regret over a finite time horizon through interacting with
the environment. We devise a simple and efficient model-based algorithm that
achieves $\widetilde{O}(LX\sqrt{TA})$ regret with high probability, where $L$
is the episode length, $T$ is the number of episodes, and $X$ and $A$ are the
cardinalities of the state space and the action space, respectively. The
proposed algorithm, which is based on the concept of ``optimism in the face of
uncertainty", maintains confidence sets of transition and reward functions and
uses occupancy measures to connect the online MDP with linear programming. It
achieves a tighter regret bound compared to the existing works that use a
similar confidence set framework and improves computational effort compared to
those that use a different framework but with a slightly tighter regret bound.
