  In online reinforcement learning (RL), instead of employing standard
structural assumptions on Markov decision processes (MDPs), using a certain
coverage condition (original from offline RL) is enough to ensure
sample-efficient guarantees (Xie et al. 2023). In this work, we focus on this
new direction by digging more possible and general coverage conditions, and
study the potential and the utility of them in efficient online RL. We identify
more concepts, including the $L^p$ variant of concentrability, the density
ratio realizability, and trade-off on the partial/rest coverage condition, that
can be also beneficial to sample-efficient online RL, achieving improved regret
bound. Furthermore, if exploratory offline data are used, under our coverage
conditions, both statistically and computationally efficient guarantees can be
achieved for online RL. Besides, even though the MDP structure is given, e.g.,
linear MDP, we elucidate that, good coverage conditions are still beneficial to
obtain faster regret bound beyond $\widetilde{O}(\sqrt{T})$ and even a
logarithmic order regret. These results provide a good justification for the
usage of general coverage conditions in efficient online RL.
