  Modern Reinforcement Learning (RL) is more than just learning the optimal
policy; Alternative learning goals such as exploring the environment,
estimating the underlying model, and learning from preference feedback are all
of practical importance. While provably sample-efficient algorithms for each
specific goal have been proposed, these algorithms often depend strongly on the
particular learning goal and thus admit different structures correspondingly.
It is an urging open question whether these learning goals can rather be
tackled by a single unified algorithm.
  We make progress on this question by developing a unified algorithm framework
for a large class of learning goals, building on the Decision-Estimation
Coefficient (DEC) framework. Our framework handles many learning goals such as
no-regret RL, PAC RL, reward-free learning, model estimation, and
preference-based learning, all by simply instantiating the same generic
complexity measure called "Generalized DEC", and a corresponding generic
algorithm. The generalized DEC also yields a sample complexity lower bound for
each specific learning goal. As applications, we propose "decouplable
representation" as a natural sufficient condition for bounding generalized
DECs, and use it to obtain many new sample-efficient results (and recover
existing results) for a wide range of learning goals and problem classes as
direct corollaries. Finally, as a connection, we re-analyze two existing
optimistic model-based algorithms based on Posterior Sampling and Maximum
Likelihood Estimation, showing that they enjoy sample complexity bounds under
similar structural conditions as the DEC.
