  We study the problem of offline decision making, which focuses on learning
decisions from datasets only partially correlated with the learning objective.
While previous research has extensively studied specific offline decision
making problems like offline reinforcement learning (RL) and off-policy
evaluation (OPE), a unified framework and theory remain absent. To address this
gap, we introduce a unified framework termed Decision Making with Offline
Feedback (DMOF), which captures a wide range of offline decision making
problems including offline RL, OPE, and offline partially observable Markov
decision processes (POMDPs). For the DMOF framework, we introduce a hardness
measure called the Offline Estimation Coefficient (OEC), which measures the
learnability of offline decision making problems and is also reflected in the
derived minimax lower bounds. Additionally, we introduce an algorithm called
Empirical Decision with Divergence (EDD), for which we establish both an
instance-dependent upper bound and a minimax upper bound. The minimax upper
bound almost matches the lower bound determined by the OEC. Finally, we show
that EDD achieves a fast convergence rate (i.e., a rate scaling as $1/N$, where
$N$ is the sample size) for specific settings such as supervised learning and
Markovian sequential problems~(e.g., MDPs) with partial coverage.
