  This work investigates the offline formulation of the contextual bandit
problem, where the goal is to leverage past interactions collected under a
behavior policy to evaluate, select, and learn new, potentially
better-performing, policies. Motivated by critical applications, we move beyond
point estimators. Instead, we adopt the principle of pessimism where we
construct upper bounds that assess a policy's worst-case performance, enabling
us to confidently select and learn improved policies. Precisely, we introduce
novel, fully empirical concentration bounds for a broad class of importance
weighting risk estimators. These bounds are general enough to cover most
existing estimators and pave the way for the development of new ones. In
particular, our pursuit of the tightest bound within this class motivates a
novel estimator (LS), that logarithmically smooths large importance weights.
The bound for LS is provably tighter than all its competitors, and naturally
results in improved policy selection and learning strategies. Extensive policy
evaluation, selection, and learning experiments highlight the versatility and
favorable performance of LS.
