  Recently proposed reward-conditioned policies (RCPs) offer an appealing
alternative in reinforcement learning. Compared with policy gradient methods,
policy learning in RCPs is simpler since it is based on supervised learning,
and unlike value-based methods, it does not require optimization in the action
space to take actions. However, for multi-armed bandit (MAB) problems, we find
that RCPs are slower to converge and have inferior expected rewards at
convergence, compared with classic methods such as the upper confidence bound
and Thompson sampling. In this work, we show that the performance of RCPs can
be enhanced by constructing policies through the marginalization of rewards
using normalized weight functions, whose sum or integral equal $1$, although
the function values may be negative. We refer to this technique as generalized
marginalization, whose advantage is that negative weights for policies
conditioned on low rewards can make the resulting policies more distinct from
them. Strategies to perform generalized marginalization in MAB with discrete
action spaces are studied. Through simulations, we demonstrate that the
proposed technique improves RCPs and makes them competitive with classic
methods, showing superior performance on challenging MABs with large action
spaces and sparse reward signals.
