  In this paper, we study the stochastic combinatorial multi-armed bandit
(CMAB) framework that allows a general nonlinear reward function, whose
expected value may not depend only on the means of the input random variables
but possibly on the entire distributions of these variables. Our framework
enables a much larger class of reward functions such as the $\max()$ function
and nonlinear utility functions. Existing techniques relying on accurate
estimations of the means of random variables, such as the upper confidence
bound (UCB) technique, do not work directly on these functions. We propose a
new algorithm called stochastically dominant confidence bound (SDCB), which
estimates the distributions of underlying random variables and their
stochastically dominant confidence bounds. We prove that SDCB can achieve
$O(\log{T})$ distribution-dependent regret and $\tilde{O}(\sqrt{T})$
distribution-independent regret, where $T$ is the time horizon. We apply our
results to the $K$-MAX problem and expected utility maximization problems. In
particular, for $K$-MAX, we provide the first polynomial-time approximation
scheme (PTAS) for its offline problem, and give the first $\tilde{O}(\sqrt T)$
bound on the $(1-\epsilon)$-approximation regret of its online problem, for any
$\epsilon>0$.
