  Finding optimal policies which maximize long term rewards of Markov Decision
Processes requires the use of dynamic programming and backward induction to
solve the Bellman optimality equation. However, many real-world problems
require optimization of an objective that is non-linear in cumulative rewards
for which dynamic programming cannot be applied directly. For example, in a
resource allocation problem, one of the objectives is to maximize long-term
fairness among the users. We notice that when an agent aim to optimize some
function of the sum of rewards is considered, the problem loses its Markov
nature. This paper addresses and formalizes the problem of optimizing a
non-linear function of the long term average of rewards. We propose model-based
and model-free algorithms to learn the policy, where the model-based policy is
shown to achieve a regret of $\Tilde{O}\left(LKDS\sqrt{\frac{A}{T}}\right)$ for
$K$ objectives combined with a concave $L$-Lipschitz function. Further, using
the fairness in cellular base-station scheduling, and queueing system
scheduling as examples, the proposed algorithm is shown to significantly
outperform the conventional RL approaches.
