  This paper considers the problem of finding a solution to the finite horizon
constrained Markov decision processes (CMDP) where the objective as well as
constraints are sum of additive and multiplicative utilities. Towards solving
this, we construct another CMDP, with only additive utilities under a
restricted set of policies, whose optimal value is equal to that of the
original CMDP. Furthermore, we provide a finite dimensional bilinear program
(BLP) whose value equals the CMDP value and whose solution provides the optimal
policy. We also suggest an algorithm to solve this BLP.
