  In this paper, we rigorously study the problem of cost optimisation of hybrid
(mixed) institutional incentives, which are a plan of actions involving the use
of reward and punishment by an external decision-maker, for maximising the
level (or guaranteeing at least a certain level) of cooperative behaviour in a
well-mixed, finite population of self-regarding individuals who interact via
cooperation dilemmas (Donation Game or Public Goods Game). We show that a mixed
incentive scheme can offer a more cost-efficient approach for providing
incentives while ensuring the same level or standard of cooperation in the
long-run. We establish the asymptotic behaviour (namely neutral drift, strong
selection, and infinite-population limits). We prove the existence of a phase
transition, obtaining the critical threshold of the strength of selection at
which the monotonicity of the cost function changes and providing an algorithm
for finding the optimal value of the individual incentive cost. Our analytical
results are illustrated with numerical investigations. Overall, our analysis
provides novel theoretical insights into the design of cost-efficient
institutional incentive mechanisms for promoting the evolution of cooperation
in stochastic systems.
