  Some of the most relevant future applications of multi-agent systems like
autonomous driving or factories as a service display mixed-motive scenarios,
where agents might have conflicting goals. In these settings agents are likely
to learn undesirable outcomes in terms of cooperation under independent
learning, such as overly greedy behavior. Motivated from real world societies,
in this work we propose to utilize market forces to provide incentives for
agents to become cooperative. As demonstrated in an iterated version of the
Prisoner's Dilemma, the proposed market formulation can change the dynamics of
the game to consistently learn cooperative policies. Further we evaluate our
approach in spatially and temporally extended settings for varying numbers of
agents. We empirically find that the presence of markets can improve both the
overall result and agent individual returns via their trading activities.
