  Recent work on promoting cooperation in multi-agent learning has resulted in
many methods which successfully promote cooperation at the cost of becoming
more vulnerable to exploitation by malicious actors. We show that this is an
unavoidable trade-off and propose an objective which balances these concerns,
promoting both safety and long-term cooperation. Moreover, the trade-off
between safety and cooperation is not severe, and you can receive exponentially
large returns through cooperation from a small amount of risk. We study both an
exact solution method and propose a method for training policies that targets
this objective, Accumulating Risk Capital Through Investing in Cooperation
(ARCTIC), and evaluate them in iterated Prisoner's Dilemma and Stag Hunt.
