  This paper examines a novel type of multi-agent problem, in which an agent
makes multiple identical copies of itself in order to achieve a single agent
task better or more efficiently. This strategy improves performance if the
environment is noisy and the task is sometimes unachievable by a single agent
copy. We propose a learning algorithm for this multicopy problem which takes
advantage of the structure of the value function to efficiently learn how to
balance the advantages and costs of adding additional copies.
