  In iterated games, a player can unilaterally exert influence over the outcome
through a careful choice of strategy. A powerful class of such "payoff control"
strategies was discovered by Press and Dyson (2012). Their so-called
"zero-determinant" (ZD) strategies allow a player to unilaterally enforce a
linear relationship between both players' payoffs. It was subsequently shown by
Chen and Zinger (2014) that when the slope of this linear relationship is
positive, ZD strategies are robustly effective against a selfishly optimizing
co-player, in that all adapting paths of the selfish player lead to the maximal
payoffs for both players (at least when there are certain restrictions on the
game parameters). In this paper, we investigate the efficacy of selfish
learning against a fixed player in more general settings, for both ZD and
non-ZD strategies. We first prove that in any symmetric $2\times 2$ game, the
selfish player's final strategy must be of a certain form and cannot be fully
stochastic. We then show that there are prisoner's dilemma interactions for
which selfish optimization does not always lead to maximal payoffs against
fixed ZD strategies with positive slope. We give examples of selfish adapting
paths that lead to locally but not globally optimal payoffs, undermining the
robustness of payoff control strategies. For non-ZD strategies, these
pathologies arise regardless of the original restrictions on the game
parameters. Our results illuminate the difficulty of implementing robust payoff
control and selfish optimization, even in the simplest context of playing
against a fixed strategy.
