  An artificial general intelligence (AGI) might have an instrumental drive to
modify its utility function to improve its ability to cooperate, bargain,
promise, threaten, and resist and engage in blackmail. Such an AGI would
necessarily have a utility function that was at least partially observable and
that was influenced by how other agents chose to interact with it. This
instrumental drive would conflict with the orthogonality thesis since the
modifications would be influenced by the AGI's intelligence. AGIs in highly
competitive environments might converge to having nearly the same utility
function, one optimized to favorably influencing other agents through game
theory.
