  We want artificial intelligence (AI) to be beneficial. This is the grounding
assumption of most of the attitudes towards AI research. We want AI to be
"good" for humanity. We want it to help, not hinder, humans. Yet what exactly
this entails in theory and in practice is not immediately apparent.
Theoretically, this declarative statement subtly implies a commitment to a
consequentialist ethics. Practically, some of the more promising machine
learning techniques to create a robust AI, and perhaps even an artificial
general intelligence (AGI) also commit one to a form of utilitarianism. In both
dimensions, the logic of the beneficial AI movement may not in fact create
"beneficial AI" in either narrow applications or in the form of AGI if the
ethical assumptions are not made explicit and clear.
  Additionally, as it is likely that reinforcement learning (RL) will be an
important technique for machine learning in this area, it is also important to
interrogate how RL smuggles in a particular type of consequentialist reasoning
into the AI: particularly, a brute form of hedonistic act utilitarianism. Since
the mathematical logic commits one to a maximization function, the result is
that an AI will inevitably be seeking more and more rewards. We have two
conclusions that arise from this. First, is that if one believes that a
beneficial AI is an ethical AI, then one is committed to a framework that
posits 'benefit' is tantamount to the greatest good for the greatest number.
Second, if the AI relies on RL, then the way it reasons about itself, the
environment, and other agents, will be through an act utilitarian morality.
This proposition may, or may not, in fact be actually beneficial for humanity.
