  We present a novel bilateral negotiation model that allows a self-interested
agent to learn how to negotiate over multiple issues in the presence of user
preference uncertainty. The model relies upon interpretable strategy templates
representing the tactics the agent should employ during the negotiation and
learns template parameters to maximize the average utility received over
multiple negotiations, thus resulting in optimal bid acceptance and generation.
Our model also uses deep reinforcement learning to evaluate threshold utility
values, for those tactics that require them, thereby deriving optimal utilities
for every environment state. To handle user preference uncertainty, the model
relies on a stochastic search to find user model that best agrees with a given
partial preference profile. Multi-objective optimization and multi-criteria
decision-making methods are applied at negotiation time to generate
Pareto-optimal outcomes thereby increasing the number of successful (win-win)
negotiations. Rigorous experimental evaluations show that the agent employing
our model outperforms the winning agents of the 10th Automated Negotiating
Agents Competition (ANAC'19) in terms of individual as well as social-welfare
utilities.
