  We study how to exploit the notion of strategy templates to learn strategies
for multi-issue bilateral negotiation. Each strategy template consists of a set
of interpretable parameterized tactics that are used to decide an optimal
action at any time. We use deep reinforcement learning throughout an
actor-critic architecture to estimate the tactic parameter values for a
threshold utility, when to accept an offer and how to generate a new bid. This
contrasts with existing work that only estimates the threshold utility for
those tactics. We pre-train the strategy by supervision from the dataset
collected using "teacher strategies", thereby decreasing the exploration time
required for learning during negotiation. As a result, we build automated
agents for multi-issue negotiations that can adapt to different negotiation
domains without the need to be pre-programmed. We empirically show that our
work outperforms the state-of-the-art in terms of the individual as well as
social efficiency.
