  We consider negotiation settings in which two agents use natural language to
bargain on goods. Agents need to decide on both high-level strategy (e.g.,
proposing \$50) and the execution of that strategy (e.g., generating "The bike
is brand new. Selling for just \$50."). Recent work on negotiation trains
neural models, but their end-to-end nature makes it hard to control their
strategy, and reinforcement learning tends to lead to degenerate solutions. In
this paper, we propose a modular approach based on coarse di- alogue acts
(e.g., propose(price=50)) that decouples strategy and generation. We show that
we can flexibly set the strategy using supervised learning, reinforcement
learning, or domain-specific knowledge without degeneracy, while our
retrieval-based generation can maintain context-awareness and produce diverse
utterances. We test our approach on the recently proposed DEALORNODEAL game,
and we also collect a richer dataset based on real items on Craigslist. Human
evaluation shows that our systems achieve higher task success rate and more
human-like negotiation behavior than previous approaches.
