  We present a novel negotiation model that allows an agent to learn how to
negotiate during concurrent bilateral negotiations in unknown and dynamic
e-markets. The agent uses an actor-critic architecture with model-free
reinforcement learning to learn a strategy expressed as a deep neural network.
We pre-train the strategy by supervision from synthetic market data, thereby
decreasing the exploration time required for learning during negotiation. As a
result, we can build automated agents for concurrent negotiations that can
adapt to different e-market settings without the need to be pre-programmed. Our
experimental evaluation shows that our deep reinforcement learning-based agents
outperform two existing well-known negotiation strategies in one-to-many
concurrent bilateral negotiations for a range of e-market settings.
