  Customer services are critical to all companies, as they may directly connect
to the brand reputation. Due to a great number of customers, e-commerce
companies often employ multiple communication channels to answer customers'
questions, for example, chatbot and hotline. On one hand, each channel has
limited capacity to respond to customers' requests, on the other hand,
customers have different preferences over these channels. The current
production systems are mainly built based on business rules, which merely
considers tradeoffs between resources and customers' satisfaction. To achieve
the optimal tradeoff between resources and customers' satisfaction, we propose
a new framework based on deep reinforcement learning, which directly takes both
resources and user model into account. In addition to the framework, we also
propose a new deep-reinforcement-learning based routing method-double dueling
deep Q-learning with prioritized experience replay (PER-DoDDQN). We evaluate
our proposed framework and method using both synthetic and a real customer
service log data from a large financial technology company. We show that our
proposed deep-reinforcement-learning based framework is superior to the
existing production system. Moreover, we also show our proposed PER-DoDDQN is
better than all other deep Q-learning variants in practice, which provides a
more optimal routing plan. These observations suggest that our proposed method
can seek the trade-off where both channel resources and customers' satisfaction
are optimal.
