  Despite its notable success in adversarial learning approaches to
multi-domain task-oriented dialog system, training the dialog policy via
adversarial inverse reinforcement learning often fails to balance the
performance of the policy generator and reward estimator. During optimization,
the reward estimator often overwhelms the policy generator and produces
excessively uninformative gradients. We proposes the Variational Reward
estimator Bottleneck (VRB), which is an effective regularization method that
aims to constrain unproductive information flows between inputs and the reward
estimator. The VRB focuses on capturing discriminative features, by exploiting
information bottleneck on mutual information. Empirical results on a
multi-domain task-oriented dialog dataset demonstrate that the VRB
significantly outperforms previous methods.
