  Deep neural networks tend to underestimate uncertainty and produce overly
confident predictions. Recently proposed solutions, such as MC Dropout and
SDENet, require complex training and/or auxiliary out-of-distribution data. We
propose a simple solution by extending the time-tested iterative reweighted
least square (IRLS) in generalised linear regression. We use two sub-networks
to parametrise the prediction and uncertainty estimation, enabling easy
handling of complex inputs and nonlinear response. The two sub-networks have
shared representations and are trained via two complementary loss functions for
the prediction and the uncertainty estimates, with interleaving steps as in a
cooperative game. Compared with more complex models such as MC-Dropout or
SDE-Net, our proposed network is simpler to implement and more robust
(insensitive to varying aleatoric and epistemic uncertainty).
