  The neural linear model is a simple adaptive Bayesian linear regression
method that has recently been used in a number of problems ranging from
Bayesian optimization to reinforcement learning. Despite its apparent successes
in these settings, to the best of our knowledge there has been no systematic
exploration of its capabilities on simple regression tasks. In this work we
characterize these on the UCI datasets, a popular benchmark for Bayesian
regression models, as well as on the recently introduced UCI "gap" datasets,
which are better tests of out-of-distribution uncertainty. We demonstrate that
the neural linear model is a simple method that shows generally good
performance on these tasks, but at the cost of requiring good hyperparameter
tuning.
