  We consider the problem of controlling an unknown stochastic linear system
with quadratic costs - called the adaptive LQ control problem. We re-examine an
approach called ''Reward Biased Maximum Likelihood Estimate'' (RBMLE) that was
proposed more than forty years ago, and which predates the ''Upper Confidence
Bound'' (UCB) method as well as the definition of ''regret'' for bandit
problems. It simply added a term favoring parameters with larger rewards to the
criterion for parameter estimation. We show how the RBMLE and UCB methods can
be reconciled, and thereby propose an Augmented RBMLE-UCB algorithm that
combines the penalty of the RBMLE method with the constraints of the UCB
method, uniting the two approaches to optimism in the face of uncertainty. We
establish that theoretically, this method retains
$\Tilde{\mathcal{O}}(\sqrt{T})$ regret, the best-known so far. We further
compare the empirical performance of the proposed Augmented RBMLE-UCB and the
standard RBMLE (without the augmentation) with UCB, Thompson Sampling, Input
Perturbation, Randomized Certainty Equivalence and StabL on many real-world
examples including flight control of Boeing 747 and Unmanned Aerial Vehicle. We
perform extensive simulation studies showing that the Augmented RBMLE
consistently outperforms UCB, Thompson Sampling and StabL by a huge margin,
while it is marginally better than Input Perturbation and moderately better
than Randomized Certainty Equivalence.
