  Decision-making under uncertainty is a fundamental problem encountered
frequently and can be formulated as a stochastic multi-armed bandit problem. In
the problem, the learner interacts with an environment by choosing an action at
each round, where a round is an instance of an interaction. In response, the
environment reveals a reward, which is sampled from a stochastic process, to
the learner. The goal of the learner is to maximize cumulative reward. In this
work, we assume that the rewards are the inner product of an action vector and
a state vector generated by a linear Gaussian dynamical system. To predict the
reward for each action, we propose a method that takes a linear combination of
previously observed rewards for predicting each action's next reward. We show
that, regardless of the sequence of previous actions chosen, the reward sampled
for any previously chosen action can be used for predicting another action's
future reward, i.e. the reward sampled for action 1 at round $t-1$ can be used
for predicting the reward for action $2$ at round $t$. This is accomplished by
designing a modified Kalman filter with a matrix representation that can be
learned for reward prediction. Numerical evaluations are carried out on a set
of linear Gaussian dynamical systems and are compared with 2 other well-known
stochastic multi-armed bandit algorithms.
