  Game theory provides essential analysis in many applications of strategic
interactions. However, the question of how to construct a game model and what
is its fidelity is seldom addressed. In this work, we consider learning in a
class of repeated zero-sum games with unknown, time-varying payoff matrix, and
noisy feedbacks, by making use of an ensemble of benchmark game models. These
models can be pre-trained and collected dynamically during sequential plays.
They serve as prior side information and imperfectly underpin the unknown true
game model. We propose OFULinMat, an episodic learning algorithm that
integrates the adaptive estimation of game models and the learning of the
strategies. The proposed algorithm is shown to achieve a sublinear bound on the
saddle-point regret. We show that this algorithm is provably efficient through
both theoretical analysis and numerical examples. We use a dynamic honeypot
allocation game as a case study to illustrate and corroborate our results. We
also discuss the relationship and highlight the difference between our
framework and the classical adversarial multi-armed bandit framework.
