  Blackjack or "21" is a popular card-based game of chance and skill. The
objective of the game is to win by obtaining a hand total higher than the
dealer's without exceeding 21. The ideal blackjack strategy will maximize
financial return in the long run while avoiding gambler's ruin. The stochastic
environment and inherent reward structure of blackjack presents an appealing
problem to better understand reinforcement learning agents in the presence of
environment variations. Here we consider a q-learning solution for optimal play
and investigate the rate of learning convergence of the algorithm as a function
of deck size. A blackjack simulator allowing for universal blackjack rules is
also implemented to demonstrate the extent to which a card counter perfectly
using the basic strategy and hi-lo system can bring the house to bankruptcy and
how environment variations impact this outcome. The novelty of our work is to
place this conceptual understanding of the impact of deck size in the context
of learning agent convergence.
