  Reinforcement Learning (RL) is a research area that has blossomed
tremendously in recent years and has shown remarkable potential for artificial
intelligence based opponents in computer games. This success is primarily due
to vast capabilities of Convolutional Neural Networks (ConvNet), enabling
algorithms to extract useful information from noisy environments. Capsule
Network (CapsNet) is a recent introduction to the Deep Learning algorithm group
and has only barely begun to be explored. The network is an architecture for
image classification, with superior performance for classification of the MNIST
dataset. CapsNets have not been explored beyond image classification.
  This thesis introduces the use of CapsNet for Q-Learning based game
algorithms. To successfully apply CapsNet in advanced game play, three main
contributions follow. First, the introduction of four new game environments as
frameworks for RL research with increasing complexity, namely Flash RL, Deep
Line Wars, Deep RTS, and Deep Maze. These environments fill the gap between
relatively simple and more complex game environments available for RL research
and are in the thesis used to test and explore the CapsNet behavior.
  Second, the thesis introduces a generative modeling approach to produce
artificial training data for use in Deep Learning models including CapsNets. We
empirically show that conditional generative modeling can successfully generate
game data of sufficient quality to train a Deep Q-Network well.
  Third, we show that CapsNet is a reliable architecture for Deep Q-Learning
based algorithms for game AI. A capsule is a group of neurons that determine
the presence of objects in the data and is in the literature shown to increase
the robustness of training and predictions while lowering the amount training
data needed. It should, therefore, be ideally suited for game plays.
