  Deep reinforcement learning has been successful in a variety of tasks, such
as game playing and robotic manipulation. However, attempting to learn
\textit{tabula rasa} disregards the logical structure of many domains as well
as the wealth of readily available knowledge from domain experts that could
help "warm start" the learning process. We present a novel reinforcement
learning technique that allows for intelligent initialization of a neural
network weights and architecture. Our approach permits the encoding domain
knowledge directly into a neural decision tree, and improves upon that
knowledge with policy gradient updates. We empirically validate our approach on
two OpenAI Gym tasks and two modified StarCraft 2 tasks, showing that our novel
architecture outperforms multilayer-perceptron and recurrent architectures. Our
knowledge-based framework finds superior policies compared to imitation
learning-based and prior knowledge-based approaches. Importantly, we
demonstrate that our approach can be used by untrained humans to initially
provide >80% increase in expected reward relative to baselines prior to
training (p < 0.001), which results in a >60% increase in expected reward after
policy optimization (p = 0.011).
