  The field of Reinforcement Learning (RL) is concerned with algorithms for
learning optimal policies in unknown stochastic environments. Programmatic RL
studies representations of policies as programs, meaning involving higher order
constructs such as control loops. Despite attracting a lot of attention at the
intersection of the machine learning and formal methods communities, very
little is known on the theoretical front about programmatic RL: what are good
classes of programmatic policies? How large are optimal programmatic policies?
How can we learn them? The goal of this paper is to give first answers to these
questions, initiating a theoretical study of programmatic RL.
