  Delays are inherent to most dynamical systems. Besides shifting the process
in time, they can significantly affect their performance. For this reason, it
is usually valuable to study the delay and account for it. Because they are
dynamical systems, it is of no surprise that sequential decision-making
problems such as Markov decision processes (MDP) can also be affected by
delays. These processes are the foundational framework of reinforcement
learning (RL), a paradigm whose goal is to create artificial agents capable of
learning to maximise their utility by interacting with their environment.
  RL has achieved strong, sometimes astonishing, empirical results, but delays
are seldom explicitly accounted for. The understanding of the impact of delay
on the MDP is limited. In this dissertation, we propose to study the delay in
the agent's observation of the state of the environment or in the execution of
the agent's actions. We will repeatedly change our point of view on the problem
to reveal some of its structure and peculiarities. A wide spectrum of delays
will be considered, and potential solutions will be presented. This
dissertation also aims to draw links between celebrated frameworks of the RL
literature and the one of delays.
