  Markov decision processes (MDPs) in queues and networks have been an
interesting topic in many practical areas since the 1960s. This paper provides
a detailed overview on this topic and tracks the evolution of many basic
results. Also, this paper summarizes several interesting directions in the
future research. We hope that this overview can shed light to MDPs in queues
and networks, and also to their extensive applications in various practical
areas.
