  Intelligent Transportation Systems (ITSs) are envisioned to play a critical
role in improving traffic flow and reducing congestion, which is a pervasive
issue impacting urban areas around the globe. Rapidly advancing vehicular
communication and edge cloud computation technologies provide key enablers for
smart traffic management. However, operating viable real-time actuation
mechanisms on a practically relevant scale involves formidable challenges,
e.g., policy iteration and conventional Reinforcement Learning (RL) techniques
suffer from poor scalability due to state space explosion. Motivated by these
issues, we explore the potential for Deep Q-Networks (DQN) to optimize traffic
light control policies. As an initial benchmark, we establish that the DQN
algorithms yield the "thresholding" policy in a single-intersection. Next, we
examine the scalability properties of DQN algorithms and their performance in a
linear network topology with several intersections along a main artery. We
demonstrate that DQN algorithms produce intelligent behavior, such as the
emergence of "greenwave" patterns, reflecting their ability to learn favorable
traffic light actuations.
