  We consider a mixed autonomy scenario where the traffic intersection
controller decides whether the traffic light will be green or red at each lane
for multiple traffic-light blocks. The objective of the traffic intersection
controller is to minimize the queue length at each lane and maximize the
outflow of vehicles over each block. We consider that the traffic intersection
controller informs the autonomous vehicle (AV) whether the traffic light will
be green or red for the future traffic-light block. Thus, the AV can adapt its
dynamics by solving an optimal control problem. We model the decision process
of the traffic intersection controller as a deterministic delay Markov decision
process owing to the delayed action by the traffic controller. We propose
Reinforcement-learning based algorithm to obtain the optimal policy. We show -
empirically - that our algorithm converges and reduces the energy costs of AVs
drastically as the traffic controller communicates with the AVs.
