  Taking advantage of both vehicle-to-everything (V2X) communication and
automated driving technology, connected and automated vehicles are quickly
becoming one of the transformative solutions to many transportation problems.
However, in a mixed traffic environment at signalized intersections, it is
still a challenging task to improve overall throughput and energy efficiency
considering the complexity and uncertainty in the traffic system. In this
study, we proposed a hybrid reinforcement learning (HRL) framework which
combines the rule-based strategy and the deep reinforcement learning (deep RL)
to support connected eco-driving at signalized intersections in mixed traffic.
Vision-perceptive methods are integrated with vehicle-to-infrastructure (V2I)
communications to achieve higher mobility and energy efficiency in mixed
connected traffic. The HRL framework has three components: a rule-based driving
manager that operates the collaboration between the rule-based policies and the
RL policy; a multi-stream neural network that extracts the hidden features of
vision and V2I information; and a deep RL-based policy network that generate
both longitudinal and lateral eco-driving actions. In order to evaluate our
approach, we developed a Unity-based simulator and designed a mixed-traffic
intersection scenario. Moreover, several baselines were implemented to compare
with our new design, and numerical experiments were conducted to test the
performance of the HRL model. The experiments show that our HRL method can
reduce energy consumption by 12.70% and save 11.75% travel time when compared
with a state-of-the-art model-based Eco-Driving approach.
