  This paper introduces a library for cross-simulator comparison of
reinforcement learning models in traffic signal control tasks. This library is
developed to implement recent state-of-the-art reinforcement learning models
with extensible interfaces and unified cross-simulator evaluation metrics. It
supports commonly-used simulators in traffic signal control tasks, including
Simulation of Urban MObility(SUMO) and CityFlow, and multiple benchmark
datasets for fair comparisons. We conducted experiments to validate our
implementation of the models and to calibrate the simulators so that the
experiments from one simulator could be referential to the other. Based on the
validated models and calibrated environments, this paper compares and reports
the performance of current state-of-the-art RL algorithms across different
datasets and simulators. This is the first time that these methods have been
compared fairly under the same datasets with different simulators.
