  Deep reinforcement learning is actively used for training autonomous car
policies in a simulated driving environment. Due to the large availability of
various reinforcement learning algorithms and the lack of their systematic
comparison across different driving scenarios, we are unsure of which ones are
more effective for training autonomous car software in single-agent as well as
multi-agent driving environments. A benchmarking framework for the comparison
of deep reinforcement learning in a vision-based autonomous driving will open
up the possibilities for training better autonomous car driving policies. To
address these challenges, we provide an open and reusable benchmarking
framework for systematic evaluation and comparative analysis of deep
reinforcement learning algorithms for autonomous driving in a single- and
multi-agent environment. Using the framework, we perform a comparative study of
discrete and continuous action space deep reinforcement learning algorithms. We
also propose a comprehensive multi-objective reward function designed for the
evaluation of deep reinforcement learning-based autonomous driving agents. We
run the experiments in a vision-only high-fidelity urban driving simulated
environments. The results indicate that only some of the deep reinforcement
learning algorithms perform consistently better across single and multi-agent
scenarios when trained in various multi-agent-only environment settings. For
example, A3C- and TD3-based autonomous cars perform comparatively better in
terms of more robust actions and minimal driving errors in both single and
multi-agent scenarios. We conclude that different deep reinforcement learning
algorithms exhibit different driving and testing performance in different
scenarios, which underlines the need for their systematic comparative analysis.
The benchmarking framework proposed in this paper facilitates such a
comparison.
