  In this study, we introduce the DriveEnv-NeRF framework, which leverages
Neural Radiance Fields (NeRF) to enable the validation and faithful forecasting
of the efficacy of autonomous driving agents in a targeted real-world scene.
Standard simulator-based rendering often fails to accurately reflect real-world
performance due to the sim-to-real gap, which represents the disparity between
virtual simulations and real-world conditions. To mitigate this gap, we propose
a workflow for building a high-fidelity simulation environment of the targeted
real-world scene using NeRF. This approach is capable of rendering realistic
images from novel viewpoints and constructing 3D meshes for emulating
collisions. The validation of these capabilities through the comparison of
success rates in both simulated and real environments demonstrates the benefits
of using DriveEnv-NeRF as a real-world performance indicator. Furthermore, the
DriveEnv-NeRF framework can serve as a training environment for autonomous
driving agents under various lighting conditions. This approach enhances the
robustness of the agents and reduces performance degradation when deployed to
the target real scene, compared to agents fully trained using the standard
simulator rendering pipeline.
