unknownuser6666/R2SE_model

🌟 Stay up to date at opendrivelab.com!

R2SE: Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Driving

• Haochen Liu, Tianyu Li, Haohan Yang, Li Chen, Caojun Wang, Ke Guo, Haochen Tian, Hongchen Li, Hongyang Li and Chen Lv
• Paper

• If you have any questions, please feel free to contact: Haochen Liu ( [email protected] )

---

[2025-11] Initial project released.

Overview

R2SE is a RL finetuning framework targeting hard cases and forgetting in End-to-end Autonmous Driving;

Get Started

Data Preparation

We follow the PARA-Drive pipeline in formating train/test data.

python scripts/data_converter/nuplan/e2e_nuplan_openscenes_navsim_multiprocessing.py

R2SE-GP Training

Following the pipeline in UniAD using specific configs in config/paradrive

1. Pretrain the perception / backbone:

configs/paradrive/exp_100pct/e2e_r2se_gp_perception.py

2. Pretrain the full R2SE-GP:

configs/paradrive/exp_100pct/e2e_r2se_gp.py

R2SE RL Finetuning

1. Hard case sampling: User may rerun the navsim inference and sort for hard case tokens, we provide some preprocessed ones in /data_yaml for your convenience.

2. RL Finetuning:

configs/paradrive/exp_100pct/e2e_r2se.py

After finetuning, run the testing script for both R2SE-GP and R2SE, you are expected to get two .pkl for saved token and planning traj.

3. OOD Inference:

Load .pkl from ```R2SE``. Run this script to get the switched planning results

python ood_inference.py

Testing

run the script in navsim, ensure you load all the cached data and inference result of R2SE-GP in .csv format for PDMS and Forgetting metric calculations.

Weights

All weights are provided in unknownuser6666/R2SE_weights

TODO List

• Initial release
• [] Code Reorganization

Citation

If you find the project helpful for your research, please consider citing our paper:

@article{liu2025reinforced,
  title={Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Driving},
  author={Liu, Haochen and Li, Tianyu and Yang, Haohan and Chen, Li and Wang, Caojun and Guo, Ke and Tian, Haochen and Li, Hongchen and Li, Hongyang and Lv, Chen},
  journal={arXiv preprint arXiv:2506.09800},
  year={2025}
}