  Reinforcement learning has been applied in operation research and has shown
promise in solving large combinatorial optimization problems. However, existing
works focus on developing neural network architectures for certain problems.
These works lack the flexibility to incorporate recent advances in
reinforcement learning, as well as the flexibility of customizing model
architectures for operation research problems. In this work, we analyze the
end-to-end autoregressive models for vehicle routing problems and show that
these models can benefit from the recent advances in reinforcement learning
with a careful re-implementation of the model architecture. In particular, we
re-implemented the Attention Model and trained it with Proximal Policy
Optimization (PPO) in CleanRL, showing at least 8 times speed up in training
time. We hereby introduce RLOR, a flexible framework for Deep Reinforcement
Learning for Operation Research. We believe that a flexible framework is key to
developing deep reinforcement learning models for operation research problems.
The code of our work is publicly available at https://github.com/cpwan/RLOR.
